Do you test new modules before uploading them to your live site?

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Chris Charlton's picture
I test every module I download before installing it on my live site(s).
60% (92 votes)
I have tested some modules before deploying.
25% (39 votes)
I have not tested any modules before uploading them to my site.
5% (7 votes)
I have not tested any modules, but really should.
8% (12 votes)
I code every module I use and never, ever need to test my own code.
3% (4 votes)
Total votes: 154

Comments

To not test modules is

bonobo's picture

To not test modules is madness! Down that road lies peril!


FunnyMonkey
Click. Connect. Learn.
Using Drupal in Education

never and always are big

mike stewart's picture

never and always are big words. Ok, always was only implied, and while I agree with Bill in theory, I'm a bit more pragmatic in practice.

it kinda depends on what you mean by "test" - sometimes a module author's reputation or community feedback at a user group meetup / IRC is enough for me. for example, I did virtually "no testing" of CCK module before the first time I deployed it. granted, in the process of building content types also came a better understanding of the module. But I performed no regression testing.

also, I recently deployed the wysiwyg module with very little testing. It was on a small site, low traffic, in this case I felt the benefit outweighed the risk. plus I knew that wysiwyg wasn't sun's first module...

one of the biggest values I get from being involved at user group meetups is the feedback on contrib module reliability, etc..

--
mike stewart { twitter: @MediaDoneRight | IRC nick: mike stewart }

uploading modules to live

asifnoor's picture

uploading modules to live site without testing is inviting problems home :)
Asif
Azri Solutions (http://www.azrisolutions.com/)

I do, but not manually. Unit

Alexandr Kosarev-gdo's picture

I do, but not manually. Unit tests do it for me.

johnnykickball's picture

I am toying with the idea of burning in modules before I build custom code around them. What I mean is, take a contrib module, plop it in my test environment, enable it, but make no use of the functionality. Do some basic regression (or test other new changes and thereby test the module). Then push it up to production.

I should be able to see if the module has any memory leaks or other big problems before I both writing custom code to integrate this module into my site. does anyone else do this, and does it make sense to you?

Johnny

...

sun's picture

Fully functional tests on dev/staging/live sites: http://drupal.org/project/demo

Daniel F. Kudwien
unleashed mind

Daniel F. Kudwien
netzstrategen

Yes - always test first. If

Jamie Holly's picture

Yes - always test first. If the module does database queries I also check to make sure they aren't doing anything too nasty (ie: filesorts, temp tables, no index, etc.)


HollyIT - Grab the Netbeans Drupal Development Tool at GitHub.

This is kind of relative

marvil07's picture

This is kind of relative :D

I'd answer this as on big it depends(All sites does not have the same priorities).

So for example, if you really need to build a site for collaborating, hear that og can do it, and it would live not so many time(this is relative too) you do not want to test all code you need, because it is time consuming. But if you want to build a decent middle size, kind of longer life site, you really want to test all you can.

Well, I think answers may differ a lot because we all do not have the same idea of testing.
Some approaches to do that:

  • just try it and verify it does not break the site(maybe usually in non-tech drupal end users)
  • run simpletest tests
  • reads the code and verify if it's enough efficient
  • try to bring it down to prove security

.. it could be a really big list, but, I think the idea is clear.

It's about trust

highermath's picture

If you are working on your personal site, and can tolerate a catastrophic failiure (i.e., have a good, current backup) then perhaps you can live without testing.

If you are doing work for clients, it could be considered negligent to not test in a staging environment before deploying new and/or updated code.

Too pessimistic options in poll?

markus_petrux's picture

Probably, there's people that do not test, but I think most of us do, so the question would be how far you go when testing modules before deploying them on production?

For example, I do not only test, but also read the code, and look at the issues queue, the project stats, the background of the module author, etc. There maybe things here that tell me it's better to find an alternative to a certain module. It depends on how critical it is for the project I'm working on too.

Using this method I've found bugs several times, and depending of several factors, I may opt to report them, or even try to fix them, or just forget and try alternative modules, etc.

I never maintain a website

ceardach's picture

I never maintain a website without separate production and development branches. No one should ever get into the habit of deploying untested changes to a production site. Its these bad habits that cause huge headaches and are hard to break on a conceptual level when you're trying to grow and expand.

So yes. I always test. I make my changes in a development environment and when things seem to be working fine, I'll then merge with production.

For me, the answer is easy: YES!

Miguel-gdo's picture

Being an old QA guy in various dev & webdev environments, my background is testing. I actually find just as much pleasure in testing out a new module as I do in the moment that I fully deploy to production. Both are fun & rewarding. While unit testing gets the job done lots of the time, I like getting down & dirty with manual testing. I usually feel confident in my breadth of testing & end up learning a thing or 3 about the module to boot!

I have a background in QA,

ceardach's picture

I have a background in QA, too, and I also believe in manual testing.

Automated testing can only capture bugs you can predict, while manual testing (although also limited to your prediction capabilities...) can more easily uncover unpredictable bugs.

trick question, right?

As If's picture

I actually find it hard to believe this isn't a joke question. Unless/until you have deep familiarity with a particular module, installing it on a production site without IN-CONTEXT system testing is practically suicidal - especially if you are dealing with Drupal 6. (sorry, but still true!) I say "in-context" because when you run into serious conflicts, they will usually be between 2 contributed modules. The only way to do proper system testing is to use a dev/staging server with a similar setup to the production machine. If the client does not have a staging server, I use one of my own.

No, this is not a trick

ceardach's picture

No, this is not a trick question. However, it is preaching to the choir.

I have been attending a lot of meetups lately, and I speak mostly on version control and deployment practices. It is a completely foreign concept to even have a separate development and production environments. I have been literally asked, "How do you merge development with production when you've made some module upgrades and configuration changes in production?"

It is a completely new concept to many people. I also find it very hard to break them out of that mindset.

To be fair, it gets tricky

eli's picture

To be fair, it gets tricky when you've got to roll out complex changes to things that exist only in the database. To my knowledge, there's no easy way to put a set of changes to e.g. the block layout into version control.

Staging and Deployment - A

ceardach's picture

Staging and Deployment - A Panel Discussion

  • Option #1 - Moving admin to code
  • Option #2 - DBscripts module
  • Option #3 - Deploy module

Panel Discussion

solutiondrop's picture

Thanks for that link. That panel discussion was on my short list to watch. Unfortunately the video stops in the middle of your discussion Kathleen. I was curious if you have posted the slides you used during the discussion anywhere?

Your post Development Environment for Drupal was also very helpful when I was in the process of separating my workspaces (which is essential to the idea of testing modules). Also Matt Petrosky's post's Quick Tip: SVN Filtering, and Overview: Drupal CVS/SVN Setup where helpful.

No posting yet :(

ceardach's picture

No, I haven't posted it. But I should. Also, the slides themselves are very sparse because they were meant to be complimentary to the talk. I started writing up everything I was talking about, but haven't finished it yet.

I just found out last night that the videographer walked out. Wow! How many other times did that happen?

Video

gdd's picture

As far as I know, our talk was the only one where the video guy left early. I have pinged the conference organizers several times to see if the rest of the video exists somewhere, and never heard anything back, so I assume it is lost for good.

Just our luck, huh?

ceardach's picture

Just our luck, huh?

And... well... everyone else, too! That was a valuable presentation. I am constantly referring to it as a good introductory to the various deployment solutions. Little did I know the video wasn't complete. heh.

For example

highermath's picture

We were testing a module the other day that had an uninstall routine so thorough that it pretty much uninstalled our dev server. Not good, embarrassing, but not too painful either -- svn up and some config restores from backup fixed it. Certainly would have been nasty on a production site.

I almost always test during

solutiondrop's picture

I almost always test during development on a dev/staging server which is sometimes a test domain on the same dedicated server that I end up putting the client site on. But sometimes I just go for it and place a contrib module on a live site and hardly check anything, but that is usually a module I've reused over and over and know fairly well. Plus a little version control goes a long way.

Oh, yes

eli's picture

We've got a dev server to screw around on, and a QA server that is designed to be very similar to production. All code gets tests on the QA server before going live. (In fact, because of the way the deployment scripts are set up you pretty much have to deploy to the QA server first.) Some day I'll get around to setting up an automatic run of simpletest.

*_*

beautifulmind's picture

I always set up a local development environment and I create a clone and make it live or you can say the live site is the clone of my local pack.
If some thing goes wrong, the live site will not have any effect since I test it on my local pack.

:)
Beautifulmind

Maybe I'm a bit mad but I

oskar_calvo's picture

Maybe I'm a bit mad but I work with a lab where I tested everything I need.
Them I move to pre-production, where I check that It works fine with the customers web
And then I go to customers website.

It's a little slow, but I have never a problem with a customers website.

oskar

This should be done before release !

EDDYL's picture

What I am surprised off is that this is not done by the "community managers" at a certain stage to evaluate the modules before including them at d.o.

Some of them are real craps that will completly fu** your site and no correction is done by contributors on some major bugs even if issues / patches are months / years old. (i remember bitcache module deleting all my /var content !)

Lots of module work only on a clean core install, once you have hundreds of module interacting (even if they should not), such modules are not working as initialy expected...

Reading the issue queue is also very instructive on contributor involvment...

Well... not testing is Madness !

"L'exces de modération nuit gravement à la consommation"


Materiel et services agrométéorologiques : http://www.promete.fr

.

Michelle's picture

The only answer you're going to get to this is the standard one. If you feel every module should be thoroughly tested by some "community managers" before being added to the repository, including testing it with all possible combinations of the other 7K+ contrib modules, you're welcome to step up and volunteer for the position.

Michelle

Perhaps it would be madness

highermath's picture

Perhaps it would be madness if you were paying $500 for these modules, but in fact, everyone who participates is part of the FOSS process. If you download the foobar module, noting that it is in use on 3 sites, it is on you to test this in a non-production environment (or production, if you don't care), and report any problems or anomalies to the module issue queue.

If a module is being used by, say, 25,000 sites, you should take a look at the issue queue. While it might be safe to install it with a little less rigor, you need to be aware that if you are running some other very lightly used module, the odds are that they haven't been tested together.

If you have been around Drupal long, you have likely seen the term, doacracy. That is what we are, and we would welcome you to get involved and do.