Do you test new modules before uploading them to your live site?

Chris Charlton's picture
I test every module I download before installing it on my live site(s).
63% (78 votes)
I have tested some modules before deploying.
26% (32 votes)
I have not tested any modules before uploading them to my site.
3% (4 votes)
I have not tested any modules, but really should.
6% (7 votes)
I code every module I use and never, ever need to test my own code.
2% (3 votes)
Total votes: 124
Login to post comments

To not test modules is

bonobo's picture
bonobo - Fri, 2009-05-22 00:27

To not test modules is madness! Down that road lies peril!

FunnyMonkey
Click. Connect. Learn.
Using Drupal in Education


never and always are big

mike stewart's picture
mike stewart - Sat, 2009-05-23 02:02

never and always are big words. Ok, always was only implied, and while I agree with Bill in theory, I'm a bit more pragmatic in practice.

it kinda depends on what you mean by "test" - sometimes a module author's reputation or community feedback at a user group meetup / IRC is enough for me. for example, I did virtually "no testing" of CCK module before the first time I deployed it. granted, in the process of building content types also came a better understanding of the module. But I performed no regression testing.

also, I recently deployed the wysiwyg module with very little testing. It was on a small site, low traffic, in this case I felt the benefit outweighed the risk. plus I knew that wysiwyg wasn't sun's first module...

one of the biggest values I get from being involved at user group meetups is the feedback on contrib module reliability, etc..


uploading modules to live

asifnoor's picture
asifnoor - Fri, 2009-05-22 05:24

uploading modules to live site without testing is inviting problems home :)
Asif
Azri Solutions (http://www.azrisolutions.com/)


I do, but not manually. Unit

Alexandr Kosarev-gdo - Fri, 2009-05-22 08:41

I do, but not manually. Unit tests do it for me.


Do you burn in module before you develop custom code around it?

jwlehane's picture
jwlehane - Sat, 2009-05-23 01:05

I am toying with the idea of burning in modules before I build custom code around them. What I mean is, take a contrib module, plop it in my test environment, enable it, but make no use of the functionality. Do some basic regression (or test other new changes and thereby test the module). Then push it up to production.

I should be able to see if the module has any memory leaks or other big problems before I both writing custom code to integrate this module into my site. does anyone else do this, and does it make sense to you?

Johnny


...

sun.core's picture
sun.core - Sat, 2009-05-23 01:41

Fully functional tests on dev/staging/live sites: http://drupal.org/project/demo

Daniel F. Kudwien
unleashed mind


Yes - always test first. If

intoxination - Sat, 2009-05-23 03:02

Yes - always test first. If the module does database queries I also check to make sure they aren't doing anything too nasty (ie: filesorts, temp tables, no index, etc.)


This is kind of relative

marvil07's picture
marvil07 - Sat, 2009-05-23 16:37

This is kind of relative :D

I'd answer this as on big it depends(All sites does not have the same priorities).

So for example, if you really need to build a site for collaborating, hear that og can do it, and it would live not so many time(this is relative too) you do not want to test all code you need, because it is time consuming. But if you want to build a decent middle size, kind of longer life site, you really want to test all you can.

Well, I think answers may differ a lot because we all do not have the same idea of testing.
Some approaches to do that:

  • just try it and verify it does not break the site(maybe usually in non-tech drupal end users)
  • run simpletest tests
  • reads the code and verify if it's enough efficient
  • try to bring it down to prove security

.. it could be a really big list, but, I think the idea is clear.


It's about trust

highermath's picture
highermath - Mon, 2009-05-25 17:22

If you are working on your personal site, and can tolerate a catastrophic failiure (i.e., have a good, current backup) then perhaps you can live without testing.

If you are doing work for clients, it could be considered negligent to not test in a staging environment before deploying new and/or updated code.


Too pessimistic options in poll?

markus_petrux's picture
markus_petrux - Mon, 2009-05-25 18:22

Probably, there's people that do not test, but I think most of us do, so the question would be how far you go when testing modules before deploying them on production?

For example, I do not only test, but also read the code, and look at the issues queue, the project stats, the background of the module author, etc. There maybe things here that tell me it's better to find an alternative to a certain module. It depends on how critical it is for the project I'm working on too.

Using this method I've found bugs several times, and depending of several factors, I may opt to report them, or even try to fix them, or just forget and try alternative modules, etc.


I never maintain a website

ceardach's picture
ceardach - Mon, 2009-05-25 22:47

I never maintain a website without separate production and development branches. No one should ever get into the habit of deploying untested changes to a production site. Its these bad habits that cause huge headaches and are hard to break on a conceptual level when you're trying to grow and expand.

So yes. I always test. I make my changes in a development environment and when things seem to be working fine, I'll then merge with production.


For me, the answer is easy: YES!

Miguel-gdo's picture
Miguel-gdo - Tue, 2009-05-26 19:21

Being an old QA guy in various dev & webdev environments, my background is testing. I actually find just as much pleasure in testing out a new module as I do in the moment that I fully deploy to production. Both are fun & rewarding. While unit testing gets the job done lots of the time, I like getting down & dirty with manual testing. I usually feel confident in my breadth of testing & end up learning a thing or 3 about the module to boot!


I have a background in QA,

ceardach's picture
ceardach - Wed, 2009-05-27 14:29

I have a background in QA, too, and I also believe in manual testing.

Automated testing can only capture bugs you can predict, while manual testing (although also limited to your prediction capabilities...) can more easily uncover unpredictable bugs.


trick question, right?

As If's picture
As If - Wed, 2009-05-27 15:03

I actually find it hard to believe this isn't a joke question. Unless/until you have deep familiarity with a particular module, installing it on a production site without IN-CONTEXT system testing is practically suicidal - especially if you are dealing with Drupal 6. (sorry, but still true!) I say "in-context" because when you run into serious conflicts, they will usually be between 2 contributed modules. The only way to do proper system testing is to use a dev/staging server with a similar setup to the production machine. If the client does not have a staging server, I use one of my own.


No, this is not a trick

ceardach's picture
ceardach - Wed, 2009-05-27 16:37

No, this is not a trick question. However, it is preaching to the choir.

I have been attending a lot of meetups lately, and I speak mostly on version control and deployment practices. It is a completely foreign concept to even have a separate development and production environments. I have been literally asked, "How do you merge development with production when you've made some module upgrades and configuration changes in production?"

It is a completely new concept to many people. I also find it very hard to break them out of that mindset.


To be fair, it gets tricky

eli's picture
eli - Thu, 2009-06-04 03:23

To be fair, it gets tricky when you've got to roll out complex changes to things that exist only in the database. To my knowledge, there's no easy way to put a set of changes to e.g. the block layout into version control.


Staging and Deployment - A

ceardach's picture
ceardach - Thu, 2009-06-04 04:44

Staging and Deployment - A Panel Discussion

  • Option #1 - Moving admin to code
  • Option #2 - DBscripts module
  • Option #3 - Deploy module

Panel Discussion

droplits's picture
droplits - Thu, 2009-06-04 13:12

Thanks for that link. That panel discussion was on my short list to watch. Unfortunately the video stops in the middle of your discussion Kathleen. I was curious if you have posted the slides you used during the discussion anywhere?

Your post Development Environment for Drupal was also very helpful when I was in the process of separating my workspaces (which is essential to the idea of testing modules). Also Matt Petrosky's post's Quick Tip: SVN Filtering, and Overview: Drupal CVS/SVN Setup where helpful.


No posting yet :(

ceardach's picture
ceardach - Thu, 2009-06-04 16:22

No, I haven't posted it. But I should. Also, the slides themselves are very sparse because they were meant to be complimentary to the talk. I started writing up everything I was talking about, but haven't finished it yet.

I just found out last night that the videographer walked out. Wow! How many other times did that happen?


Video

heyrocker - Thu, 2009-06-04 16:55

As far as I know, our talk was the only one where the video guy left early. I have pinged the conference organizers several times to see if the rest of the video exists somewhere, and never heard anything back, so I assume it is lost for good.


Just our luck, huh?

ceardach's picture
ceardach - Fri, 2009-06-05 13:22

Just our luck, huh?

And... well... everyone else, too! That was a valuable presentation. I am constantly referring to it as a good introductory to the various deployment solutions. Little did I know the video wasn't complete. heh.


For example

highermath's picture
highermath - Wed, 2009-05-27 17:01

We were testing a module the other day that had an uninstall routine so thorough that it pretty much uninstalled our dev server. Not good, embarrassing, but not too painful either -- svn up and some config restores from backup fixed it. Certainly would have been nasty on a production site.


I almost always test during

droplits's picture
droplits - Wed, 2009-06-03 20:15

I almost always test during development on a dev/staging server which is sometimes a test domain on the same dedicated server that I end up putting the client site on. But sometimes I just go for it and place a contrib module on a live site and hardly check anything, but that is usually a module I've reused over and over and know fairly well. Plus a little version control goes a long way.


Oh, yes

eli's picture
eli - Thu, 2009-06-04 03:19

We've got a dev server to screw around on, and a QA server that is designed to be very similar to production. All code gets tests on the QA server before going live. (In fact, because of the way the deployment scripts are set up you pretty much have to deploy to the QA server first.) Some day I'll get around to setting up an automatic run of simpletest.


*_*

beautifulmind's picture
beautifulmind - Mon, 2009-06-08 13:48

I always set up a local development environment and I create a clone and make it live or you can say the live site is the clone of my local pack.
If some thing goes wrong, the live site will not have any effect since I test it on my local pack.

:)
Beautifulmind


Maybe I'm a bit mad but I

oskar_calvo's picture
oskar_calvo - Tue, 2009-06-16 15:42

Maybe I'm a bit mad but I work with a lab where I tested everything I need.
Them I move to pre-production, where I check that It works fine with the customers web
And then I go to customers website.

It's a little slow, but I have never a problem with a customers website.

oskar