We are pretty ahead with functional tests but unit tests are a completely different problem. The function we want to test calls other functions and so on. While theoretically one can unit test the leaves of the call tree and work from there, saying "drupal_validate_utf8 is already tested and we know it works. So when testing check_plain, I can rely on the fact that drupal_validate_utf8 works as it should so I will just call check_plain with various values and test with assertEqual whether the return value is what I want." However, you need to unit test every code flow and some parts might be incredibly hard to reach. For example, drupal_get_form calls form_get_cache, drupal_prepare_form, drupal_process_form, form_set_cache, drupal_rebuild_form, drupal_render_form. Even if all these called functions (and everything "below") them are unit tested, providing an environment (database, other loaded functions for hooks and various callbacks, files) that will actually result in certain variables having the value we want might be incredibly difficult.
The traditional solution is to create mock objects, in our case, mock functions. Create a version of drupal_prepare_form which returns exactly the value we want for a particular test and move on. This is not easy with PHP. Once the function is loaded, you can't override it unless you are using runkit which is a PHP extension from PECL. I am aware of two solutions besides runkit, both of which requires Drupal being parsed.
- Put every function in a separate file, when you want to use the mock version of a function, load that instead of the original. This is just a bit more difficult than runkit, because you can override only on loadtime not runtime but at least you can override.
- Generate mock functions for every Drupal function, load these into memory. Generate a copy of Drupal functions with an original_ prefix, load these too. Call the original function which will in turn will call the mock versions.
Advantages/disadvantages:
- When you need the original again to be tested, you need a different PHP instance. However, this also allows to change the mock functions from one test to the next. You can even generate them on the fly, either by copying files or just running eval. So you can have a drupal_validate_utf8 which is mocked to a function which returns TRUE and you can have a version which returns FALSE.
- As you need the mock versions to do different (but fixed!) things from one test to the next and unlike the previous ones, these stay in memory, you probably need to call a central function -- which I call the clearinghouse -- which says what this function needs to do.
You basically need to encode some way "I need db_fetch_object to return an object on the first two calls and FALSE on the third". You can call a "createMockFunction" method with the necessary arguments which describe such a thing. Or you can
<?php
array(
'incoming' => 'db_fetch_object',
'return' => $object1,
),
array(
'incoming' => 'db_fetch_object',
'return' => $object2,
),
array(
'incoming' => 'db_fetch_object',
'return' => FALSE,
)
?>The first solution described above better yields to the "createMockFunction" function which in this case would probably load the necessary code from some include, slap the necessary function header on it and eval. The second solution works better with the array as it literally describes what the central function can expect as the test runs. However, the array can be iterated and createMockFunction calls can be issued for the first solution. Or you can build the array as createMockFunction calls arrive and then as the test starts, the central function can pull the array in and act accordingly. createMockFunction is a bit more like what other systems do. The array version is probably a bit more easier to write on the other hand. The difference is not that big.
And of course, you can always say no to mock functions and just test with the assumption that you have the underlying functions tested and set up the surrounding environment (database, loaded functions, files) so that the code flows the way you wanted. This is the worst, IMO. The load time overrides (first method) is quite a bit cumbersome despite it is what other systems do. Those systems, however, are likely using classes which are more easily overrideable. The central clearinghouse seems to be best suited for Drupal even if the tests are strange arrays.
Note that this is not unlike what happened to our unique form API: that also uses arrays, it's totally unique to Drupal but at the end it proved itself to be a very strong tool.
Comments
putting unit tests in old monsters of code
http://www.amazon.com/Working-Effectively-Legacy-Robert-Martin/dp/013117...
My 2 cents..
This stems from my freak-out when I saw http://drupal.org/node/30008 which introduces some Drupal-specific functions for unit testing (and worse, seems to require mocking those for every possible input/output combination?)
My stance on test-driven development in Drupal is the following:
The most important thing about integrating unit testing with core is that the syntax of our unit tests match as closely as possible those in JUnit, SimpleTest, and other testing frameworks.
Yes, you heard me right. More important than 100% test coverage. More important than 10% coverage! More important than ferreting out every single bug. More important than a proper mock object/function layer so that we can do true "clean" unit testing.
Blasphemy! Well, here's my justification:
By matching syntax with existing testing frameworks, developers who already know how to do unit testing will be able to jump into Drupal easily. Zero learning curve, other than the Drupal-specific getInfo() function.
Those who aren't familiar with unit testing frameworks already will have an incentive to figure it out, as we'll be giving them a transferable skill in exchange for their labour.
If we don't catch every possible permutation of drupal_get_form() the first time, so what? We can always add in more tests later if we figure out that, "Oops, we forgot to test NULL as the #submit value for a form." The important thing is never to have the same bug twice, and if we're diligent about writing tests for problems that arise, then so be it.
Right now, we have zero tests. If we end the summer testing only thoroughly testing 25 Drupal functions, then that is better than what we have now. And if we choose carefully what those 25 Drupal functions are, we can get a big bang for our buck. :)
We want people to be able to read a great unit testing tutorial like http://junit.sourceforge.net/doc/testinfected/testing.htm and immediately be able to apply that knowledge to Drupal.
Here's a JUnit test:
public class SpreadsheetTest extends TestCase {
public void testCellReference() {
Spreadsheet sheet = new Spreadsheet();
sheet.put("A1", "5");
sheet.put("A2", "=A1");
assertEquals("5", sheet.get("A2"));
}
public void testCellChangePropagates() {
Spreadsheet sheet = new Spreadsheet();
sheet.put("A1", "5");
sheet.put("A2", "=A1");
sheet.put("A1", "10");
assertEquals("10", sheet.get("A2"));
}
public void testFormulaCalculation() {
Spreadsheet sheet = new Spreadsheet();
sheet.put("A1", "5");
sheet.put("A2", "2");
sheet.put("B1", "=A1*(A1-A2)+A2/3");
assertEquals("15", sheet.get("B1"));
}
}
Here's a SimpleTest test:
class TestOfLogging extends UnitTestCase {function TestOfLogging() {
$this->UnitTestCase('Log class test');
}
function testCreatingNewFile() {
@unlink('../temp/test.log');
$log = new Log('../temp/test.log');
$this->assertFalse(file_exists('../temp/test.log'), 'No file created before first message');
$log->message('Should write this to a file');
$this->assertTrue(file_exists('../temp/test.log'), 'File created');
}
}
http://drupal.org/node/30008 looks absolutely nothing like these. That's BAD. That means the barrier to learning Drupal's testing framework is raised, which makes people resistant to learning it, especially since once learned, people can't transfer their knowledge. Developer uptake is fundamental to Drupal's success with unit testing, especially if Dries is going to start throwing down the hammer about tests being written.
And finally, I also don't really see this whole mock object thing as being something we even need to worry about:
http://api.drupal.org/api/function/drupal_validate_utf8
This function calls the PHP functions strlen() and preg_match(). However, it would be absurd for us to create mock functions for strlen() and preg_match(), right? Those functions are "black boxes" and we assume that these already "just work."
So why can't we treat drupal_validate_utf8() as a black box when we're testing check_plain(), if drupal_validate_utf8() has been properly unit tested?
I dunno. Am I on crack? Could you or Douglas or someone chime in here?
Three concerns
There are a handful of underlying assumptions that seem to be present here that I see as very damaging.
All three of these are seriously, fundamentally flawed assumptions that will lead our project into a quagmire unless they're dealt with effectively before more energy is thrown into D7 tests.
This is, unfortunately for purists, the way the world works. This is why unit tests are called "unit tests" rather than "function call tests." Unit Testing is about testing functional units of a program, not necessarily specific functions. There is nothing wrong with a test that is written against, say, check_plain() that walks through specific behavior and ensures that a specific result is returned. You say in your post that this is 'roblematic because check_plain calls other functions, but that's a silly objection. You have still tested check_plain(), and if it fails a test, it fails a test. As time goes on it may be useful to generate more granular tests for the individual sub-functions, but there is no moral requirement that we test down to 'bare metal'. Even XP, with its emphasis on testing everything, stresses the importance of testing things that are likely to fail rather than thrashing to create tests for every random operation.
You're presenting a straw man, here. If I write my test for check_plain(), I am not 'assuming' that every sub-function has been explicitly tested. I am simply writing a test for check_plain(), a worthwhile endeavor in and of itself. This brings us to the second point -- the assumption that 100% coverage is the only acceptable goal.
We live in a world with finite developer resources, and 100% coverage of every function, every input, and every output in Drupal is a pipe dream. Even more importantly, there is no reason for it to happen, save an OCD drive to mark off lots and lots of checkboxes. Setting that goal, rather than identifying high-priority APIs and functions that should get comprehensive coverage first, is a recipe for thrashing. One can say that this will result in incomplete coverage, and this is true. This is not a problem, because no automated testing system is capable of complete coverage. We should aim for the biggest wins possible, and refine the focus as time goes on and new testable defects are discovered.
As Larry Garfield noted in his blog post earlier this week, adoption of good testing practices means, in many cases, refactoring your code so that it is testable. There are many, many areas in Drupal where transient context drives behavior, and it is those cases that make writing tests for those units of code difficult. Removing globals from formAPI is one such difficult refactoring. While formAPI is still far from 'easily testable,' those kinds of changes are critical and should absolutely come before we pour energy into testing every discrete function in Drupal's namespace.
(As a side note, on a purely technical basis, the PECL-based runkit approach, where functions are dynamically redefined and mock versions swapped in, is IMO vastly preferable to a system that requires reams of new meta-data to be defined for every single test. There is a nontrivial risk of building a system complex enough that IT needs as much testing as the Drupal code it is designed to automate the testing of. The major objection to runkit based approaches seems to be that those without PECL installed will be unable to run unit tests. Personally, I have no problem with that -- a reliable method of mocking functions (what the runkit system provides) is more important than the theoretical benefits of having everyone on the planet running Drupal's unit tests. For developers who have no access to PECL, the 'patch testing server' we rolled out previously is a possible solution.)
Finally, the most dangerous assumption that I see in the current wave of enthusiasm for unit testing is that this project will mean that more Drupal development can occur. The idea is repeated quite often -- Dries' commitment to extending 'open season' for D7 patches if we have comprehensive unit tests in place is part of the issue. What is overlooked is important: on average, 5 lines of test are required for every 1 line of functional code. Tests do not come for free. They take as much time to write an maintain as the code they exercise -- often far more time, as testing many esoteric edge cases requires a tremendous amount of setup work and careful validation. Like debugging, writing effective unit tests often requires more knowledge of the units in question than writing them in the first place.
The benefits of unit testing are twofold. First, it helps improve the reliability of code when well-defined APIs are internally refactored. Second, when tests are written to trigger specific reproducible defects, they can ensure that 'fixed' bugs to not unexpectedly reappear. (Checkboxes, cough cough). But those two goals come at the cost of time. We 'spend' time to improve reliability. That is the only realistic way to look at this endeavor.
Runkit is not Satan
Just chiming in to say I don't think runkit is that much of a barrier. It comes with XAMPP, for instance; to install it, you uncomment one line in php.ini. Anyone who's motivated enough to run unit tests can surely either install XAMPP and uncomment one line, or figure out PECL. And how many people will need to run them in the first place? I come from the Rails world, and I don't think I ran the core tests once... but I was very glad they were there.
I'm not saying runkit is the right solution. That's above my pay grade. I'm just saying it's a low enough barrier that there's no need to route around it. If I can get it running, anybody can.
The problem with runkit...
Is that currently, in order to contribute to Drupal core, you need:
That's it. Of course, it's much easier for you if you also have CVS so you can have an up-to-the-minute checkout, but you can always diff against dev tarballs. Patch is also a plus, but only needed if you want to test other peoples' patches.
If we require runkit in order to verify that the tests you contribute with your patches are working, now suddenly you need to have AMP+PECL+runkit. And under normal circumstances, that means recompiling PHP, which is a HUGE barrier of entry. :(
That's cool that XAMPP provides this .so by default under Windows. It doesn't in the Mac version. And MAMP doesn't either. There are A LOT of Drupal devs on Macs, so this is going to be a problem.
Can you suggest a place to document...
...the installation steps for installing runkit on various platforms?
Maybe there's something we can do to fix the issues with MAMP, php world at-large would be grateful I'm sure.
I would say as a sub-page of Automated testing...
Here: http://drupal.org/simpletest
Maybe something like "Before you start: installing runkit"?
this is why we have handbook
this is why we have handbook pages and screencasts. and folks can share pre-built binaries for various flavors of OS and so on.
Hm
Hm. That does put a damper on things. I spent an hour plus this morning trying to get runkit working under MAMP and no joy. I do think this is something we could solve either by contributing back to XAMPP or by making runkit.so available in whatever flavors people need it in. That is, of course, if runkit turns out to be part of the unit testing solution.
There are several points
There are several points to this discussion that are being overlooked.
______
______
______
______
______
This is how Drupal looks at the moment:
__|_|__|_|__
__|_|_||_|__
__|___||_||_
__|___||_||_
__|____|__|_
__|_______|_
The encapsulation is terrible.
$this->assertNull($result, 'Valid mail');looks exactly the same asassertEquals("15", sheet.get("B1"));for any self-respecting coder. There is no point in forcing Drupal to work with unit tests that weren't written with it in mind.If so, why is it? And also, what will having tests on people's websites achieve?
If not, why do we need the tests on people's websites, anyways?
I think there's some confusion...
Re #3: My post above was in regards to http://drupal.org/node/30008/revisions/286438/view, which was since reverted based on a discussion in #drupal yesterday. My point was that what's there looks absolutely nothing like assertX(something, something), and thus no knowledge is transferable between the two. Not to mention other problems, like the requirement to read up on all of the functions that is called by a particular function in order to test it, etc.
Re #5: During the Drupal 6 develpment cycle, there were 800 people who submitted patches. If trends continue, there should be at least 1200 who do during the Drupal 7 cycle. All of these people need to be able to run and write unit tests, because their patches won't get committed without them. And it's imperative that we make it as easy for them to do so as possible, so that we keep up our momentum.
It's a separate discussion entirely as to whether or not these tests hang around in the "shippable" version of Drupal 7 or not. And please help improve the docs if you see specific areas that are lacking. If you came up to speed on PHP in two weeks, your input into where you struggled (especially if you can share what you know now in the form of handbook pages, API doc corrections, etc.) could have a tremendous impact.
While it is useful to
While it is useful to discuss this, this isn't necessarily a decision that needs to be made right now. No need to rush things. Let's give ourselves the time to get used to SimpleTest, and let's keep all options open with regard to testing. If it is difficult or tedious to write certain SimpleTests, alternative solutions will emerge. If that happens to be a runkit based unit test approach, that might be entirely acceptable.
Simply put, there is no need to rule out certain tools or approaches at this point in time. What happens, happens so let things evolve and improve over time. And let's give people the flexibility they want.
I guess this is where I struggle...
Back at the testing BoF in Barcelona, I stepped up to "own" the part of getting the community ramped up and excited about doing unit testing. I've written articles, I've tried to lead by example, I've submitted and provided feedback on numerous improvements to SimpleTest module to make it more intuitive for new developers to learn.
At the workshops we teach, I get students asking me how to get started with testing. I have Summer of Code students who are going to be required to submit tests with their projects. I have a constant stream of new people asking me "What's the best way for a new developer to get involved in Drupal?" I want to help all of these people tear into improving test coverage. :)
But the biggest barrier for me to move forward on this is that everything still seems in flux. I thought that the testing sprint would lend clarity to this, and would give us "The Plan" that I could then execute on. But instead, it still seems like we're in this state of "eh, no biggie. we'll figure it out as we go." Unfortunately, that paralyzes me in my quest to help newbies sink their teeth into this. :( I'm forced to answer these people with, "Eh. Best to wait a couple of more weeks."
Newbies need:
Until these things are in place, I can't do my job as community wrangler. :( And we are running short on time; July is just around the corner.
I hear you webchick. We've
I hear you webchick. We've worked hard at the code sprint, and I think we made a tremendous step forward. We've now standardized on SimpleTest for functional testing. In other words, people can start writing SimpleTests today and you can test almost everything with SimpleTests. It just got a lot easier to execute on that, so I wouldn't say that "everything" is still in flux. SimpleTests are a fact, things are more stable and predictable than last week.
I realize that we have to work on documenation and support. Hopefully these things will come together soon. I'll help make that possible.
Ok cool...
That's at least a HUGE step in the right direction. :)
I'll work with chx/dhubler/others to try and sort out the conventions stuff; the rest can follow.
start out with standard tools, replace as nec.
I would underscore much of what eaton mentioned. I also symapthize with webchick, to that end I would have loved to have come away from the testing sprint on-board with chx's plan, but in order for me to "get there" I need to validate to myself existing tools are ineffective.
I put together a copy of drupal HEAD and integrated runkit, simpletest and mock-functions and wrote a unit test for includes/cache.inc. I also put together a page to describe my efforts.
http://dhubler.googlepages.com/drupalunittestingusingstandardtools
Pardon my ignorance, but...
Why couldn't cache.inc.test have been written like this? (totally untested)
<?php
// $Id$
class CacheTest extends UnitTestCase {
function test_cache_get() {
// Test retrieving with default options.
$cache = new DummyCache;
$cache->cid = 'simpletest_test_get_cache_default';
$cache->data = 'simpletest_test_get_cache_default';
cache_set($cache->cid, $cache->data);
$this->assertEquals($cache->data, cache_get($cache->cid), t('cache_get(), default options.'));
// Test retrieving from a specific table.
$cache = new DummyCache;
$cache->cid = 'simpletest_test_get_cache_table';
$cache->data = 'simpletest_test_get_cache_table';
$cache->table = 'cache_page';
cache_set($cache->cid, $cache->data);
$this->assertEquals($cache->data, cache_get($cache->cid, 'cache_page'), t('cache_get(), specific table.'));
// Test retrieving temporary value.
variable_set('cache_flush', 1);
$cache = new DummyCache;
$cache->cid = 'simpletest_test_get_cache_temporary_expired';
$cache->data = 'simpletest_test_get_cache_temporary_expired';
cache_set($cache->cid, $cache->data, 'cache', CACHE_TEMPORARY);
$this->assertNull(cache_get($cache->cid), t('cache_get(), temporary expired value'));
// Etc.
}
/** I populated this based on the defaults of cache_set() **/
class DummyCache {
var $cid = 'simpletest_cache_inc_test';
var $data = '[cache data]';
var $table = 'cache';
var $expire = CACHE_PERMANENT;
var $headers = NULL;
}
}
?>
Big advantages:
1. It took me about 2 minutes to write that.
2. I'm using the same function calls in my tests as I do in the code I write. As long as I've /used/ cache_get() at some point, I can figure out how to write a test for it.
3. I don't need to know anything about what functions cache_get() calls, and what THOSE functions' arguments/return values are. I only need to know what types of things cache_get() expects, and what its conditions are. I treat db_fetch_object and others as a black box, something that's already tested elsewhere in the system.
4. If I missed a condition (what happens when I am retrieving a CACHE_PERMANENT cache item when the expiry time is set to a negative value?), someone can simply make a patch against cache.inc.test that adds in that test. No biggie.
5. I also get to test other parts of the API, such as cache_set(), etc. that are all tied together. I suppose this is a -1 in your view though. ;)
If we could at least start with this type of test throughout core, and then move into scary, crazy runkit Mock Function madness later on either in a future release after the general development community is up to speed on even this stuff, or else on an API-per-API basis where there are scary multi-tier nested functions, I think that would help assuage a lot of my personal fears about the Drupal development community completely stagnating.
Why not?
Sure. That's a test. It's a functional level testing, nice and happy. It's not a unit test. It's a useful test. We already have one in core, it's called search.test and xml-rpc testing is similar. Not a problem. We can write unit tests later.
webchick, it's a perfectly
webchick, it's a perfectly fine test, and it shows that we can accomplish a lot with the tools at hand.
The minor problem with this test, is that you aren't testing the cache functions in isolation. If you break something in, say, the database layer, the cache tests will fail. With proper unit tests, the cache functions would not fail, only the database specific tests would fail. It makes it a bit easier to debug and track bugs. As said, in this case, it is a minor issue, but one could imagine more complex scenario's where there are a lot more dependencies. Let's wait for test coverage reports -- I think that might illustrate this point a bit better.
But you're right, there are a lot of tests that can be written with just SimpleTest, and I encourage we do exactly that. As said, it is something that we can execute on. The lack of unit tests is not a showstopper by any means. It's just that in certain situations, unit tests could be slighty more convenient and accurate than SimpleTests.
Right, that's all true...
But consider that right now if I broke something in the database layer, I'd get fatal errors all over my page, and would still have to start debugging stuff to figure out where it all stemmed from. If we have database tests in place, chances are good that those tests failing would give me a much stronger clue, much earlier, where the problem lied. Especially since we assume that all tests for all of core pass by default, so I can limit it even further to whatever I was working on at the time.
"Just" SimpleTest tests are a) a vast improvement over the situation we have now and b) writable by mere mortals. :) It sounds like you realize this, and are cool with a 200+ person testing team who knows "just" SimpleTest and not runkit and mock functions and this other stuff, rather than a 3-10 person team who does. I am happy to start building one. :D
Heh..
It seems that we're on the same page.
http://www.workhabit.org/mock-function-testing-drupal
For the most part, I agree with what chx suggested, but after digging in, the amount of mocking needed in order to mock all of core was fairly daunting.. I think it makes sense from the perspective of unit testing to mock for modules, but for all of core? I tend to agree with webchick on that. It would also make things somewhat confusing for developers (e.g. "Am I hitting the database or not?").
Would love to hear everyone weigh in here. I'm more or less playing devil's advocate, so feel free to shoot me down.
(P.S. Sorry I'm so late to the discussion.. catching up now).
listen to the code, it's trying to tell you something
agreed, the amount of mocking get's nasty.
Consider this, instead of building a framework to circumvent that, we listen to the code rewrite it so it's more testable. In the long run, we contribute back to simpletest and wind up with testable, decoupled code in core.
Catch mentioned w/so many references to database layer, if/when that changes a lot of tests stop working. Great point. In some cases it should drive code to create a (business data) layer of abstraction to consolidate code and SQL, almost always a good thing.
Writing tests means changing core!? This may leave newbies, SOC folks in a tough spot, but what would you rather have: 10K lines of same-old, untestable core and 50K lines of tests written in anger, or 10K lines of testable, improved core, and 10K lines of simple tests?
Believe it or not, we're converging I think. We have some usable approaches, let's start digging in and meet in the middle.
(P.S. aaron, I posted a comment to your mock functions blog entry, i like it.)
Tests being core patches is
Tests being core patches is very, very important. Newbies and SoC folk should be thrown in the deep end contributing to core. IMO it's a lot easier to write/review patches for core and get help with them than do the same for contrib (although not necessarily get them committed of course) - but perceptions are often the opposite.
Breaking tests with API changes is of course a very good thing long term, as is testing requiring refactoring - that's what they're there for. However to start with I think we should concentrate on testing patches that look likely to go in, and parts of core which we can reasonably expect not to be rewritten within a few weeks (unless it's to make them testable) to get wide coverage as early as possible. That means new code is tested, old code that's not immediately on it's way out also gets tested. It's impossible to plan for this with any accuracy but I'll have a think.
For example, I managed to get throttle.module removed from core entirely last week, it had no tests written for it. But someone could have spent a couple of hours writing those tests at the same time I was doing the patch - instead of tests for, say, update status - which we know isn't going anywhere and had a lot of work done on it in D6.
(Douglas: responded there as
(Douglas: responded there as well :))
This generally is the case in most test scenarios.. XP calls it "writing legacy code." I've seen it a lot with java code we've written. It's a tough balance between making code testable, and not generating a "code smell" just to make sure your functions are testable. For example, in the Java world I saw a developer create a factory for every conceivable entity type (irrespective of whether it was merely a POJO).. This made the code exceptionally complex and increased our load times significantly. What we developed internally was a process that indicated that this should be done on a case-by-case basis. Best practices around coding guidelines go a long way in this regard.
And yes, if you want your code to be testable, you will likely need to architect it to be so. This means encouraging separation of concerns, and creating a model layer (business data layer if you prefer) is a good example of that.
I agree that we're converging on it.. what are next steps? Has anyone kicked off a best practices doc on this? Or is that what we're aiming at here?
So now....
Drupal 8.
In Drupal 7 we wrote a huge amount of tests -- it's great -- but there are edge cases which are very hard to cover. I know, I know. We should refactor instead of mocking the living hell out of everything. Writing the clearinghouse - based unit testing is mostly done and writing -- heck, generating -- a bunch of unit tests won't harm anyone. It will just add more tests. It's all good. So, is it a go for D8?
If so, great news!
Looking forward to testing Drupal 8 & beyond!
I think one way to cover those hard-to-reach edge cases is to simply evangelize how awesome SimpleTest is & how it's so cool that it's one of the core modules included in D7. Both developers & themers seem to be afraid of testing as it can add a lot of extra work if it's done after the fact. But if you educate both groups (as well as contributors), then they can incorporate their testing into their projects beforehand which really makes everything a LOT easier for them. Once they realize it's not that hard to achieve this, everyone wins. At least that was one of my goals with my SimpleTest presentations (evangelize it & D7 & show how it helps everyone involved) @ LA Drupal & DrupalCamp LA:
http://migshouse.com/content/simpletest-drupal-7-you (this is part 1 & more of an intro)
I have plans to share these & future presentations w/the Drupal Dojo community & also to translate them into Spanish (to start). Any other thoughts on how we can help "spread the word" so that the whole Drupal community wins?
Miguel Hernandez - www.migshouse.com
Founder & CEO - The OpenMindz Group
Writer- Linux Journal & TechZulu