Welcome to the group for the plumbers who keep the community plumbing from leaking. ;)
Testing frameworks such as SimpleTest and Selenium, automated testing, and general community QA initiatives are all on-topic here.
Manual testing is also totally on-topic.
broken tests?
see: http://drupal.org/node/252920
This profile module test suite also has a lot of redundant code.
Are other tests broken? Is it possible that I'm the first one noticing this?
Read moreHOWTO: Submit tests with your patch
Note: This document is under development and targeted towards Drupal 7
What is Automated Testing?
Automated testing encompasses UI, API and unit tests. It's a technique for testing small sections of code to ensure it works as expected. For example, a UI test might check a form submission process to ensure form display, validation of input and submission works as expected. A unit test might test a function for checking street addresses by passing a variety of well-formed and mal-formed addresses to ensure it handles them properly.
Read more
Tests in core: What's next?
We now have a testing framework in Drupal core. What we need to move forward is some structure that makes writing tests easy and efficient so that we can go towards our 100% code coverage goal without wasting development resources.
Read more
Unit testing plan
We are pretty ahead with functional tests but unit tests are a completely different problem. The function we want to test calls other functions and so on. While theoretically one can unit test the leaves of the call tree and work from there, saying "drupal_validate_utf8 is already tested and we know it works.
Read more
Unit Test Failure Plan
I ran all the unit tests and I get
Test cases run: 57/58, Passes: 3123, Failures: 282, Exceptions: 34
Is this what other folks get?
Question 1: Do we need to coordinate the fixing of them now?
Question 2: Are we striving for 0 failures at the end of all this?
Drupal 7 Automated Testing Sprint in Paris Funding proposal
Summary: 20 000 patches submitted by 1200 contributors for Drupal 7 will require:
* testing tools development with patch testing automation
* 1200 core contributors to be trained on how to write and maintain core tests
Unit -vs- UI testing
Please correct me if I'm wrong, but it seems that this group's purpose is to concentrate on unit testing and code coverage. Excellent. This is a definite sign of maturity for Drupal.
But I also see some posts in here that refer more to UI centric testing. Is there yet an effort for organizing this testing as well?
Read moreSimpleTest Roadmap for Drupal 7
Last updated by jacob.embree on Wed, 2018-12-12 17:15
There are many things that need to be done to SimpleTest is to provide full test coverage, as requested by Dries, and be placed in the core. A lot of work has already been completed and we are making great strives towards making this a reality, but there is still much to do.
I have provided a general overview of what needs to be accomplished.
- Complete functional tests. (list)
- Review functional tests and ensure that they pass against HEAD and test necessary functionality.
- Plan unit testing
- Generate stub unit tests using SimpleTest Unit Testing
- Write unit tests.
Develop an xss and sql injection scanner based on SimpleTest
What I wanna develop for SOC 2008 is a module called security (or add security function to simpletest existing module) to enable users checking their drupal installation against xss and sql injection vulnerabilities.
It will be also good for module developers, in fact they can check their module before submitting them to drupal website. Users could be more protected against vulnerabilities that became from third part modules.
The objective of this work is to realize automated penetration test on drupal installation.
It will be based upon SimpleTest, already used by Rasmus (php core developer) to develop his own closed source xss scanner. SimpleTest is a jUnit similar library written for php.
My module could easily been extended to add more functionalities about security, but basically I think that this two are the most important.
If someone has functionality ideas to improve my project and make it better I'm here, listening for more proposal.
Drupal QA in the Next Year
<p>By: Robin Monks</p>
<h1>Introduction</h1>
<p>Drupal is community. It’s a community of visionaries that make Drupal what it is, we all want to see Drupal be more powerful, more flexible, and just plain better than the competition!</p>
Test stub generator...
Hello, I've written a perl script (fastens fire-proof suit) that generates module files as well as a test file complete with a test class and stubbed methods. I've posted the code as an issue to the simpletest module as webchick suggested: http://drupal.org/node/233261
From the documentation:
Features:
- creates a module folder and associated file skeletons
- generates a test class and test methods and places them in a test file
recognized by SimpleTest. This currently only covers unit tests,
and not higher level tests like functional or integration tests.
SimpleTest DROP Tasks
Last updated by skywalker2208 on Wed, 2008-03-19 00:09
Some SimpleTests would make great DROP tasks. So I propose we create a list of the most-needed SimpleTests, and at any given time, the top 3-5 are active DROP tasks. Edit the table below; I'll be updating the DROP issue queue(s) as necessary.
Read more
Automated Core Testing - Roadblock
I've been working away at a Drupal 6 module that will automate the process of performing a Drupal cvs checkout, installation and testing against core. The intent of this module is to be used on http://testing.drupal.org/ ... I was able to get the checkout and simpletest based installation working up to the point where the JS generated progress bar starts. There is a redirect in place that I believe is generated from the Batch API that does not play well with simpletest. As such, using simpletest to perform an installation currently does not seem to be possible. The details of the problem are outlined in two tickets that I have found.
http://drupal.org/node/204374 has been marked as a duplicate of http://drupal.org/node/229905 ... reading both issues helps understand the problem as it relates to writing a simpletest based installation.
I am in search of feedback/ideas on how to progress with this issue.
Read moreDrupal let the quality boat sail by
Hold on, hear me out :o) We're a really focused and strong (and almost too commercial) community, and we really have put our focus on features and output, and we've lacked a lot when it's come to making it all work correctly in every possible case.
That's where quality testing comes in. I've been thinking about Drupal's quality assurance process for a couple years now, but in the last few months it's really come to a head for me. My personal "dam-breaking" moment came when the Batch API in Drupal 6 prevented install, and other tasks depending on the API from running automatically, or in text based browsers, or in browsers were people have been privacy conscious and disabled JS and Meta refresh tags.
Without proper quality trails this will continue to happen. And here's another point I'd like to make:
Continue reading and comment at DrupalFiles >
Read moreDrupal 7 Automated Testing Sprint in Paris
Kieran's trying to put together a testing sprint this spring. We should come up with an agenda of what we're going to need to cover:
Goals and Deadlines
The Drupal 7 development life cycle may be extended for as long at 4 months from June, to October, 2008. This is possible if Drupal 7 has solid test coverage including functional, unit, and JavaScript testing.
Read more
Develop an automated javascript testing framework
Added to official ideas list at http://drupal.org/node/234712
(Draft SoC 2008 project proposal)
Drupal will be using simpletest for php functional and unit testing, but currently we have no equivalent framework for javascript.
This means that every time there's a minor jQuery update (or any other javascript patch to core), about six different people have to spend hours clicking around to see if anything broke.
Read more
Automated SimpleTest Management and Contribution Assistance
Background and Purpose
After the SimpleTest session Wednesday morning I talked with sethcohn about some possible solutions to some of the issues that are being faced with full scale implementation of the unit test coverage. Some of the ideas are specific to the SimpleTest framework, but much could be applied to other testing frameworks. According to chx it sounds like the SimpleTest framework is going to be used which makes this information especially relevent.
To start off the biggest unknown currently is how to go about implementing "function unit testing" (which I beleive is the term). In other words calling an individual function with different arguements and checking the results against the know correct output. Until now this has been virtually untested and, to my knowledge, not really thought about at all. Instead "functional" testing has been worked on (as I was apart of).
We received, by show of hands, a considerable number of people who claim to be interested in contributing to the testing challenge, as introduced by Dries in his keynote. If anywhere near the number of people that raised their hands contribute, it could create a very chaotic situation and put a lot of pressure on the SimpleTest maintainer(s). In an attempt to address this I have compiled some ideas and thoughts.
Read more@test ?
A question which a bunch of people asked at Drupalcon (including me) was how to know which modules and functions have tests (and which don't). This is useful both for computing test coverage and for knowing where to focus efforts as we try to build initial coverage for core.
There are tools which do % of test coverage, but according to the BOF today, they tell you line numbers rather than functions - a useful metric but not perfect.
Read more
Drupalcon: Simpletest and the Future of Test-Driven Development in Drupal
In Dries' keynote on Monday, he announced that he was willing to push back the code freeze to mid-October if Drupal has full test coverage. This is going to require a cultural shift towards test-driven development that is supported by a community who writes both functional and unit tests. This will ensure that improvements in one area of Drupal don't cascade and break other areas, which means that more time is spent in the development cycle fixing bugs rather than adding new features.
The unit testing working group met with Dries on Tuesday morning, and they came up with a tentatively-approved plan to start including SimpleTests in core. This will emphasize the importance of writing testable code, but requires active support from the Drupal community to help write functional and unit tests.
If there isn't enough progress by May, then Dries will be forced to shorten the code freeze and focus on fixing bugs. On Monday morning, four members of the testing working group presented the following information on the SimpleTest framework and the future of test-driven development in Drupal.
Read moreFunctional SimpleTest Status - List of all tests
Purpose
This list serves as on overview of the current functional testing status. This will help keep track of who is writing tests and what tests are left to be written.
How you can help
We need people to help write tests. If you can program then feel free to write some tests. If you are not into coding you can help with running simpletest_automator and recording tests.
Maintainer
Jimmy Berry (boombatower) is maintaining this list. If you begin work on a test please contact me and I will be happy to update the list. If there is any information that is incorrect please notify me.
If you have any questions please contact me.
Status
We currently have 38 tests taken out of 43 for a test coverage of 88%. This will change when the misnamed tests are merged.
|
|
