Building Selenium Tests for Drupal.org

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

The steps involved in taking forward the Selenium test for Drupal.org initiative:

Link to drupal.org issue queue: http://drupal.org/node/991040

  • Identifying a groups of people interested in building Selenium test-cases
  • Working on Identifying critical tests
  • Writing tests
  • Working with the infrastructure team to run these tests

If you are interested in building Selenium tests and supporting the same, please add you name and the area of support.

Implementers:

Name Drupal userid How can i support?
Shyamala Shyamala Coordinate the activity, Build Selenium tests

About Selenium:

The Selenium-IDE provides for a browser plugin to develop test cases.These tests can then be exported to any supported programming language.

Selenium RC comes in two parts.
A server which automatically launches and kills browsers, and acts as a HTTP proxy for web requests from them.
Client libraries for your favorite computer language.

Selenium Grid: Coordinates multiple Selenium RC at the same time, to help speed test feedback.
"In-browser tests are inherently slow, but since commodity hardware is insanely cheap nowadays there is no excuse for not running all these tests in parallel on multiple machines and dramatically speeding up your feedback cycle. This is exactly what Selenium Grid is about."

The Key Differentiator with Simple Tests are

  • Resource utilization benefit on using Selenium Grid
  • The Browser plugin that eases the preparation of test cases
  • Support for Javascript
  • A more stable, capable iPhone driver
  • Updated Android driver

For more information on Selenium visit: http://seleniumhq.org

Comments

Enabled comments

rfay's picture

I enabled comments on this post - hope that's OK.

I think this is a fantastic initiative. Not sure it's in the right group, but great initiative.

ygerasimov's picture

This would be very good feature.

As I understand in order to run selenium tests we need to have RC run and then execute your tests.

I have taken look to http://drupal.org/project/selenium project. There we can only run selenium tests recorded with Selenium IDE and they should be saved as nodes. It is definitively outdated method and Selenium has gone much further than that.

There is another interesting project about integration for Selenium is http://drupal.org/project/drunit. It does connection with Selenium RC (what we need!) and runs tests saved in module files. Before test is run we should generate xml config file and run "phpunit --configuration sites/all/modules/drunit/config/config.xml". This should open new browser and run the tests.

What ideally we are looking for:
1. Create sandboxed drupal like it is done in simpletest.
2. Have our tests as php files saved in folders of modules with special extension (like *.selenium.test)
3. Run tests with or without opening browsers.
4. Show logs in seimpletest style with saving screenshots in verbose folder (or maybe even saving html's of sandboxed drupal).
5. To have checkboxes in settings what browsers you would like selenium tests run on.

I hope I will have some time to look more deeply how simpletest module works under the hood and see how we can get into it.

Selenium project is a "bridge"

rfay's picture

The "Selenium" project is just a bridge to simpletest, and is really hardly used, if at all. IMO it's not a very good solution going forward. You end up with tests that have all the limitations that Simpletest has (no javascript, no live browser, etc).

Kudos to everyone working on this. I think it's a fantastic initiative.

integration with simpletest

ygerasimov's picture

I have managed to connect to Selenium RC and open the browser from simpletest. I have opened issue on simpletest as I have some difficulties in understanding how I can get from Selenium to sandbox.

Issue is http://drupal.org/node/1123194

I will very appreciate some help.

research results

ygerasimov's picture

I have some results of my research of interacting with Selenium.

With the patch I have placed there is possibility to interact with Selenium 1 RC that is great. The problem is that we do not know how to get to the drupal sandbox. For that we need to change User-Agent header of the browser when it opens our site. I have very big doubts about possibility to do that in Selenium RC.

New version of Selenium 2 has much wider possibilities and might do what we need (http://code.google.com/p/selenium/wiki/TipsAndTricks#Changing_the_user_a...). The only problem we have is that there is no library that works with Selenium2 from PHP. I have found documentation about protocol of interaction http://code.google.com/p/selenium/wiki/JsonWireProtocol . So we just need to call REST calls with json data and looks like we are done. So it really looks like doable thing to implement. I will continue my research and will play with that protocol.

Keep you posted.

Agreed!

jdonson's picture

Selenium is excellent and Drupal sandbox idea is GREAT.

SimpleTest is the next big step in Drupal development.

Jeremy Donson
Database and Systems Engineer
New York City

Good stuff!

cfennell's picture

I love the idea of introducing Selenium-based regression tests to the community, especially if this can not only gain us coverage of the core system but also improve the process for module maintainers. I am completely swamped for the next few months, but I am definitely interested in helping out with Selenium sometime this year. I've done a little work with Selenium Grid for some internal projects and would like to do more.

On a slightly related note, there was some talk of using Selenium as a cheap means of running test plans for automated performance tests on core at Drupalcon Chicago http://groups.drupal.org/node/133679 Even though this is not a browser-related regression test per se, it might be a good place for Selenium to get a toe-hold in the community and to get some resources behind an infrastructure to support it. I would ping @catch on IRC and run it by him - not sure where we are w/r/t the automating test-runs, but it might be worth a look.

another connection

cfennell's picture

Actually, I think Owen Barton, the maintainer of the old Selenium module you posted, is also involved in the automated performance testing group (http://groups.drupal.org/node/133679/revisions), so there's another connection to look into.

several high-level terms and env variables...

jdonson's picture

To fail to test is to Plan to Fail! :-O

Simplifying this for those with relevant skills and little time seems wise!

TERMS:

Unit Testing

PHPUnit

SimpleTest (ONLY this is Drupal-specific!!)

Selenium and SauceLabs

Regression Testing

Source Code Management

Cruise Control

Also, looking at how Drupal core already makes SimpleTest
core and required for D7 patches is the workflow to watch!

That should be applied to all kinds of php and jqeury code!!

Therefore it is relevant to all LAMP engineers, developers and themers!

Adding Selenium testing with cross-browser sensibilities
and integrating this with hudson/git/cruise control
and database reporting is simply PRUDENT!!

ISP's should be setting this up for us ALREADY!

Jeremy Donson
Database and Systems Engineer
New York City

I just saw these two tweets

mlangfeld's picture

I just saw these two tweets below. I don't know if it's relevant, since it's built in Symphony, but is on github so could be reviewed for applicability. Hope you find them relevant.

BeCircle - andre
Continuous test server built with a single php file. Sismo = Silex + Symfony2 Components github.com/fabpot/Sismo /via @fabpot

BeCircle - andre
RT @fabpot: Little gift to my followers... Sismo is now available, no kidding! Powered by #Silex and #Symfony http://sismo-project.org/

Best, Marilyn

Remember, we are looking for

drumm's picture

Remember, we are looking for tests, not test systems. When there are a few tests to run, that is the time to invest time/resources in infrastructure. I don't want to see valuable time wasted on infrastructure until we know it will be used.

Sandbox project created

ygerasimov's picture

I have released my code to http://drupal.org/node/1131210/

Now new firefox window starts up and opens node created in sandbox.

I think it is reasonable to add API as close to generated by Seleinum IDE as possible so creating tests will be maximum easy.

All asserts should be done via normal simpletest asserts.

As there are quite a lot of methods on selenium server I think we should define priority of implementation. As we can already test really a lot with current simpletest I believe we should concentrate on parts that only selenium can test.

I believe these parts are:
1. Clicking on elements to check javascripts actions
2. Hove on elements
3. Drag and drop
4. Forms submission

I have committed test for

ygerasimov's picture

I have committed test for drag and drop weights of taxonomy terms.