Posted by kasperg on January 4, 2012 at 10:32am
We've been working on implementing continuous integration for our projects to improve code quality. In the process we've built two templates which should make it easy to setup CI for new projects and want to share these with the community. Hopefully you'll find it useful!
The templates are:
- Phing Drupal Template: Orchestrates the build
- Jenkins job template for Drupal projects: Monitors repositories for changes, executes the build and displays results.
The build currently uses the following tools:
- PHP: Lints all PHP files to check for syntax errors
- PHPCPD: Looks for copy/pasted code which is ripe for refactoring
- PHPMD: Looks for messy code and potential problems
- PHPLOC: Generate metrics for the project
- Coder module: Check for Drupal Coding Standards compliance
- SimpleTest: Runs tests
- CSSLint: Basic syntax checking and identification of problematic patterns or signs of inefficiency
- JSLint': Static code analysis for JavaScript files
If you are using git integrating the build should be a matter of including
https://github.com/reload/phing-drupal-template.git as a submodule and adding a build.properties file.
The Phing template has been integrated in a demo project and we also have a Jenkins server building this project.
Comments and suggestions are very welcome!
Comments
Tankyou very much kasperg! In
Tankyou very much kasperg!
In case it could help, I recommend you to install Sonar to analyze your projects. It is an awesome app that covers some of the quality aspects you are talking about.
This is a screenshot of a non-Drupal project analyzed by Sonar. Sonar can also be configured to analyze any Drupal project and there is a Jenkins plugin.
Sonar
Sonar looks interesting. The focus on code analysis fits nicely with the current scope of the Phing Drupal Template and the reporting seems much better. However I'm not sure it can be integrated easily.
The Sonar website and [http://docs.codehaus.org/display/SONAR/PHP+Plugin|the PHP plugin] seem very much focused on language support and using specific tools to analyse a project. Some of these tools can be directly applied to Drupal projects (PHPMD, PHPCPD), others require a Drupal specific variant (Coder module instead of PHP_Codesniffer, simpletest instead of PHP_Unit) and then some tools used in the template are neither PHP specific nor have a Sonar plugin (CSSLint, JSLint').
The code analysis plugins in Jenkins are based on reporting formats. While it requires you to orchestrate your build yourself to produce files in these formats it allows us to use all sorts of tools as long as the support these formats.
I'm been looking for documentation on whether Sonar can use existing report files and in what formats but without much luck. I haven't tried installing Sonar yet - it required a bit more work than
sudo apt-get jenkins.Since you have experience using Sonar perhaps you can clarify?
Hi kasperg.Sonar uses the
Hi kasperg.
Sonar uses the different tools you mentioned (PHPCpd, PHPMD, PHPDepend...) to generate its own xml report files. This can be made and cronified thanks to ant or maven scripts. All this info is gathered and showed up very nicely in the Sonar dashboard.
I haven't tested the Jenkins plugin yet, just heard about it.
Although using Coder module for the coding standards compliance might be the best idea, you can do the same with CodeSniffer. das-peter committed a specific drupal sniff time ago and it worked for us (with some minor modifications). We don't manage the coding standards rules from Sonar nor from Jenkins, we use the svn hook pre-commit (available in git too) to reject any commit that does not pass the codesniffer analysis.
The problem with Simpletest is bigger. Sonar uses xdebug and PHPUnit to find out what code is actually being tested thanks to the Cobertura plugin. I think there is no way to use this feature with simpletest, so this is one of the Sonar handicaps. All the testing work should be done via Jenkins then, IMO.
I'm just back from holidays and trying to organize myself and finding out the next steps in the beginning of the year, but please give me a few days and I'll write a post about how we did CI and QA in our devteam.
Cheers!
Ensuring coding standards
I was aware of das-peters sniffs but did not consider is a serious alternative to Coder. Now there's a official project, Drupal Code Sniffer, which in my opinion makes it a lot more compelling option.
For our projects coding standards extends beyond PHP. We also check CSS and JavaScript. There is info in the Sonar docs regarding how Sonar can collect reports generated by other tools but no mention of support for CheckStyle XML reports.
I look forward to your blog post regarding your setup.
Come across Upal yet?
Love the discussion guys!
Re: simpletest
Might want to check out Moshe's pet project, which had some serious discussion in Denver :)
https://github.com/weitzman/upal
PHPUnit
I am pursuing the work of Moshe to integrate PHPUnit in Drupal Core, you guys can follows/helps on this issue: http://drupal.org/node/1801176.
This would allow tools like sonar to be integrated in the build/test cycle of jenkins easily by reusing .xml reports for instance.
Hi, I'm trying to install a
Hi,
I'm trying to install a Jenkins server and I would like know if its possible have only one build config for all projects ?
Thanks,
Re: Jenkins
While it is possible, I can not think of a single instance or reason why one would want to do such a thing. If you must, or think you must, then look into using paramters for the build, and mapping those parameters to repos / branches. We use a single job per repository, and then pass a parameter to the job to indicate which branch of the repository to run the build/tests on.
John
John W Smith
IT / IS Contractor
Impressions 1st Consulting
I finally reach my goal,
I finally reach my goal, thanks for your comments.
I still keep config per project, but outside of the workspace, so I changed the config file and its working nicely now.
Currently working in the same
Currently working in the same domain but working with capistrano rather than phing? Aside from PHP tools being preferred above ruby ones, can you offer any insight into why Phing might be better?
My impression was the Phing was a build tool that doubled as a deploy tool. Cap seemed to be more full-featured and supported, plus drush (make) seems to be our in-house build tool that does the trick already. And besides, I really wanted to get my hands dirty with ruby :)
Overlapping tools
Capistrano is for deployments, drush_make is for builds and Phing does some of both.
If a build only requires assembling code from various sources drush_make is fine. Our template extends the build process by performing static code analysis of the assembled code to identify potential problems i.e. debug code, syntax errors, security problems etc. If these are detected the code probably shouldn't be deployed.
In this perspective the value of Phing is in the orchestration of other PHP tools such as the ones listed above which perform the actual analysis.
Our build template doesn't handle deployments yet.
Hi, kasperg. First of all,
Hi, kasperg.
First of all, thank you for the documentation.
I followed all the steps, but I have a problem when I try to enable the new Job, copied from drupal-template conf. Jenkins gives me a nullpointer exception and I am not able to save it.
It seems it is no compatible or I am missing any dependencies...
Have you got similar problems? Any Idea?
Thanks
www.monchacos.com/monchacos
Have not had similar issues
Sorry, rafinskipg, I have not had any similar issues. Perhaps you can try to remove the individual elements of the build to identify what part is causing the problem?
I've just updated my test server to the latest version of Jenkins with plugins and this have not required changing the config.xml.
In general please keep issues on GitHub. There we have a much nicer interface for dealing with these and linking them to changes in the code.
Thank you kasperg, I will do
Thank you kasperg, I will do it.
First of all i will try it on linux, as it seems better for this.
www.monchacos.com/monchacos
Thank you! And a quick question while I try setting this up.
Just wanted to know if the templates will work for profile development; i.e. if the root of the project being built is the 'profiles' and not 'sites' directory, and the drush make scripts are of the current drupal_org format (i.e. drupal-org.make + drupal-org-core.make + build-[PROFILE].make.)?
Thanks again!
Profile development supported
The template works for profile development but things can get a bit tricky regarding setup and make files.
Take a look at this project - especially the drupal.make.nocore setting and the referenced build file.
Thanks! And great work!
Thanks! And great work!
Run all tests
Just got this setup and from what I can tell the simpletests which get run are indicated in the project.code.prefix and project.code.custom parameters in the build.properties file. If I wanted to run all simpletests for all enabled modules, is there an easy way to do so or would I have to add each module to the project.code.custom parameter?
Thanks.
No easy way
There is no easy way to achieve this.
You could add each module to the project.code.custom parameter but that would also mean that each of these modules would be run through the static code quality analysis tools. This creates a noisy baseline for the results.
If you are up for it it should be relatively easy to bypass the logic filtering the test cases and instead use
--allbased on the value of a property e.g.project.simpletest.all.Pull requests are very welcome.
kasperg
Hey Kasper G,
Do you want to make a drupal project out of this? I've been using this a lot and got some proposed improvements on it.
We can probably set-up a hackathon for this during drupalcamp wisconsin.
http://midwest-developer-summit.com/program/session-schedule
What do you think?
** BTW as of July 3 2012, drupalcs http://drupal.org/project/drupalcs has progressed and prolly has passed by coder-review http://drupal.org/project/coder .
Proper Drupal project
I thought about it. The template didn't not really match any of the project types on d.o when it was initially built half a year ago. Since then there there has been an increasing number of such projects and if there is demand for it I'll certainly reconsider.
I'm glad you find it useful. Pull requests and issues are very welcome.
Agreed. Replacing the coder-review code with drupalcs would be a nice addition. It should be fairly easy with the PhpCodeSnifferTask Phing task
First let me said thank you
First let me said thank you to @kasper for his great job on this template. Recently we have been start to use in our initial Continues Integration with Jenkins.
Currently we are using the phing template for deploy a development server that concentrates the job from all our staff and validate the code, regarding that I would like if you could point me on the direction on how accomplish two goals:
a) Make Jenkins failds build if the style checker found a high level warning
b) We are going to start working on a project that mandatory needs to be validate agains W3 for markup and css. As soon as I have been reading, the CSSLint are not goint to be enought for make the code pass the validation. Do you know another scripts that could help us to validate and pass the W3 test?
Thanks a lot for your help in advance
a) Make Jenkins failds build
You can configure Jenkins to fail the build if any or Checkstyle high level warning under the Advanced settings for the analysis results.
Apparently you can run the W3C CSS validation tool from the command line. I haven't tried this though.
The tricky part is to validate the rendered HTML. This requires that the Drupal site is deployed to a web server during the build process. We haven't cracked that nut in a reusable way yet.
Integration with Aegir or using the build in webserver in PHP 5.4 and SQLite could prove useful here.
Once you solve that you need to figure out how to instantiate a set of representative content on the deployed site, crawl it and send the pages to the W3C HTML validator. They have an API.
Thanks for your answer
Thanks for your answer Kasper.
We have solved the html for now just simlink the site folder from Jenkins wokspace to our www apache dir and is working.
Thank you for point us to the W3F api, I'm going to start playing with it to make it run on our server.
The big challenge is going to be the content for the deployed site.
I'm going to post what is our final solution.
Thanks for sharing Kasper. I
Thanks for sharing Kasper.
I have a question how you deal with various settings environments ? Simple example is if you use this as a build env for development on the workstation side a common case is that every developer has his own database settings / files (that we dont want to sync but needed on development time)
Thanks for your ideas
Dennis
It's forbidden to download drupal core into an existing core.
fixed
www.darvein.org
Testing with jenkins+simpletest+drupal
Can anyone help me , how can I test one file or custom module using simpletest+jenkins+drupal ?????
And I have to see the results ???/