Diving into Phing II

This is the second in a series of articles on Phing the fantastic PHP application build framework which DAST extends. Although DAST will come with quite a number of ready-made scripts, its real power lies in providing the foundation for Drupal developers, testers, admins and distro maintainers to quickly and easily create their own build scripts. A distro builder may choose to bundle their build script which painlessly downloads the required modules, sets up the user's database, and runs basic smoke tests to verify the Drupal web site on the user's web server is ready to use. DAST applies the concept of a software project build to creating Drupal sites and has a wide range of uses - See the upcoming article "10 cool Drupal things to do with DAST."

In Part I I looked at the basic structure of a Phing build file - projects, targets, and tasks. I posted a use case http://groups.drupal.org/node/4477 which is now implemented as the dast-patch build file for automating creating patch test-beds. In Part III we'll look at how this build file was implemented using the stuff we've covered before. First I'll wrap up this dive into basic Phing with a a description of properties and types which are present in every useful build file, mappers and filter chains which are used for file I/O, and the ever-important conditional execution and iteration. Remember Phing is a feature complete implementation of Ant - all concepts in Phing are identical to their Ant counterparts, so if you're familiar with writing Ant build files, using Phing will be easy like Sunday morning.

Properties

Variables in Phing are implemented as properties. Properties in a build script are referenced as ${propertyname}. Properties can be passed to the build script in three standard ways - in properties files, as command-line arguments, and as shell environment variables. The first two are used in the majority of cases. All DAST build-scripts are configured by property files.For example, in the dast-patch script, the dast-patch.properties file looks like this:

[1]

Properties in a file are simple name/value pairs with comments delimited by #.

The second way of passing property values into the build script is using the -D command-line parameter. The syntax is simply

dast -f <buildfiile> -D"<property1>=<value>" -D"<property2>=<value>"...[2]

By default, properties you specify this was override whatever you define in the properties file. So in dast-patch if you don't wish to run the Drupal installer at the end of the build, you could pass in the drupal.installproperty on the command line:

./bin/dast -f dast-patch.xml -D"drupal.install=no"[3]

Properties are loaded into the environment of the executing script using the tag:

[4]

Even though I've used the term variable to describe Phing properties, they are actually more like tokens or symbolic constants. When the build-file parser encounters ${propx} it simply replaces the string ${propx} with whatever is currently defined for that property...keep this in mind if you start thinking about doing fancy stuff with properties.

A second thing to note is that you can use property values inside properties and property files - e.g. in the dast-patch and site-fetch build scripts there is the property definition:
drupal.modules.Cvs.Output = ${build.log.defaultdir}${php.directory_separator}site-fetch-cvs-modules-output.log
build.log.defaultdir is actually a property defined earlier in the properties file while php.directory_separator is defined directly in the build script, before the properties file is loaded. As long as property1 is defined anywhere before property2, then property2 can reference it using property1 = ${property2} + ...

Types

In addition to simple string and integer values Phing property types can be more complex - the most common complex types are those used to implement sets, lists, and containers. For example the type is used to select a set of files using filemasks, e.g.:

[5]

creates a collection of filenames which have the extension .po in every subdirectory of the drupal directory, except the fr sub-directory. The id attribute can be used to name a property for reference later in the build. Custom types are defined by sub-classing the DataType class and implementing various interfaces. For example, we would like to able to specify in a Drupal build file

[6]

In this example we define a custom task called DrupalCVSTask which inherits most of itsfunctionality from the standard CVSTask class. In order to use the module elements like this we must create a DrupalCVSModuleType class which inherit. Defining custom types and tasks is probably the most powerful feature of Phing and is relatively straight-forward. Extending tasks and types will be covered in part III.

Filters and filter-chains

Filters are operations which transform the contents of a file. The power of filters comes from their ability to simulate the functionality of OS pipes - they can be chained-together, and these chains used as input to ordinary file I/O operations. For example, when setting up a Drupal site for the first time, usually the sites/default/default.settings.php is copied to settings.php and the $db_url variable modified to point to the database created to host the site. In a build file this step is automated like so:

[7]
In this example, the chain begins at the bottom, which selects the existing default.settings.php file. This file is then passed through the replaceregexp filter which performs a standard search-and-replace operation to set the $db_url (the regex is kinda crude but workable for now.) This modified file is then passed to the copy task.

Mappers

Mappers are similar to to filters but instead operate on file names. For example, you can rename files according to a regular expression pattern, or flatten file paths. For example,
<mapper type="regex" from="^(.*)\.php$$" to="\1.inc"[8]
Will rename all .php files to inc.
In a build script, mappers aren't chainable (I have to verify this but chaining mappers may not be logically possible/desirable) But they can accept the output of a filter-chain. e.g

[8]
Will pass .php files through a filter to replace DOS/Windows newlines with Unix newlines and then the mapper renames them to .php.

Conditional execution

Phing supports conditional execution using the tasks and nested elements and attributes which are specifically recognised as carrying a Boolean true or false, such as and . For example, in dast-patch we want to check that the properties file specified exists:

[9]
As you can see you can use the standard boolean operators . Testing equality with is simply <equals arg1="${myprop}" arg2="myval" />There is one gotcha in testing equality - you cannot define a property with a literal value of true like: <code><property name="mybooleanprop" value="true" />
If you do this then the test
<equals "${mybooleanprop}" "true" /> will return false because Phing evalutes ${mybooleanprop} as being set to 1...a PHP quirk I guess. Instead of true/false use yes/no.

Iteration

Iteration is done exclusively using the task. The syntax is; for example in dast-patch there is:
<foreach list="${drupal.modules.list}" param="module.name" target="fetchModulefromCVS"/> [10]
list is a delimited set of items to iterate over, the default delimiter is comma, which can be changed with the delimiter attribute. In this case drupal.modules.list is the comma delimited list of modules to fetch. Each item in the list is assigned to the property module.name and the fetchModulefromCVS target is called in every iteration - this means that the fetchModulefromCVS will always have a module.name property available containing the current module name being processed and can do its work for each module in the list.

Conclusion

So that wraps up the discussion of the main elements and constructs of a Phing build file. In Part III I'll examine the out-of-the box tasks provided for build operations and how I used them to put together the dast-patch build script. Part IV will then cover extending Phing - as I mentioned earlier the key to making DAST accessible to a broad an audience as possible is to encapsulate Drupal specific-functionality into custom elements that Drupallers can learn to use with minimum effort.

The best way to learn Phing concepts is to study actual build files for complex projects. Phing is a cousin to Propel - an object-persistence library for PHP, and these two are both sub-projects of the binarycloud framework. On the DAST wiki page I've posted some build files from BC and Propel which are good illustrations of how the concepts we've discussed so far are used together.