Automatic testing security issues?
One of the biggest concerns about automatic testing is how to prevent potentially dangerous code to cause harm to the server and other tests.
Securing Apache
Apache can be secured with chroot or virtual environment with this way we jail apache in its own directory so even if it is compromised nothing serious can be done. Chrooting apache can be achieved with two apache modules mod_security http://www.modsecurity.org/documentation/modsecurity-apache/1.9.3/html-m... and mod_chroot http://core.segfault.pl/~hobbit/mod_chroot/. These two modules make chrooting apache very simple. Virtualization is a bit more tricky using Xen or Vmware.
Securing php
With carefully set php.ini we can tighten the security even more. For example with disable functions directive we can disable some potentially dangerous functions ( exec and probably some other potentially dangerous functions that drupal doesn't use). There si probably a lot more directives that can be adjusted to achieve greater security.
Securing test automation
Currently test automation server requires a password in order to accept the patch into automation process this password is set on simpletestautomation module settings page on both servers this can be extended with use of https protocol so data between this two servers will be encrypted.
One other possibilty is to create a system similar to cvs_log which will allow only authorized users to access test automation.
Making sure that test doesn't temper with another tests environment
Database
Every test has the same database user name and password so it can theoretically delete another tests database. One way to solve this is by creating a new user for every test but in order to do this we would need a superuser username and password to be saved in clear text on automation server.
Test files
What is stopping a test to delete files of another test?
Please write your own concerns or comment ones that I wrote above. In order to create a usefull test automation system this issues need to be resolved.

database
how about precreating 30 databases and db users and then when setting up a new DB you pick the oldest used one. this way, tests can't fiddle with each other.