Today I finally decided to start my Mercury server project with Aegir. Once mastered it will be the way for me to maintain multiple sites on Drupal.
I was following instructions from documentation on http://groups.drupal.org/node/70268, however I was stopped at step 3 that was handled just like explained in http://groups.drupal.org/node/145919 (however I would like to know why is it happening and some fix it or update instructions).
After that, everything seemed to go along fine until Hudson instruction installation that is also flawed as port 8080 is used by Solr (java) and has to be changed manually to 8081 which I also easily figured myself and got to next step.
Dreaded step 11 also prevented some more folks to move further and this was tough one for me: It is stated that once Hudson is started, there will be Mercury_init somewhere right as soon as I go to http://localhost:8081.
I did that but as some folks noted, there was nothing to start, no Mercury_init and nothing to start building.
What are we missing?
It appears as there is no /etc/mercury/init.sh, just fabric/ logs/ mercury.make teplates/
Comments
...in the same boat
I wonder where to look for mercury_init....
Steve Oliver
almost found something... ?
I did find reference to what seems should be in hudson/jobs:
/var/lib/bcfg2/Bundler/hudson-jobs.xml
/var/lib/bcfg2/TGenshi/hudson/jobs/mercury_init/config.xml
...tried running Hudson command ("/etc/mercury/init.sh --headless 2>&1") directly, but there is no /etc/mercury/init.sh.
... hmmm...
Steve Oliver
Update Server Environment > Incorrect Entries
When updating the server environment with
bcfg2 -vqed, I notice the very end:Phase: finalCorrect entries: 454
Incorrect entries: 37
Path:/etc/apache2/apache2.conf Path:/etc/tomcat6/Catalina/localhost/default.xml
Path:/etc/apache2/conf.d/mercury Path:/etc/tomcat6/server.xml
Path:/etc/apache2/ports.conf Path:/etc/varnish/default.vcl
Path:/etc/apparmor.d/usr.sbin.mysqld Path:/usr/local/bin/update_mercury.sh
Path:/etc/default/hudson Path:/usr/local/bin/update_pressflow.sh
Path:/etc/default/tomcat6 Path:/var/lib/hudson/config.xml
Path:/etc/default/varnish Path:/var/lib/hudson/jobs/import_site/config.xml
Path:/etc/memcached.conf Path:/var/lib/hudson/jobs/mercury_cron/config.xml
Path:/etc/mercury/boot.d/S100main.sh Path:/var/lib/hudson/jobs/mercury_init/config.xml
Path:/etc/mercury/init.sh Service:network-interface
Path:/etc/mercury/templates/cron Service:network-interface-security
Path:/etc/mercury/templates/iptables Service:plymouth
Path:/etc/mercury/templates/tuneables Service:plymouth-log
Path:/etc/mercury/templates/vhost Service:plymouth-splash
Path:/etc/mysql/my.cnf Service:plymouth-stop
Path:/etc/php5/apache2/php.ini Service:tty1
Path:/etc/php5/cli/php.ini Service:ureadahead
Path:/etc/php5/conf.d/apc.ini Service:ureadahead-other
Path:/etc/rc.local
Total managed entries: 491
Unmanaged entries: 46
Package:at Package:language-pack-en Package:liblwres60 Package:strace
Package:bind9-host Package:language-pack-en-base Package:libparted0debian1 Package:time
Package:busybox-static Package:language-selector-common Package:libpci3 Package:traceroute
Package:dnsutils Package:libbind9-60 Package:lshw Package:ubuntu-standard
Package:dosfstools Package:libdigest-sha1-perl Package:lsof Package:usbutils
Package:ed Package:libdns64 Package:ltrace Package:whois
Package:ftp Package:libelf1 Package:memtest86+ Service:atd
Package:git-core Package:liberror-perl Package:mtr-tiny Service:hvc0
Package:hdparm Package:libgeoip1 Package:ntp Service:ntp
Package:info Package:libisc60 Package:parted Service:screen-cleanup
Package:install-info Package:libisccc60 Package:pciutils
Package:iso-codes Package:libisccfg60 Package:popularity-contest
...All kinds of goodies are left out?
Steve Oliver
Entries not handled by any tool
running
bcfg2 -veqd, i get:The following entries are not handled by any tool:Path:None:/etc/apache2/apache2.conf Path:None:/etc/mysql/my.cnf
Path:None:/etc/apache2/conf.d/mercury Path:None:/etc/php5/apache2/php.ini
Path:None:/etc/apache2/ports.conf Path:None:/etc/php5/cli/php.ini
Path:None:/etc/apparmor.d/usr.sbin.mysqld Path:None:/etc/php5/conf.d/apc.ini
Path:None:/etc/default/hudson Path:None:/etc/rc.local
Path:None:/etc/default/tomcat6 Path:None:/etc/tomcat6/Catalina/localhost/default.xml
Path:None:/etc/default/varnish Path:None:/etc/tomcat6/server.xml
Path:None:/etc/memcached.conf Path:None:/etc/varnish/default.vcl
Path:None:/etc/mercury/boot.d/S100main.sh Path:None:/usr/local/bin/update_mercury.sh
Path:None:/etc/mercury/init.sh Path:None:/usr/local/bin/update_pressflow.sh
Path:None:/etc/mercury/templates/cron Path:None:/var/lib/hudson/config.xml
Path:None:/etc/mercury/templates/iptables Path:None:/var/lib/hudson/jobs/import_site/config.xml
Path:None:/etc/mercury/templates/tuneables Path:None:/var/lib/hudson/jobs/mercury_cron/config.xml
Path:None:/etc/mercury/templates/vhost Path:None:/var/lib/hudson/jobs/mercury_init/config.xml
Those are the ones we want handled, right?!
What are we missing??
Steve Oliver
Having the same problem here
Once hudson is installed and manually changed to 8081. I can login to the Hudson interface but there are no Mercury jobs!
When I try to run bcfg2 -veqd I get:
{'filelog': None, 'verbose': True, 'extra': True, 'ca': '/etc/bcfg2.crt', 'remove': False, 'file': False, 'help': False, 'password': '1lmjkgd1', 'omit-lock-check': False, 'certificate': '/etc/bcfg2.crt', 'paranoid': False, 'indep': False, 'decision': False, 'cache': False, 'servicemode': 'default', 'bundle-quick': False, 'profile': False, 'dryrun': False, 'kevlar': False, 'args': [], 'bundle': [], 'quick': True, 'user': 'root', 'key': '/etc/bcfg2.key', 'lockfile': '/var/lock/bcfg2.run', 'retries': '3', 'setup': '/etc/bcfg2.conf', 'serverCN': None, 'server': 'https://localhost:6789', 'encoding': 'ascii', 'decision-list': False, 'debug': True, 'drivers': ['RPMng', 'Pacman', 'RcUpdate', 'Chkconfig', 'YUMng', 'MacPorts', 'Encap', 'launchd', 'FreeBSDPackage', 'FreeBSDInit', 'SMF', 'VCS', 'SYSV', 'YUM24', 'Portage', 'Action', 'POSIX', 'Upstart', 'Blast', 'Systemd', 'APT', 'IPS', 'DebInit'], 'interactive': False}SSL Key error
Failed to download probes from bcfg2
Server Failure
Bingo!
Found the problem.
It wasn't making sense to me why all of a sudden this would break when the 1.1 codebase hasn't changed in months. Ironically, I found a typo in BCFG2's Upstart.py file ('return Fasle' instead of 'return False') that alerted me to the fact that Mercury is pointing to the testing PPA for BCFG2.
So, in tracking this bug down, I realized that no Genshi templates were being ran properly, resulting in a huge amount of things being wrong. I wondered if BCFG2 had changed something in their testing branch that was rendering the templates non-usable.
On a hunch, I replaced the testing PPA's with the stable PPA's for BCFG2, re-installed things, and all is as it should be.
To fix the problem, immediately after you download mercury.list.1.1, run this command:
sed -i 's/lucidtesting/ppa/g' /etc/apt/sources.list.d/mercury.listAnd follow the rest of the directions as normal. I'll alert one of the Pantheon guys to the problem and see if they can update the mercury.list.1.1 file with the stable PPA's so this step isn't needed.
On a very related note, I should have a working Mercury 1.1 + Ubuntu 10.04 Linode Stackscript available this week :)
So far so good....
After including your fix in the setup process, after Step 5. starting the BCFG2 client, it seems things are looking good:
Restarting service hudson
> /usr/sbin/invoke-rc.d hudson restart
< * Restarting Hudson Continuous Integration Server hudson
< ...done.
Restarting service vnstat
> /usr/sbin/invoke-rc.d vnstat restart
< * Stopping vnStat daemon vnstatd
< ...done.
< * Starting vnStat daemon vnstatd
< Zero database found, exiting.
< ...fail!
Stopping service rsync
> /usr/sbin/invoke-rc.d rsync stop
< * Stopping rsync daemon rsync
< ...done.
Running Action vnstat_db_eth0
> vnstat -u -i eth0
< Error: Unable to read database "/var/lib/vnstat/eth0".
< Info: -> A new database has been created.
Action: vnstat -u -i eth0 got rc 0
Restarting service mysql
> /usr/sbin/service mysql restart
< mysql start/running, process 19510
Phase: final
Correct entries: 486
Incorrect entries: 5
Service:network-interface Service:network-interface-security Service:tty1 Service:ureadahead Service:ureadahead-other
Total managed entries: 491
Unmanaged entries: 43
Package:at Package:hdparm Package:libbind9-60 Package:liblwres60 Package:mtr-tiny Package:traceroute Service:screen-cleanup
Package:bind9-host Package:info Package:libdns64 Package:libparted0debian1 Package:ntp Package:ubuntu-standard
Package:busybox-static Package:install-info Package:libelf1 Package:libpci3 Package:parted Package:usbutils
Package:dnsutils Package:iso-codes Package:libgeoip1 Package:lshw Package:pciutils Package:whois
Package:dosfstools Package:language-pack-en Package:libisc60 Package:lsof Package:popularity-contest Service:atd
Package:ed Package:language-pack-en-base Package:libisccc60 Package:ltrace Package:strace Service:hvc0
Package:ftp Package:language-selector-common Package:libisccfg60 Package:memtest86+ Package:time Service:ntp
Steve Oliver
...that does it.
Thanks, @justintime - just your one command included in the steps worked great! Hudson ends up on the right port, the certificate gets generated without problems... looking good. :)
Steve Oliver
Step-by-Step instructions updated
I've updated the step-by-step instructions (http://groups.drupal.org/node/70268) to include this additional command.
Thanks again, @justintime.
Steve Oliver
Not actually needed anymore
While it doesn't hurt to run sed on the proper file, it's not technically needed anymore. Aaron with Pantheon updated the file so that it has the right information in it. I'll leave it up to you if you want to leave it in the wiki doc or not.
Took it out
I took it out.
Steve Oliver
Hrm, tried this all over
Hrm, tried this all over again with the intention of using your solution. Right at the start when I put in:
sudo wget http://pantheon-storage.chapterthree.com/mercury.list.1.1 -O /etc/apt/sources.list.d/mercury.list
This error occurred:
--2011-05-09 21:11:47-- http://pantheon-storage.chapterthree.com/mercury.list.1.1
Resolving pantheon-storage.chapterthree.com... 216.137.33.17, 216.137.33.43, 216.137.33.91, ...
Connecting to pantheon-storage.chapterthree.com|216.137.33.17|:80... connected.
HTTP request sent, awaiting response...
403 Forbidden
2011-05-09 21:11:48 ERROR 403: Forbidden.
Any ideas?
That file is getting updated,
That file is getting updated, but something's jacked up with CloudFront. For the time being, instead of running wget and sed, just do this:
cat <<EOD > /tmp/mercury.listdeb http://archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
deb http://ppa.launchpad.net/bzr/ppa/ubuntu lucid main
deb-src http://ppa.launchpad.net/bzr/ppa/ubuntu lucid main
deb http://ppa.launchpad.net/bcfg2/ppa/ubuntu lucid main
deb-src http://ppa.launchpad.net/bcfg2/ppa/ubuntu lucid main
deb http://archive.ubuntu.com/ubuntu/ karmic main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ karmic main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ karmic-updates main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu/ karmic-updates main restricted universe multiverse
deb http://security.ubuntu.com/ubuntu karmic-security main restricted universe multiverse
deb-src http://security.ubuntu.com/ubuntu karmic-security main restricted universe multiverse
EOD
sudo mv /tmp/mercury.list /etc/apt/sources.list.d/
Aaron got it fixed
Pinged Aaron in IRC, he got it working it looks like. Can you try the wget command again?
Mercury_init on Hudson
I've modified the wiki some. It seemed to be outdated. http://groups.drupal.org/node/70268
mercury.list wasn't finding the hudson package: deb http://pkg.hudson-labs.org/debian binary/
I replaced that line with: deb http://hudson-ci.org/debian binary/
reference: http://wiki.hudson-ci.org/display/HUDSON/Installing+Hudson+on+Ubuntu
Whala! mercury_init is at http://localhost:8081
Edit: Hudson gives this Error:
Reading package lists...Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be upgraded:
hudson
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0B/38.7MB of archives.
After this operation, 922MB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
hudson
E: There are problems and -y was used without --force-yes
Finished: FAILURE
I got around this by commenting out:
# apt-get -y update; apt-get -y dist-upgradeEdit2: This seems to kill hudson all together. Will post back when I have more information. If anyone has any ideas I'd love to hear them.
Scott Alan Henry
I have the same issue -
I have the same issue - Hudson fails Mercury_init :
WARNING: The following packages cannot be authenticated!hudson
E: There are problems and -y was used without --force-yes
Finished: FAILURE
Any ideas how to fix this?
Did you ever get this
Did you ever get this resoleved.. im also stuck on this.. ..
im also getting this
im also getting this now..
brand new install today from the steps outlined
eading package lists...Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be upgraded:
hudson
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0B/56.2MB of archives.
After this operation, 1,000kB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
hudson
E: There are problems and -y was used without --force-yes
[DEBUG] Skipping watched dependency update for build: mercury_init #8 due to result: FAILURE
Finished: FAILURE
when i do any sort of update with ubuntu hudson seems to update but when I run the same command again it wants to update it over and over.. its like its not taking the update or at least not telling ubuntu its been updated to 2.1.1
I fixed this by editing
I fixed this by editing /etc/mercury/init.sh
and changing both lines that say
# Update Packagesapt-get -y update; apt-get -y dist-upgrade
to
# Update Packagesapt-get -y --force-yes update; apt-get -y --force-yes dist-upgrade
and the one below the get mercury pressflow updates...
however I did notice one thing, when running this manually instead of from hudson like so
sudo /etc/mercury/init.sh --headless 2>&1everything worked but my init.sh file was set back to its original state again.. how do i keep this from happening so I can use Hudson to build.. btw why Hudson and not Jenkins?
Thanks, gateway69, your
Thanks, gateway69, your solution was crucial for me!
I updated the two "Update Packages" lines as above, then tried to run mercury_init and then Build Now via Hudson, but it failed with "Build was aborted".
However, running
sudo /etc/mercury/init.sh --headlessworked just fine.
I confirm your finding that init.sh, after it runs, returns to the previous state. I simply edited it second time then. And I had to start Hudson again using
sudo /etc/init.d/hudson restartbecause I saw it was not running.
---
Tomáš J. Fülöpp
http://twitter.com/vacilandois