Ubuntu Karmic added to Mercury 1.0

Events happening in the community are now at Drupal community events on www.drupal.org.
Greg Coit's picture
Posted by Greg Coit on March 20, 2010 at 1:51am

Files for Ubuntu Karmic have been added to Mercury 1.0. Please consider these beta and unstable until we've hada a chance to test them some more. Also please post any problems/questions that arise here.

Thanks!

Greg

Comments

It's still broken for me

Posted by kcoop on March 22, 2010 at 12:34am
kcoop's picture

I found a few issues and tried again, but they weren't the only culprits. bcfg2 -qv freezes as it's installing services, and then the instance is in bad shape. Wish I understood bcfg2 better to help out more here. I'm using ami-6743ae0e, in us-east-1b (I find that setting no preference causes me to get a non-booting instance), and stepping through the instructions verbatim through step 5, then in steps 6 & 7 replacing jaunty with karmic in groups.xml and clients.xml.

Known issues:
- /var/lib/bcfg2/Packages/config.xml needs to have jaunty replaced by karmic. This explains why I first saw bcfg2-server trying to fetch jaunty packages.
- /var/lib/bcfg2/Cfg/etc/apt/sources.list/sources.list is jaunty specific. I doubt this affects the initial process, but I'm guessing it will end up replacing /etc/apt/sources.list, and future updates will be hosed.

re: It's still broken for me

Posted by Greg Coit on March 22, 2010 at 3:55am
Greg Coit's picture

Let's start with the issue of ami-6743ae0e. I've not used ebs-boot AMI's yet, but it would surprise me if they are limited to a specific Availability Zone. Have you asked Eric about this?

As for /var/lib/bcfg2/Packages/config.xml, we do need to provide it, but I've added instructions to the wiki explaining that this file needs to be edited for all versions of Ubuntu but Jaunty.

I had intended to remove /var/lib/bcfg2/Cfg/etc/apt/sources.list/sources.list from launchpad - sorry (the sources.list which comes with ! It's been done now.

Getting close! :)

Greg

--
Greg Coit
Systems Administrator
http://www.chapterthree.com

One mystery solved, and a couple more tweaks, but still no joy

Posted by kcoop on March 22, 2010 at 7:19am
kcoop's picture

I just tried creating another instance of ami-6743ae0e in us-east-1a, and found if I waited long enough it came up (it was a long time, I SWEAR!). ADD strikes again...

I also found a few more little issues with the configuration:

  • There is a stale reference to /etc/apt/sources.list in /var/lib/bcfg2/Bundler/mercury.xml
  • There is no APT-Source for the alestic repository for amazon services. Here's my tweaked /var/lib/bcfg2/Packages/config.xml:
    <Sources>
      <APTSource>
        <Group>amazon-web-services</Group>
        <URL>http://ppa.launchpad.net/alestic/ppa/ubuntu</URL>
        <Version>karmic</Version>
        <Component>main</Component>
        <Arch>amd64</Arch>
        <Arch>i386</Arch>
      </APTSource>
      <APTSource>
        <Group>ubuntu-karmic</Group>
        <URL>http://us.archive.ubuntu.com/ubuntu</URL>
        <Version>karmic</Version>
        <Component>main</Component>
        <Component>multiverse</Component>
        <Component>restricted</Component>
        <Component>universe</Component>
        <Arch>amd64</Arch>
        <Arch>i386</Arch>
      </APTSource>
    </Sources>

As for the freeze up, after trying it yet again with these changes, here's the output of starting bcfg2-server:

<b>ubuntu@domU-12-31-39-0B-76-06:~$ sudo  /etc/init.d/bcfg2-server start; tail -f /var/log/syslog</b>
Starting Configuration Management Server:  * bcfg2-server
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: Failed to load /var/lib/bcfg2/Pkgmgr indices
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: Failed to instantiate plugin Pkgmgr
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: Failed to load /var/lib/bcfg2/Rules indices
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: Failed to instantiate plugin Rules
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: Failed to load SSHbase repository from /var/lib/bcfg2/SSHbase
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: (2, 'No such file or directory')
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: Failed to instantiate plugin SSHbase
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: No Metadata Plugin loaded; failed to instantiate Core
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: ('No Metadata Plugin',)
Mar 22 06:31:40 ubuntu bcfg2-server[12710]: exiting
Mar 22 06:55:25 ubuntu bcfg2-server[13140]: Packages: File read failed; falling back to file download
Mar 22 06:55:25 ubuntu bcfg2-server[13140]: Packages: Updating http://ppa.launchpad.net/alestic/ppa/ubuntu/dists/karmic/main/binary-amd...
Mar 22 06:55:25 ubuntu bcfg2-server[13140]: Packages: Updating http://ppa.launchpad.net/alestic/ppa/ubuntu/dists/karmic/main/binary-i38...
Mar 22 06:55:26 ubuntu bcfg2-server[13140]: Packages: File read failed; falling back to file download
Mar 22 06:55:26 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/main/binary-amd64/Packa...
Mar 22 06:55:28 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/main/binary-i386/Packag...
Mar 22 06:55:29 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/multiverse/binary-amd64...
Mar 22 06:55:30 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/multiverse/binary-i386/...
Mar 22 06:55:31 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/restricted/binary-amd64...
Mar 22 06:55:31 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/restricted/binary-i386/...
Mar 22 06:55:31 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/universe/binary-amd64/P...
Mar 22 06:55:33 ubuntu bcfg2-server[13140]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/universe/binary-i386/Pa...
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: Failed to read file probed.xml
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: Loading experimental plugin(s): Packages
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: NOTE: Interfaces subject to change
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: service available at https://ip-10-242-199-82:6789
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: serving bcfg2-server at https://ip-10-242-199-82:6789
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: serve_forever() [start]
Mar 22 06:56:17 ubuntu bcfg2-server[13140]: Handled 35 events in 0.026s
Mar 22 06:56:18 ubuntu bcfg2-server[13140]: Handled 31 events in 0.002s
Mar 22 06:56:18 ubuntu bcfg2-server[13140]: Handled 83 events in 0.060s
Mar 22 06:56:18 ubuntu bcfg2-server[13140]: Handled 64 events in 0.015s
Mar 22 06:56:18 ubuntu bcfg2-server[13140]: Handled 38 events in 0.030s
Mar 22 06:56:18 ubuntu bcfg2-server[13140]: Handled 7 events in 0.003s

And here's the output of starting the client:

<b>ubuntu@domU-12-31-39-0B-76-06:~$ sudo bcfg2 -vq</b>
Running probe groups
Probe groups has result:
i386
Loaded tool drivers:
APT      Action   DebInit  POSIX  
Package apache2 not installed
Package autoconf not installed
Package automake not installed
Package automake1.4 not installed
Package ghostscript not installed
Package libapache2-mod-rpaf not installed
Package libpaper1 not installed
Package libpaper-utils not installed
Package libssl-dev not installed
Package libt1-5 not installed
Package libtool not installed
Package mysql-client-5.0 not installed
Package mysql-server-5.0 not installed
Package php5 not installed
Package php5-cli not installed
Package php5-curl not installed
Package php5-dev not installed
Package php5-gd not installed
Package php5-imagick not installed
Package php5-mysql not installed
Package php5-xmlrpc not installed
Package php-apc not installed
Package php-pear not installed
Package psfontmgr not installed
Package shtool not installed
Package subversion not installed
Package zlib1g-dev not installed
Package varnish not installed
Package postfix not installed
Package ssl-cert not installed
Package alien not installed
Package dpkg-dev not installed
Package debhelper not installed
Package ec2-ami-tools not installed
Package ec2-consistent-snapshot not installed
Package rpm not installed
Package apache2-mpm-prefork not installed
Package binutils not installed
Package build-essential not installed
Package deborphan not installed
Package dialog not installed
Package emacs22-nox not installed
Package g++ not installed
Package g++-4.3 not installed
Package gcc not installed
Package gcc-4.3 not installed
Package gettext not installed
Package html2text not installed
Package intltool-debian not installed
Package klogd not installed
Package libalgorithm-c3-perl not installed
Package libapache2-mod-php5 not installed
Package libbeecrypt7 not installed
Package libc6-dev not installed
Package libclass-c3-perl not installed
Package libclass-c3-xs-perl not installed
Package libclass-mop-perl not installed
Package libcompress-raw-zlib-perl not installed
Package libcompress-zlib-perl not installed
Package libcroco3 not installed
Package libdata-optlist-perl not installed
Package libdevel-globaldestruction-perl not installed
Package libdigest-hmac-perl not installed
Package libdigest-sha1-perl not installed
Package libfile-slurp-perl not installed
Package libgd2-xpm not installed
Package libgomp1 not installed
Package libhtml-template-perl not installed
Package libio-compress-base-perl not installed
Package libio-compress-zlib-perl not installed
Package liblist-moreutils-perl not installed
Package libmail-sendmail-perl not installed
Package libmoose-perl not installed
Package libmro-compat-perl not installed
Package libneon27 not installed
Package libparams-util-perl not installed
Package libparams-validate-perl not installed
Package libscope-guard-perl not installed
Package libstdc++6-4.3-dev not installed
Package libsub-exporter-perl not installed
Package libsub-identify-perl not installed
Package libsub-install-perl not installed
Package libsub-uplevel-perl not installed
Package libsys-hostname-long-perl not installed
Package libtest-exception-perl not installed
Package libtest-simple-perl not installed
Package libxml-namespacesupport-perl not installed
Package libxml-sax-expat-perl not installed
Package libxml-sax-perl not installed
Package libxml-simple-perl not installed
Package linux-libc-dev not installed
Package lynx not installed
Package lynx-cur not installed
Package mktemp not installed
Package nmap not installed
Package po-debconf not installed
Package reiserfsprogs not installed
Package sysklogd not installed
Package xfsprogs not installed
Package ca-certificates-java not installed
Package default-jre-headless not installed
Package gcj-4.3-base not installed
Package icedtea-6-jre-cacao not installed
Package java-common not installed
Package libaccess-bridge-java not installed
Package libcommons-collections-java not installed
Package libcommons-dbcp-java not installed
Package libcommons-pool-java not installed
Package libcups2 not installed
Package libecj-java not installed
Package libecj-java-gcj not installed
Package libgcj9-0 not installed
Package libgcj9-jar not installed
Package libgcj-bc not installed
Package libgcj-common not installed
Package liblcms1 not installed
Package libservlet2.5-java not installed
Package rhino not installed
Package tomcat6 not installed
Package tzdata-java not installed
Package memcached not installed
Package php5-memcache not installed
Package snmp not installed
Package snmpd not installed
Package apache2-utils not installed
Package apache2.2-bin not installed
Package apache2.2-common not installed
Package autoconf2.13 not installed
Package autotools-dev not installed
Package cpp-4.3 not installed
Package emacs22-bin-common not installed
Package emacs22-common not installed
Package emacsen-common not installed
Package g++-4.4 not installed
Package gcc-4.3-base not installed
Package gcc-4.4 not installed
Package gcj-4.4-base not installed
Package gsfonts not installed
Package jsvc not installed
Package libapr1 not installed
Package libaprutil1 not installed
Package libaprutil1-ldap not installed
Package libasound2 not installed
Package libavahi-client3 not installed
Package libavahi-common-data not installed
Package libavahi-common3 not installed
Package libc-dev-bin not installed
Package libcommons-daemon-java not installed
Package libcupsimage2 not installed
Package libdbd-mysql-perl not installed
Package libdbi-perl not installed
Package libdjvulibre-text not installed
Package libdjvulibre21 not installed
Package libevent-1.4-2 not installed
Package libfile-temp-perl not installed
Package libgcj10 not installed
Package libgs8 not installed
Package libilmbase6 not installed
Package libjasper1 not installed
Package libjline-java not installed
Package liblua5.1-0 not installed
Package libmagickcore2 not installed
Package libmagickwand2 not installed
Package libmysqlclient15off not installed
Package libmysqlclient16 not installed
Package libneon27-gnutls not installed
Package libnet-daemon-perl not installed
Package libnspr4-0d not installed
Package libnss3-1d not installed
Package libopenexr6 not installed
Package libopenssl-ruby not installed
Package libopenssl-ruby1.8 not installed
Package libperl5.10 not installed
Package libplrpc-perl not installed
Package libreadline5 not installed
Package librpm0 not installed
Package librpmbuild0 not installed
Package librpmio0 not installed
Package libruby1.8 not installed
Package libsensors3 not installed
Package libsnmp-base not installed
Package libsnmp15 not installed
Package libstdc++6-4.4-dev not installed
Package libsvn1 not installed
Package libtiff4 not installed
Package libtomcat6-java not installed
Package libvarnish1 not installed
Package libwmf0.2-7 not installed
Package m4 not installed
Package mysql-common not installed
Package mysql-server-core-5.0 not installed
Package openjdk-6-jre-headless not installed
Package openjdk-6-jre-lib not installed
Package php5-common not installed
Package python-genshi not installed
Package ruby not installed
Package ruby1.8 not installed
Package tomcat6-common not installed

Phase: initial
Correct entries:  339
Incorrect entries:  257
Total managed entries:  596
Unmanaged entries:  97

Installing Service klogd
Installing Service atd
Installing Service rmnologin
Installing Service bootlogs.sh
Installing Service varnish
Installing Service ec2-run-user-data
Installing Service snmpd
Installing Service ec2-get-credentials
Installing Service postfix
Installing Service tomcat6
Installing Service mysql
Installing Service sysklogd
Installing Service varnishncsa
Installing Service apache2
Installing Service mysql-ndb-mgm
Installing Service varnishlog
Installing Service memcached
Installing Service cron
Installing Service mysql-ndb

And here's the output to syslog from when the client was started:

Mar 22 06:56:50 ubuntu bcfg2-server[13140]: ip-10-242-199-82.ec2.internal:probe:groups:['i386']
Mar 22 06:56:50 ubuntu bcfg2-server[13140]: Generated config for ip-10-242-199-82.ec2.internal in 0.123s
Mar 22 06:56:51 ubuntu bcfg2-server[13140]: Handled 2 events in 0.000s
Mar 22 06:57:12 ubuntu kernel: Kernel logging (proc) stopped.

Any ideas?

Follow up

Posted by kcoop on March 23, 2010 at 11:05pm
kcoop's picture

I just tried this again. It turned out that, given time (5 minutes or so), bcfg2 got most of the job done, though it didn't return, and just sat there as I described above. When I waited long enough, I found apache, varnish, and tomcat running. But not memcached.

Digging deeper, I discovered that most of the configuration files had not been installed from /var/lib/bcfg2/Cfg. And when I ran bcfg2-info, I found 45 incorrect entries, including memcached's config files. So I ran bcfg2 again, and lo and behold, it got through to 'Failed to manipulate service apache2'.

I then had to stop and restart tomcat, apache, varnish, and memcached. That now done, I appear to have a working server.

Spoke to soon

Posted by kcoop on March 24, 2010 at 2:38am
kcoop's picture

I ran into a snag when attempting to restart mysql, and traced it down to a problem with the default apparmor profile not allowing access to /mnt/mysql/tmp. I'm guessing the karmic ami has apparmor enabled, whereas the jaunty one doesn't?

Observation: config.xml

Posted by jamesJonas on March 26, 2010 at 6:57pm
jamesJonas's picture

See below

Observation: config.xml

Posted by jamesJonas on March 26, 2010 at 6:44pm
jamesJonas's picture

Is there any reason for including amd64, if your running i386. Most likely makes no difference. BTW - thanks for the 'alestic' APTSource for config.xml.

<Arch>amd64</Arch>

Thanks for the updates to

Posted by pcave on March 22, 2010 at 1:18pm
pcave's picture

Thanks for the updates to support Karmic. I downloaded the latest from the repo and tried following the instructions again. Now when I start the BCFG2 client I get the following error:

Starting Configuration Management Server: * bcfg2-server
Mar 22 13:09:46 Development bcfg2-server[10111]: NOTE: Interfaces subject to change
Mar 22 13:09:46 Development bcfg2-server[10111]: Failed to bind to socket
Mar 22 13:09:46 Development bcfg2-server[10111]: Server startup failed
Mar 22 13:14:13 Development bcfg2-server[10140]: Failed to read file probed.xml
Mar 22 13:14:13 Development bcfg2-server[10140]: Loading experimental plugin(s): Packages
Mar 22 13:14:13 Development bcfg2-server[10140]: NOTE: Interfaces subject to change
Mar 22 13:14:13 Development bcfg2-server[10140]: Failed to bind to socket
Mar 22 13:14:13 Development bcfg2-server[10140]: Server startup failed
Mar 22 13:15:38 Development bcfg2-server[10156]: Packages: File read failed; falling back to file download
Mar 22 13:15:38 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/main/binary-amd64/Packa...
Mar 22 13:15:41 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/main/binary-i386/Packag...
Mar 22 13:15:42 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/multiverse/binary-amd64...
Mar 22 13:15:43 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/multiverse/binary-i386/...
Mar 22 13:15:45 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/restricted/binary-amd64...
Mar 22 13:15:45 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/restricted/binary-i386/...
Mar 22 13:15:45 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/universe/binary-amd64/P...
Mar 22 13:15:48 Development bcfg2-server[10156]: Packages: Updating http://us.archive.ubuntu.com/ubuntu/dists/karmic/universe/binary-i386/Pa...

Karmic Lessons

Posted by jamesJonas on March 26, 2010 at 6:46pm
jamesJonas's picture

Walk Path of Least Resistance
Before starting to build an EBS Boot for Mercury, create a basic server using Eric's great tutorial.
Running MySQL on Amazon EC2 with EBS (Elastic Block Store)
http://developer.amazonwebservices.com/connect/entry!default.jspa?categoryID=112&externalID=1663&fromSearchPage=true

East versus West: I'm using ami-fd5100b8 (Karmic EBS Boot) and 'us-west-1a'
[Needed to alter "Configure apt sources and get updates:" for west coast]
echo "deb http://us-west-1.ec2.archive.ubuntu.com/ubuntu/ karmic main universe multiverse" | sudo tee /etc/apt/sources.list
echo "deb-src http://us-west-1.ec2.archive.ubuntu.com/ubuntu/ karmic main universe multiverse" | sudo tee -a /etc/apt/sources.list
echo "deb http://us-west-1.ec2.archive.ubuntu.com/ubuntu/ karmic-updates main universe multiverse" | sudo tee -a /etc/apt/sources.list
echo "deb-src http://us-west-1.ec2.archive.ubuntu.com/ubuntu/ karmic-updates main universe multiverse" | sudo tee -a /etc/apt/sources.list

Gamin Trouble: Not Installing - Changed the order of installation - move bcfg2-server to the end
sudo apt-get -y install bzr
sudo apt-get -y install gamin
sudo apt-get -y install python-gamin
sudo apt-get -y install bcfg2-server

Some Config Files (still testing)

Group.xml

<Groups version='3.0'>
  <!--Note that the ubuntu-karmic option hasn't been tested as thoroughly as the ubuntu-jaunty option-->
  <Group profile='true' public='true' default='true' name='mercury-ubuntu-karmic-32'>
    <Group name='ubuntu-karmic'/>
    <Group name='i386'/>
    <Group name='amazon-web-services'/>
    <Group name='mercury'/>
    <Group name='bcfg2'/>
  </Group>

  <!-- Operating systems -->
  <Group name='ubuntu-karmic'>
    <Group name='ubuntu'/>
    <Bundle name='ubuntu-karmic'/>
  </Group>
  <Group name='ubuntu'/>

  <!-- pantheon options -->
  <Group name='mercury'>
    <Bundle name='mercury'/>
    <Bundle name='memcached'/>
    <Bundle name='postfix'/>
    <Bundle name='tomcat6'/>
    <Bundle name='varnish'/>
    <Bundle name='cacti-client'/>
  </Group>

  <!-- hosting providers -->
  <Group name='amazon-web-services'>
    <Bundle name='amazon-web-services'/>
  </Group>

  <!-- other options -->
  <Group name='i386'>
    <Bundle name='i386'/>
  </Group>

  <Group name='bcfg2'>
    <Bundle name='bcfg2'/>
  </Group>
</Groups>

Client.xml

<Clients version="3.0">
   <Client profile="mercury-ubuntu-karmic-32" pingable="Y" pingtime="0" name="WILL BE REPLACED BY GENERATED ADDRESS"/>
</Clients>

Config.xml [note the other config.xml file in this thread which includes the 'alestic' repository.]

<Sources>
  <APTSource>
    <Group>ubuntu-karmic</Group>
    <URL>http://us.archive.ubuntu.com/ubuntu</URL>
    <Version>karmic</Version>
    <Component>main</Component>
    <Component>multiverse</Component>
    <Component>restricted</Component>
    <Component>universe</Component>
    <Arch>i386</Arch>
  </APTSource>
</Sources>

BCFG2 Hanging: No solution yet

Most Recent Error

sudo bcfg2 -vq

Running probe groups
Probe groups has result:
i386
Server failure: Protocol Error: -1
Failed to upload probe data
Traceback (most recent call last):
  File "/usr/sbin/bcfg2", line 204, in run
    proxy.RecvProbeData(Bcfg2.Client.XML.tostring(probedata, encoding='UTF-8', xml_declaration=True))
  File "/usr/lib/pymodules/python2.6/Bcfg2/Proxy.py", line 60, in call
    raise xmlrpclib.Fault(20, "Server Failure")
Fault: <Fault 20: 'Server Failure'>

Mercury

Group organizers

Group categories

Post Type

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: