So, I've been using Aegir for years (I'm the adrinux that drew the ugly diagram mig5 uses in his Aegir workflow blog posts), in short I'm familiar with the Aegir/drush_make workflow. For my use of Aegir I'm a lone developer hosting a handful of Drupal sites, so a very simple setup, but not exclusively Drupal. My server also hosts a handful of static web sites and some playing with node.js. It has to handle all of this.
I need to upgrade the VM I use for development (and later my production server to match) since the OS LTS release is coming to the end of its life. I want a fresh build and thought I'd switch to nginx/maria and the most obvious choice for that with Drupal is BOA.
I've managed to get BOA installed on a VM, and it seems to be working ok. But what next. I have a bunch of sites (both Drupal and static), git repos and assorted other clutter to move across from the old VM.
What's troubling me is where to put the Drupal sites – my impression from hours of reading is that it's OK to put them under Barracuda/aegir umbrella in /var/aegir/platfoms and carry on as I normally would with Aegir. But various bits of documentation and welcome email all talk about Octopuses' /data/o1 directory tree
If I want to use my own platform builds with standard Drupal core do I just use /var/aegir/platforms or is Barracuda dependent on Pressflow for D6 sites? It's not at all clear to me how customised Barracuda is.
Hoping someone can clarify things for me. Octopus docs seem quite good, Barracuda almost non-existent...
Comments
Hey Adrian, I always rename
Hey Adrian,
I always rename my octopus user to something other than o1 but thats probably just being paranoid. :P
I store my production sites (that upgrade normally through a migrate task):
/data/disk/o1/distro/00x/The default location for automatically created platforms that you select in Octopus install file /root/.o1.octopus.cnf.
all the modules for all sites to:
/data/disk/o1/distro/00x/BUIlT-IN-PLATFORM/sites/all/modules/unless I have custom modules then I store them in the sites module folder directly restricting other sites from seeing the module at all:
/data/disk/o1/distro/00x/BUIlT-IN-PLATFORM/sites/SITENAME/modules/I store custom platforms in:
/data/disk/o1/static/CUSTOMPLATFORM001I use custom platforms with sites that have special codebase requirements where perhaps they have not been updated for what ever reason, perhaps have complicated module combinations and/or have a lot of modules that could break in an upgrade, and probably other various complications that don't upgrade well with a migrate task. Can be useful for testing core upgrades before BOA releases them officially.
Some odd sites I haven't built but do host and the above still applies so custom platforms fit perfectly as they don't interfere with other sites codebase and exporting their "make" config and rebuilding the updated platform is somewhat easier to manage.
Another use case situation for custom platforms could be allowing other developers access to sites only on that specific platforms. Suppose there's lots of scenarios once I get thinking about it :)
I'm hoping that makes sense and is of some use? Do let me know how you get on.
Cheers
hmm.
Interesting to hear how you're using it but still doesn't answer my main question: do I need Octopus? Why shouldn't I just use barracuda and put my sites in /var/aegir/platforms?
I get that Octopus shares code and that gives some benefit.
Barracuda doesn't have any
Barracuda doesn't have any pre-loaded platforms, you must build them all yourself (custom platforms).
Therefore not setting them up the same way that Octopus sets up the shared code, therefore consuming more memory and disk i/o.
Out of the box Octopus optionally can set up around 20 something platforms of custom distributions that to date have been some what tricky to install or require some special install steps to get working as well as patching etc etc.
Octopus sets things like Commerce Kickstart 2.0, and Open Atrium all of which allowing you to deploy sites on those platforms as though everything were all preconfigured for you. It's all built in.
Octopus offers significant performance enhancements with special built in caching (redis speed boost, advagg) and additional performance related modules all automagically linked to your built in platforms and after 24 hours of being created applied to your custom platforms as well. This doesn't occur on just barracuda alone.
Need I say more about the difference?
Adrinux, there's so many
Adrinux, there's so many benefits to using the BOA/Octopus provision scripts to build your hosting environment. Coming from hosting our Drupal sites on an nginx, aegir and multi-site environment, it was clear the architecture of BOA was a perfect fit for us. It's an extremely well thought out provisioner that can save hours if not days of work. BOA uses the multi-site/shared code model of hosting Drupal. This uses less memory.
Maybe you'd like to try out what I did, which was to rent out a vps with one of the providers that charge by the hour and run the script there. It helps to play around with it on a live environment. Then migrate some of your production sites into a dev domain on the BOA provisioned vps and play around with it.
I love the email login notices, package update notifications, automated blocking of IPs, file integrity checks (to see if any important files on the server have been modified), new relic integration (we use the free service) and the fantastic configuration of Nginx. In particular the disabling of all caches on dev.* aliases are a real winner. Mobile device detection at the server level and cache sensitivity to different groups of devices come built in! I must have spent 3 days last year getting this right and even that effort does not come near what BOA does. For backups we use duplicity and a separate S3 bucket for every platform.
I'd suggest running Octopus after Barracuda and put all your sites under the Octopus user. You may even want to create a separate Octopus user for production only sites. We place our custom distro in /data/disk/[octopus-user]/static and like snlnz, our older D5 sites in /data/disk/[octopus-user]/distro/00x/BUIlT-IN-PLATFORM/sites.
Platforms installed via the Octopus script have a shared code base, so all other Octopus instances will share the same code. The only finicky thing to remember here is not to create a site with the same domain from two separate aegir instances.
Setting up a hosting system on BOA and Octopus is much easier if access to the server and backups are limited to your team. It works perfectly for us because it's just myself and two developers on our team that manage all our clients' websites.
I'm sold already
You don't need to sell me on it Mojah, and as I said I already have it installed in a VM.
My main question remains – is it OK to use /var/aegir/platforms just like a normal Aegir install and ignore Octopus?
My thinking right now is just to move my sites across from my existing Aegir install to Barracuda-Aegir, and to worry about Octopus later, when I do a new site build or if I upgrade the couple of D6 sites I still have to D7.
I think today's task will be taking a VM snapshot and diving in, always the best way to learn what's possible.
Octopus also provides a level
Octopus also provides a level of enhancements and also provides a level of isolation between the core and hosting.
I asked the same question when I started with BOA and was told that there are extra things in Octopus. It was heavily recommend that I use both layers.
I install octopus without any extra platforms - I only use my own custom platforms - but I do so within Octopus.
The separation also allows a server software update - barracuda - without needing to touch the hosted sites and associated software.
Recipe for duplicity backups with separate S3 bucket/platforms?
Would you mind posting or linking to your recipe for setting up such a backup system?
Thanks!
Hi adrinux You could use the
Hi adrinux
You could use the normal aegir instance instead without any problems, if i'm not mistaking, in the past, octopus was optional
some advantages of using octopus on the other hand are:
octopus provides ssh and ftp access on a "client" node bases, however clients should never get access to this :-) it can be handy for yourself to get easy ftp access to your files folders
the settings file contains a lot of "optimalisations" for different platforms or environments, eg. error reporting, caching on/off based on keywords in the url (dev.mysite.com), redis cache integration,...
some standard modules for performance (don't think these are also in the master aegir, but could be mistaken) eg. boost, speedy, etc modules are available in your drupal 7 site
It provides 3 platforms for each drupal version (dev / staging / live) where you can migrate from/to in your development workflow
cdn domains integration
Grts
TimLeytens
not sure about stages
I'm not sure about stages in octopus, sure there are Drupal platforms suffixed with "-dev", "-prod", but their code, configuration, is the same.
What makes dev and prod different is if you set another alias with the subdomain dev to the same prod site.
Drupal would then read that there's "dev" snuck in front of the hostname, and disable most of the caching.
ok, thanks.
I know octopus has benefits – everybody is trying to sell me on it in every comment! - thanks for answering the question though. I'll go ahead and move my existing aegir hosted sites over to barracuda. I need to get things set up quickly and get working again and that seems easiest. I can learn all about Octopus later.
I'm not trying to sell you
I'm not trying to sell you anything ;-) I'm merely trying to explain the difference between the 2.
On the other hand, the whole barracuda/octopus stack you just installed looks like a lot of overhead if you are only using the master aegir instance.
why not just install manually or through debian packages.
Grts
TimLeytens
Read more carefully.
I get the feeling no one is reading my posts properly.
As I said in the original post, I want to switch to nginx/mariadb and barracuda has that well sorted. The speed optimisations are also interesting. And as I said in my previous reply: I want to move my existing setup as quickly as possible, so using barracuda aegir rather than octopus to begin with. Running BOA.sh is also a lot faster than manually installing a whole server stack :)
But I just got frustrated not being able to browse the filesystem and discovered barracuda does:
chmod 711 bin boot data dev emul etc home lib media mnt opt sbin selinux srv sys usr var &> /dev/null.Makes some sense to lock down a multi-user server, but I'm just me, no one else will be accessing the server. Starting to feel that BOA is overkill for me again.
sudo for drush
You could setup /etc/sudoers to let your user to sudo as o1 and specifically run drush.
Then do
sudo -Hu o1 -g users drush @mysite statusas current user, or setup a bash alias to make it shorter.Thanks.
In fact, I think I'll carry on with the manual build I started last week. Thanks for answering, and consider this question closed.
Hey Adrinux, there a multiple
Hey Adrinux,
there a multiple linux accounts you can use (at least when you use octopus).
MAINUSER.ftp, MAINUSER.client or MAINUSER
I think it's recommened to use: .client or .ftp
To get a proper shell:
su -s /bin/bash MAINUSER.ftp
I don't see any reason to not install octopus since it takes 2 minutes to install. If you ever need another developer working on your machine everything is set up perfectly. Just don't install any shipped platforms and use the static folder to deploy own platforms. It will take some time until you get used and understand all the magic which happens under the hood like fixing permissions and stuff but you get a lot back. There was no urgent need for BOA users to install the latest security release of drupal since install.php was blocked already...
Sorry didn't saw your
Sorry didn't saw your reply... If you want to start from scratch but want to stick on nginx this could be interesting: https://github.com/perusio/drupal-with-nginx
Good luck!
@muschpusch
Thanks. That does look very useful indeed.
We're using barracuda without
We're using barracuda without octopus as well, for the installation you the BARRACUDA.sh file, not the BOA one, but don't forget to edit it first. It should be in /usr/local/bin/
We place all our platforms inside /var/aegir/platform and depending on our needs we just give the aegir user a shell. But most of the time we use an internal aegir server as main and use the others as remote servers.
On some servers we're running custom php sites, piwik, gitolite, gitlist without any problems.
The speed improvements that are part of octopus are also part of barracuda.
Not all of the speed
Not all of the speed improvements are bundled in both, so that part is incorrect.
And while we used barracuda for quite some time before deploying Octopus, I was gob smacked by the performance difference of using an Octopus built in platform vs a custom platform on Barracuda alone so it's worth doing some performance/unit testing on disk i/o, memory consumption and general page load times.
New Relic is a good start for benchmarking this.
@adrinux
It is great to see you here!
Here are some things to consider:
Entire BOA stack is designed (for many reasons) to use Octopus Satellite Instances for your sites, even if it is possible to use Master Instance created by Barracuda only.
Using Barracuda only (and configuring it manually before running the installer) will not save you any time - it will even add more confusion if you are not familiar with the system yet.
You don't need to read docs to be able to use it properly, just fire up BOA meta-installers, and you will be grateful later that you didn't skip Octopus.
There is no reason to run either Barracuda or Octopus installers directly - unless you have too much time to spend on digging into its internals and some more advanced settings.
There is no overkill in using full stack and installing it with command line wrapper (boa) and then upgrading with wrappers (barracuda and octopus). It is there to save you time and effort.
We should probably write some once and for all explanation to end these "Barracuda or Octopus" debates, but in the meantime take a look at some differences to realize that you don't really use BOA if you are using Barracuda only - we list here modules and extensions included in both Master and Satellite, but also those installed only in Satellite (Octopus):
http://drupalcode.org/project/octopus.git/blob/HEAD:/README.txt#l283
Also, it was discussed here: http://groups.drupal.org/node/200783, and most of pros I have mentioned are still valid: http://groups.drupal.org/node/200783#comment-662488
For example - only Octopus comes with Redis cache integration - even this one detail should be a reason to use BOA as a complete stack and forget about trying to make it pseudo-simple by skipping Octopus. There is no reason for this and many reasons to use the stack as designed.
@omega8cc
Thanks for taking the time to reply to my questions.
I currently have a mix of 7 drupal 6/7 sites, static html sites, and a static site with a smatter of php. I'm the only server admin/developer/site builder/drupal admin on these sites. None of the sites has large numbers of logged in users and all are relatively low traffic.
So I don't need half the stuff BOA offers: client access to aegis, client access to ftp/sftp, lshell and so on.
I'm also a bit of a control freak and not comfortable with all the stuff BOA installs and configures that I have never used and don't even know about. BOA is slick, but it's a slick black box until you've spent a lot of time learning what its doing.
I want to do 'crazy' stuff like mount the platforms folder via NFS so I an edit files directly in my host OS. Not strictly incompatible with BOA, but attempting to mount the shares yesterday it seems it was blocked by csf by default (sensible) but that led to:
I was finding it maddening when looking at /etc or /var of /log trees and getting permission denied unless I used sudo, and switching to root just to traverse the filesystem feels wrong. I understand why BOA locks down the filesystem, but it's overkill for my personal dev vm.
BOA is very slick, and very fast. But for me personally it's not a good fit right now, too much stuff for a dullard like me to learn at once. And I like to know what's going on, how things are working.
I'll likely end up slowly duplicating half the functionality manually, but in doing that I'll gain a better understanding. There's a good chance I'll be back trying again in a year or so, if I know myself at all :)
@adrinux
You know, some people who remember Aegir when it required so many steps with install wizard and command line to get it up and running, say that it is no longer a fun, because now you can just type "apt-get install this-stuff" to get it done.
BOA also started as a simple shortcut-install script, mainly because there were no packages for php-fpm and Nginx, there was that crazy web based wizard with its hundred of steps to get Aegir installed, and most of that system level stuff required building it from sources. And still, it was kind of fun anyway. You could still feel like a mechanic who builds this thing with his own hands, just with some helpful tools.
Now you can install BOA also with simple, one line command. And some people complain again that it is no longer a fun! And it is normal.
Some people still would prefer to code their own CMS from scratch instead of digging into all that crazy Drupal complexity. Some prefer to start with ready to use distro, just to avoid that nightmare with comparing hundreds of modules and trying if they could really work together. And it is also normal.
People have different priorities. Some love to work and learn as their own sysadmins, performance and security experts, and some prefer to get a ready to use system, get it done in 15 minutes and be free to do other things their prefer. And it is also normal.
Both sides of this story - the work and the fun, are equally important, so I fully support your decision :)