Problems with CRON since 2.2.2

Posted by geoff on April 14, 2014 at 12:17am

I am having some issues with the CRON not running for my octopus sites since upgrading to 2.2.2. I was wondering is anyone else is having the same issue.

I have even created a completely new install on a blank server and am getting the same problems.

If I su as the octopus user and try and run hosting-dispatch directly, it says that the command is unknown. However, it appears in the list of available drush functions.

Does anyone have any ideas to test this out to see what I have done wrong or determine if it is actually and issue and needs to be put on the issue queue?

Comments

Have you tried (as root) bash

Posted by ar-jan on April 14, 2014 at 9:35am

Have you tried (as root) bash /var/xdrago/run-USER? Updated your Octopus instance as well? See https://drupal.org/node/2229715 #4 and #7.

Thanks ar-jan, I have just

Posted by geoff on April 14, 2014 at 9:56am

Thanks ar-jan, I have just tried that command now.

On the new system installed fresh using 2.2.2 I get

load is 2 while maxload is 300
...now doing CTL...
The drush command 'hosting-dispatch' could not be found. [error]
CTL done

And on the old system that has been upgraded to 2.2.2 I get

load is 21 while maxload is 288
... now doing CTL...
/data/disk/sites/aegir.sh: line 4: /opt/local/bin/php: No such file or directory
CTL done

These are different errors, so that is useful to know. Thank You.

In my research today, I found others are having a similar issue.

https://drupal.org/node/1532570

From this I was made aware to check the "master" install and the queue is working fine there. So it is an issue only with the octopus instance.

Same here

Posted by attiks on April 14, 2014 at 10:03am

I ran into the same problem, I edited /data/disk/sites/aegir.sh to change the path of the php

I had a similar problem, but

Posted by jeremyr on April 14, 2014 at 12:56pm

I had a similar problem, but instead of editing aegir.sh I created a link from where aegir.sh was looking to where php was moved to. Everything started running again after that.

Not sure why the upgrade doesn't misses that. Had to do it on two servers.

This is crazy - I've been

Posted by geoff on April 14, 2014 at 1:56pm

This is crazy - I've been clicking and changing and testing all day. Even a brand new install of BOA fails with the same issues.

May I ask what path you are putting in for the new version of PHP. I have tried a few different settings. They all work as PHP commands. But I still get 'hosting-dispatch not found'.

Thanks

I have /usr/bin/php

Posted by attiks on April 14, 2014 at 1:59pm

I have

/usr/bin/php /data/disk/o1/tools/drush/drush.php '@hostmaster' hosting-dispatch

and /usr/bin/php lins to /opt/php53/bin/php

Thanks attiks, It turns out I

Posted by geoff on April 14, 2014 at 2:12pm

Thanks attiks,

It turns out I had those settings already. And it still does not work.

Thank you very much for your reply.

On to the next possible solution.......

Me too...

Posted by jim kirkpatrick on April 14, 2014 at 7:37pm

So after 2.2.2 upgrade (via 2.2.1) I have no tasks running in o1 instance (but fine in main Aegir instance) AND random load spikes that need me to log in and restart all web-serving related services.

I've reinstalled 2.2.2 twice, no fix. Am now trawling though code...

If anyone has any ideas here, would be very grateful. Main cron fine, o1 tasks not fine, also get The drush command 'hosting-dispatch' could not be found. [error] :(

EDIT: In short, I've got exactly the same symptoms as @geoff - PLUS some lovely load spikes... server running BOA for 2 years.

We are running BOA-2.2.2 in

Posted by omega8cc on April 14, 2014 at 9:13pm

We are running BOA-2.2.2 in production on thousands of hosted instances with Debian Squeeze, several instances with Debian Wheezy and many instances with Ubuntu Precise on various parent systems and hosts managed remotely. And there are zero issues like that (just some known issues with Compass Tools, fixed in HEAD).

Please open proper bug report(s) with all required config and log files attached, so we could try to find the source of your problems, since we simply can't reproduce this.

Also, make sure you have read and followed updated docs: https://github.com/omega8cc/boa/blob/master/docs/UPGRADE.txt

Hi Omega8cc... I was on my

Posted by jim kirkpatrick on April 14, 2014 at 9:21pm

Hi Omega8cc...

I was on my way to submit a bug report, but wanted to do some background research first...

You can look forward to such a post tomorrow! Thanks..

Jim

Fails on clean install

Posted by geoff on April 14, 2014 at 10:58pm

I have just completely reimaged a VPS with Ubuntu Precise (12.04) and installed BOA 2.2.2 from scratch.

This problem occurs on this newly installed image.

This is the third time I have attempted this and every time I end up with the same problem. [I think that is the definition of an idiot - someone who does the same thing the same way and expects to get a different answer?]

Master instance is running fine. Satellite CRON fails. "hosting-dispatch not found".

I will a new install on a clean Debian base system and see if that has the same problem.

Is it possible that it has something to do with the new feature that does not allow drush commands from contributed modules?

Fails on Debian

Posted by geoff on April 15, 2014 at 3:48am

I have just done a fresh install on Debian 7 (Wheezy) blank image and am getting the same error.

Works on master

Fails on satellite

Fails on Ubuntu at AWS

Posted by geoff on April 15, 2014 at 10:20am

Just to make sure it was not an issue with the VPS provider, I created a new blank Ubuntu 12.04 image at AWS and tried the fresh install again.

Same problem

This is definitely a problem with the 2.2.2 release. Time to add to the issue queue.

We have just tried another

Posted by omega8cc on April 15, 2014 at 12:42pm

We have just tried another clean (stable) install on Debian Wheezy 64bit at DigitalOcean, using the smallest 512 MB VPS and it just works with zero issues. No idea where from your problems are coming, so please submit report in the issue queue with all relevant details/logs/config files attached.

Thanks Omega8cc Issue entered

Posted by geoff on April 15, 2014 at 1:33pm

Thanks Omega8cc

Issue entered here https://github.com/omega8cc/boa/issues/301

I have same problem

Posted by aanjaneyam on April 15, 2014 at 10:29pm

I have the same problem with long running stable wheezy system upgraded to 2.2.2. Verify etc. tasks don't run on sites in satellite instance. I saw a fix related to it on github. Is this fix is a hotfix to stable as this seems to be a major bug.

What needs to be done?

There is no bug and hence no

Posted by omega8cc on April 15, 2014 at 10:44pm

There is no bug and hence no fix. The only issues we have heard about were caused either by not following the upgrade docs for major upgrades or by using restricted system user names for the octopus instance (like "sites")