Server overwhelmed by Menu

trainingcity's picture

Had a strange problem today that took my entire CentOS LAMP server offline. "top" command showed httpd using almost 100% of my CPU & a massive memory consumption within moments of starting up. "free -m" command would show the httpd process spiral up to consume all the RAM in less than one minute.

Finally isolated the problem to the menu. Used devel rebuild menus to recover site. No amount of cron, cc, rr, reboots, etc would work. It was difficult to even get to this command as the server was almost completely unresponsive. Going to try to isolate the problem, could be related to either the fact that I removed "use tokens" on a number of my "main menu" items that were, in fact, not using tokens, or the addition of several menu links to views pages?

In any case, I wanted to post this here as a reminder that no matter how much time you spend on performance, sometimes the thing that is affecting your site is not apache prefork vs worker, the php.ini memory settings, or anything else that can be controlled.

I was getting a menu related error in my drupal logs that alerted me to the possibility.


Menu problems consume CPU & RAM

trainingcity's picture

Just in case someone stumbles across this post while troubleshooting, the problem I was experiencing may have been caused by changing content types while having node/add menu links in a site specific menu:

Once again, this is just an example of how we have to be cautious when troubleshooting server performance, sometimes the issue is not the amount of RAM you have available or your my.cnf config, it is something much more basic, a broken site!

Hello there, D7 has a

MisterSpeed's picture

Hello there,

D7 has a situation where if the menu needs to be rebuilt the next visitor's visit triggers that rebuild. If by the time the second visitor comes and the menu is still not done rebuilding, it triggers a rebuild as well, and so on until a cascading effect happens where the sum of all the processes doing essentially the same thing overwhelms the server.

We solved that by recoding the rebuild process at the cost of immediate content coherency: the current menu is kept until the new menu is built (only by cron callers, not by visitors), and then it gets inserted as a whole in the DB. It was a rather drastic change to core however (we had an exceptionally slow Oracle back-end starved of resources to deal with in a governmental set-up) with Oracle-specific tricks so we never committed it back to core. The solution to your problem is in that method however.




mikeytown2's picture

"It was a rather drastic change to core however"

Being able to see what the changes are might be helpful. Sometimes a massive core change can be switched to be a small patch with some different function calls. Example: