when you call a telephone service and all operators are busy then you get a message to wait in line or call again latter, can we implement this for php as well?
correct me if i am wrong but when request for php application reaches max execution time then the script dies and never respond to the browser
so while the browser shows continues loading state, the server side may have already died long ago and not really continuing processing
but site user will continue to press reload page or waiting unnecessarily long time for page to load.
even if server side actually still processing and not died this is an unwanted situation and it is better to select to serve few users and display message for others to come back latter
so i though about several options:
measure average resource consumption for particular page and on drupal bootstrap calculate available resources and if found inadequate redirect user to an error page reporting site is under heavy load and try again latter or register for email notification when high load is over
register php error handler : http://www.php.net/manual/en/function.set-error-handler.php
but i couldn't see how to handle only performance errors and pass all others to php normal error handling
wrap drupal main execution in try-catch block and define performance handling hooks inside the catch segment using module_invoke_all
have anyone thought of these ideas before and implement them in other environments ?
Comments
Nope.
I wouldn't say so. First of all, estimating the resource consumption of a particular page out at a level (say, individual paths) that would be able to save significant effort would be extremely error-prone, as there are many factors that will dramatically affect resource consumption/response time that are unknown at that point.
Even if that part were easy, estimating the available resources is also far too difficult, because there are any number of resources that may be in contention: disk IO, bandwidth, locks on a specific database table or even row.
It's not as simple as page "X uses Y resources, we have Z, is Z > Y?" because Z and Y are complex sets which can't easily be known in advance, and knowing X is mostly useless because Y varies too much for any given value of X.
proper response
what about responding to user with error page instead of dieing when reaching max execution time
Complex Issue
This is a fairly complex issue. Ideally it sounds like you're looking for an architecture similar to NetFlix and Chaos Monkey.
Right now what you can do is have your load balancer display a nicer message if it's taken more than X seconds to load a page (or even apache). That way you have a nice looking message when things take too long. This still requires the browser to wait for the timeout.
If you have too many database connections, Drupal will fail fairly quickly; so adjusting the max # of connections might be something to look into.
Long term and more closely aligning with the Chaos Monkey, you can start to think more about ESI and how to utilize it more. ESI starts to break the webpage into smaller chunks that can then be assembled and if one chunk is taking too long you might be able to bypass it.
A different way of thinking about ESI is to parallelize it. This is one of the goals of the HTTPRL project/module; code execution in parallel. An example might be If a ESI sub-request takes longer than X seconds skip it and use a generic placeholder, for that bit of content, instead of something unique. That placeholder could be code to AJAX load that bit so the browser gets most of what you want and will load the slower parts later on; or it could be a old version of it from the cache, or something else. The idea is if something is taking too long to do, have the ability to timeout on it and do something else.
Finally I would profile your site and see why its so slow sometimes. Is it disk I/O, CPU, Network, SQL, etc; understanding what is the biggest bottleneck will help you out the most the quickest.
not slow
no matter how much optimization and profiling you do, if large enough amount of users visit the site then it will fail to respond in timely manner
so what i am saying is even at processing load the site still should respond to browser and say : "i can't respond now, come back latter .."
and what i would like to ask about is what happens when max execution time is reached? and why can't we program the site to identify that event and "handle it" instead of dying without ever responding to browser request
maybe i don't understand "max execution time" enough ..
You can handle it
Stackoverflow has a good answer: http://stackoverflow.com/questions/3156093/max-execution-time-error-hand... Looks like it can be done, but based off of what you've said limiting the database connections and failing early is what you would like to do. http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysv...
Separate issues
It sounds to me like you are confusing two related issues:
1) It takes too long to serve a page to the user
2) There are too many concurrent users for the server to handle
Page load time can be affected by many factors - server load as you've mentioned, but also poor coding, large data sets returned from a database, slow database server, faulty hardware, scheduled tasks being executed, database tables being locked, 3rd party services responding slowly and lots more reasons.
If your problem genuinely is an overloaded server, then going to all the effort of building half a page and then throwing it away because you've got bored is a waste of server resources. A much better approach is to identify that the server itself is overloaded and reject (or queue) the request before you start processing it. Have a look at http://httpd.apache.org/docs/2.4/mod/worker.html.