Running Drupal as daemon service

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Introduction

PHP Frameworks are becoming more and more complex, specially now that almost all of them are fully object oriented. In serious applications huge amount of code needs to be interpreted and loaded into the memory for every request, it's a needles waste of huge amount of resources.
But what if we could serve requests not by using traditional request/response method but by using sockets and asynchronous calls?
What if each request needed less than 500kb and tiny amount of CPU to be served? How many more requests could be served on the same servers? benchmarks for other frameworks show that in a perfect and fully compatible environment it can be even more than 2000x more!
We all love Drupal, and we all know that it's slow! sometimes even very slow, but that's the price we're willing to pay for extreme flexibility. But can't we have both the performance and flexibility? there seems to be more and more options becoming available now days like HHVM and several different PHP compilers, opcode caches, etc. But none of these solutions - although extremely effective -, are game changers. We're looking for a solution that can make Drupal fast, really fast

Title

I'm not satisfied with the title i chosen for this page, it has to clearly announce what this is all about, if you have a better idea please recommend. After all this is the purpose of wiki :) I hope anyone who's interested in improving Drupal's performance joins in and share their experiences

PHP Asynchronous libraries

ReactPHP

ReactPHP take a very different approach to PHP's performance problem, it's more memory efficient and can be used in combination with HHVM.
It's i believe how it should be, Drupal as we all know has a very expensive bootstrapping specially when many modules are installed so the performance boost we get from using ReactPHP is amazingly high! it's not comparable to any other solution available right now 1000x - 2000x boost and even more if the application is written in an asynchronous way.
Sine Drupal 8 is service based and using symphony components as core it seems possible and within reach
Also event driven design is not only for ReactPHP, there are other solutions and more will come in the future
It would be great if we could have opinion of some of core developer regarding how difficult this task is and how we can help to make it happen
Want to know more? Bring High Performance Into Your PHP App (with ReactPHP)

Drupal as daemon service

Not much options available currently but i could find few

Benchmarks

We wouldn't know how Drupal beforms as a daemon service but here are some benchmarks for other frameworks and also comparisons with nodejs.

PHP Standalone Multi-thread persistent WebServers

Sample codes for performing benchmarks

WebSocks and Push Technology

There are also libraries for supporting push technology (web socks) using reactphp which eliminate the need for nodejs. Since it's already in PHP this support can be added to Drupal out of the box.

Pushing to Core?!

It all started by an issue opened by @giorgio79 , although i think it's still to soon for that but here is issue https://drupal.org/node/2218651#comment-8655321

Comments

Hi, Some years ago (~2011) I

Sorin Sarca's picture

Hi,

Some years ago (~2011) I made some experimental projects to improve drupal response time.
The whole idea was to get the page response (or to call a function) by skipping Drupal bootstrap because it takes too long. There were several useful ways of using this response:

  • with Server Side Includes (SSI) or with ESI to replace dynamic parts from a page (while the rest of the page is cached)
  • ajax calls
  • webservices
  • and others

My solution was to create a PHP extension that will allow me to create a daemon, and the clients could talk with daemon via IPC.
The results were very good (I cannot find the banchmarks now, but I know that concurrent requests >= 100), and I remember that an ajax call loading a node took 6ms instead of normal (with bootstrap) 100ms.

This project was posted to github https://github.com/sorinsarca/ipcs and contains some examples including Drupal 7 (I don't know if extension still works with current versions of php, but it should work with php 5.3).

Back then I also tried with php sockets, or with mongodb as bridge between client and daemon (and others), with no good results.

Now ReactPHP seems a good idea and I'll be glad to help you developing a solution.

Thanks for the info, so

sinasalek's picture

Thanks for the info, so actually tried doing something similar and it's a proof that generally it works and the result worth the effort.

An interesting slide compare

sinasalek's picture

An interesting slide compare nodejs and reactphp
http://www.slideshare.net/vanphp/react-php-the-node-js-challenger

Drupal module for this

mikeytown2's picture

https://drupal.org/project/httprl/ - Has async event loops and allows for threading as well as parallel downloading. Doesn't have the daemon service setup.

non-blocking I/O

SameerJ's picture

How much of a performance boost would non-blocking database I/O give us?

If you can use reactPHP to avoid bootstrapping for each request, and then avoid the rendering process by using REST responses, then it seems that the main bottleneck would be the database I/O.

async db

mikeytown2's picture

Using async database queries speeds up writes by a ton; effectively free writes. Reads work best when you prefetch. See https://www.drupal.org/project/apdqc and the list of Similar Modules like it for some ideas.

apdqc currently does the cache, locking, & sessions reads/writes/deletes in an async or prefecting way.

The trick with async is you get the data while other things are getting processed in the background. HHVM finally added in async recently (3.6) http://docs.hhvm.com/manual/en/hack.async.php so using this you can do what httprl does and more.

Talk at DrupalCon Amsterdam 2014

SameerJ's picture

Talk at DrupalCon Amsterdam 2014: "Accelerate Drupal with ReactPHP" https://amsterdam2014.drupal.org/session/accelerate-drupal-reactphp

I haven't found an online video of this talk.

looks like it was proposed

mikeytown2's picture

looks like it was proposed but not accepted. Here are all the videos from it: https://www.youtube.com/playlist?list=PLpeDXSh4nHjQBf_SOdQgY-k6TnbCn3pSj

@mikeytown2 Which one it's?

sinasalek's picture

@mikeytown2 Which one it's? there are so many videos.

No Video

mikeytown2's picture

It was not accepted as a talk thus no video as there was no presentation given.

We that's really strange!,

sinasalek's picture

Well that's really strange!, any slides? BTW a production ready PHP App server has recently became available http://appserver.io, if only Drupal could run under it

React PHP

ravismula's picture

Has anyone integrated ReactPHP with Drupal 7??

I started working on Drupal 8 + PHP-PM + ReactPHP

kentr's picture

https://github.com/kentr/php-pm-httpkernel/tree/kentr-drupal-bootstrap

PHP-PM daemonizes PHP and starts up a ReactPHP loop. This is the setup described in Bring High Performance Into Your PHP App (with ReactPHP), which looks to be getting good support from the Symfony / Laraval end.

There are some issues, but a basic integration is there.

Here is what seems like a

GiorgosK's picture

Here is what seems like a ReactPHP integration https://www.drupal.org/project/daemons