Hi,
I wanted to get feedback about whether releasing our module sounds like a good idea.
Our company has developed a module that publishes static Drupal nodes to S3 as they are saved or during a full site crawl. We've used this on a few projects where there is a need to cope with large traffic spikes (e.g. websites that support TV broadcasts) and where the content is relatively simple but there is still a need for content management. So quite niche but hopefully useful to some.
I've had a good look and can't find anything that does the same. The Boost module, which was our inspiration in some ways, didn't provide exactly what we needed - complete separation from Drupal (never hitting it) and the Boost module crawler is deprecated.
There's more detailed information here http://numiko.com/labs/2012/12/static-drupal-sites-for-high-traffic/
A future improvement could be to allow destinations for the static files other than S3 (pluggable modules).
thanks
Andy
Comments
High Traffic related with Performance, sounds very interesting!
Hi Andy,
Thanks a lot for your contribution and letting us know about this module, it certainly sounds like a great idea.
Just a quick reply to suggest you might consider cross posting this page in the High Performance DUG, I would surely be interested in hearing if any guys in this group would have additional ideas, experiences or questions.
Future evolutions surely sound very promising.
Thanks!
Hi @Andy, Thanks for
Hi @Andy,
Thanks for feedback.
I think your module can be very helpful and it will be great if you share it.
Once done, you can update the page "Creating a static archive of a Drupal site" on wiki : http://drupal.org/node/27882
As an alternative to your project, I saw (but never tested yet) the "Save to FTP project" : http://drupal.org/project/savetoftp which looks to have an S3 option.
Some Ideas
"drupal_http_request() with an added querystring parameter"
Use a httprl non blocking request and set a custom header or user agent instead of using a query parameter.
Also look at the expire module for some simple crawler logic.
Update?
Hi Andy
Just wondering if you have released this module to the community?
I'm looking for something that does exactly what you describe to push content to S3.
Thanks
Hi, unfortunately we haven't
Hi, unfortunately we haven't been able to go any further with this due to being too busy but hopefully soon.
Also interested...
Would be a great addition...
we're looking forward to it being released!
Any word on releasing the module
I am also interested in using the module. Would it be possible to create a sandbox project to house the module?
very helpful
Hi,
It will be very helpful if you share the module.
Naveen Valecha
https://www.valechatech.net