Best practice shared file system

Events happening in the community are now at Drupal community events on www.drupal.org.
adrifl's picture
Posted by adrifl on March 1, 2011 at 2:18pm

We run a large Pressflow site with several webservers (load balanced with HAProxy). On every server the pressflow /files directory is shared via NFS3 to a master server. We see some limits in this NFS approach and would like to deploy a new, more performant and redundant system.
There are different choices like GlusterFS, MogileFS, NFS4, Lustre, ...
What do you use and how are your experiences so far with shared file systems?

Thanks for any best practice hints.

Categories:

Comments

i have no experience but

Posted by ddorian on March 1, 2011 at 4:28pm
ddorian's picture

i have no experience but http://www.cloudave.com/21/acquia-uses-gluster-storage-for-drupal-garden...
TLDR:
s3fs offers no locking
NFS goes down even when one of their servers go down
AFS is not highly available to meet their needs
They went with glusterFS

Gluster

Posted by steve.colson on March 2, 2011 at 2:22pm
steve.colson's picture

We looked at a variety of setups as well, and ended up deciding that GlusterFS had both the lowest barrier of entry while at the same time offering reasonably high performance and very good availability.

MogileFS is really interesting, but it isn't POSIX compliant and is intended as a flat namespace, so it didn't really work well for Drupal's needs. Too many things expect to read/write directly from a "local" filesystem that need to be rewritten for something like Mogile.

NFS of course is easy and cheap, but isn't really highly available, and your options for locking, at least when I used it last, were not great in terms of either having a node hang or not necessarily ensuring file consistency (2 nodes could overwrite).

image rendering performance

Posted by mikeytown2 on March 2, 2011 at 7:38pm
mikeytown2's picture

I was able to significantly speed up my page generation times with this module
http://drupal.org/project/imageinfo_cache
Prevents disk hits when looking up the images width and height. If your a media rich site, it should give you a noticeable speed increase.

GlusterFS+

Posted by geewiz_ on March 3, 2011 at 8:52pm
geewiz_'s picture

For the clusters we offer with our DrupalCONCEPT hosting, we use GlusterFS. After some effort for performance tuning, we're very happy with the choice.

Best regards,
Jochen

Jochen Lillich, CTO freistilbox Managed Drupal Hosting

subcloud

Posted by chicodasilva on May 9, 2011 at 4:40pm
chicodasilva's picture

Has anyone implemented a solution based in subcloud? Any experience?

Best Regards,
Francisco

High performance

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: