Massive D7 performance improvements - community testing needed

Posted by yched on January 2, 2013 at 6:49pm

The _field_info_collate_fields() memory usage issue has a D7 patch that brings massive performance enhancements, by implementing more granular loading of field and field instance definitions.

The exact impact depends on the number of entity types, bundles and fields. An an example, a core setup with 10 node types, 50 fields on each node type sees:
- frontpage with 10 nodes: -5% CPU, -25% MemUse
- node/%nid: -22% CPU, -60% MemUse
That's a bare core install, with a fairly large number of fields. Numbers on an actual site, with additional contrib modules taking part in the page generation, will probably be lower - still the improvements should be quite noticeable.

Special care was given to ensure that the changes maintain strict API compatibility.
However, as David Rothstein states, "this is one of the bigger and more far-reaching patches we're ever going to commit to D7". D7 maintainers would feel more comfortable committing this with a confirmation from the community that the patch does not break anything on some real-life, real-size sites.

I'm specifically thinking of large drupal distros or products: Acquia Gardens, Commerce Kickstart, OpenPublish, OpenPublic, Panopoly...
Just apply the patch, clear caches, and check that the site still works :-) (extra bonus points for products with a test suite of their own...)

Next "D7 bugfix release" window is Feb 2nd. Let's get this in !

Comments

obrienmd

Posted by obrienmd on January 2, 2013 at 8:11pm

Tested, looks great! Thanks for the reminder / advocacy on this issue.

I'll try reviewing again,

Posted by geerlingguy on January 2, 2013 at 9:37pm

I'll try reviewing again, though not with a site with as many field instances this time (that site, which had over 10,000, was refactored to fix the myriad other performance problems such a vast number of entity types and instances were causing).

Over 10k field instances?!? I

Posted by obrienmd on January 2, 2013 at 9:42pm

Over 10k field instances?!? I would love to see a case study :)

I think I may write one. It's

Posted by geerlingguy on January 2, 2013 at 10:08pm

I think I may write one. It's for Flocknote, where we were setting up new profile2 entity types for each Network added. Quickly got out of hand when our customer number started spiking—a good problem to have :)

Cool! Looking forward to it.

Posted by obrienmd on January 2, 2013 at 10:51pm

Cool! Looking forward to it.

_

Posted by shamio on January 3, 2013 at 4:07am

Will this patch be implemented in the next coming version of drupal (7.19) ? It should be tested with sites with a lot of nodes (i.e 20,000 and more nodes) and i want to do this.

Doubt is the father of invention ... ZendegiyeSabz

The number of nodes in the

Posted by yched on January 3, 2013 at 9:41am

The number of nodes in the site should not really have an impact here, but you're most welcome to give it a run :-)

very large node

Posted by liquidcms on January 3, 2013 at 6:14am

We have a project we are working on which has a very large node structure consisting of aprox 900 fields nested in field collections as deep as 5 levels. Should be a decent stress test. I'll see if i can test with your patch tomorrow and post the results here.

Peter Lindstrom
LiquidCMS - Content Solution Experts

Thanks Yves: Commerce Kickstart

Posted by perusio on January 3, 2013 at 10:54am

I'll have a look later today about the effect of this patch on Commerce Kickstart 1 and 2.

I worked on a Commerce

Posted by Artusamak on January 3, 2013 at 10:58am

I worked on a Commerce project with 10+ bundle of commerce product (among other entity types and their bundles) and with 20+ fields per bundle and with the applied patch we saw around 10/15% performance improvement both CPU and memory usage.

This is a very nice

Posted by timonweb on January 3, 2013 at 2:57pm

This is a very nice improvement! Love to see Drupal becoming faster. Will test this one with my setups too.

no improvement but everything still works

Posted by liquidcms on January 3, 2013 at 9:29pm

i tested this patch on site with our "very large" node: 800-900 fields (not sure of a way to find out exactly how many fields), nested within 4-5 levels of field collections, numerous field groups, all fields have default values so data is always written for all fields.

because cloning (node_clone module) is a problem we are currently working i used that as my first benchmark - although, granted i have not read about this patch in any detail so not sure where we would expect to see performance improvements.

update with patched files
run update script
flush all caches

view, edit, save of a test node all work correctly

cloning of the node works correctly as well; but takes exactly the same time as pre-patch: 1:11 to load a copy of the node into edit form and 23 seconds to save the new node.

Peter Lindstrom
LiquidCMS - Content Solution Experts