CCK performance (or lack of)

gordon's picture
public
gordon - Sat, 2008-03-22 22:33

Here at Lonely Planet we have been looking into CCK for our own projects so we can create easily supported sites which don't need developer input to do simple things like adding additional fields.

But during this time I discovered how bad the performance actually is. Like everyone I thought that adding fields that were not multiple or shared you would get the best performance.

Well I was wrong, and spent a while making sure that I was correct. And I was.

So during some other enhancements I needed to fix the CCK so that performance was much better.

So what I found is that sharing a field or having multiple occurrences really have no less performance impact than single fields.

Even though the fields are built in a fairly good way so that multiple single fields are in a single table, they are still being loaded 1 at a time. So if you have 30 fields added to a node it will do 30 selects to load all the fields, and I like the fact that it will be 30 deletes just to mack sure that it really have been deleted.

My team and I at Lonely Planet have been working hard on to fix these issues and are just about to submit a huge patch to fix this issue and add associated fields to CCK.

I have included a presentation that I gave to Lonely Planet and a few other people which shows how bad the problem actually is. I hope to submit this patch in the next 2 weeks (as I am on holiday this week) and we are still putting it though it's paces.

This patch is for Drupal 5, and we are hoping that this could be included in the next Drupal 5 release of CCK which I think is really needed.

I know that this has been fixed in D6, but actually my patch is still faster than D6 esp. when it comes to multiple fields.

AttachmentSize
CCK Performance.pdf342.8 KB

Good stuff

johnforsythe@drupal.org's picture
johnforsythe@dr... - Sat, 2008-03-22 22:42

It's great to see someone working on this. If CCK can start to approach the speed of a module-based custom content type, it would be a huge benefit to everyone.

--
John Forsythe
Drupal Modules - Find the module you need for your project!


In fact with multiples I

gordon's picture
gordon - Sun, 2008-03-23 02:00

In fact with multiples I think that my approach is faster than the D6 version of CCK.

I will have a little overhead, but it should be pretty good.

Gordon Heydon
Lead Drupal Developer - Lonely Planet


As you noted, this is done

KarenS@drupal.org - Sun, 2008-03-23 10:48

As you noted, this is done better in D6 already. Also we have found that the node_load isn't the biggest problem since it is cached (the node load is slow, but not done very often, only when the node is changed), it is Views which really slows things done since there we have to join each field in separately and there is usually no caching. And fixing the way Views joins works isn't really possible for D5 and D6 because we have to alias every field since their tables change.

We are discussing the field structure and performance for CCK in D7 and beyond at http://groups.drupal.org/node/9297, so if you have ideas you should post them there.

Performance improvements are always of interest!

Thanks, node_load() is not

gordon's picture
gordon - Sun, 2008-03-23 12:01

Thanks,

node_load() is not really the biggest issue. But as soon as you start adding more fields the problem just get compounded by insert and update.

If you take a look at my figures it just gets bigger and bigger still even with caching.

Gordon Heydon
Lead Drupal Developer - Lonely Planet


Nice

macrocosm's picture
macrocosm - Wed, 2008-03-26 04:17

Thanks for the report, I was curious about how CCK performance would stack up to large amounts of data. It looks like your CCK performance tweak is going to be extremely helpful! You rock!


110 CCK fields!

scor@drupal.org's picture
scor@drupal.org - Wed, 2008-04-02 20:28

The New York Observer site has 110 CCK fields! (though I don't know how many content types they are spread over). See the nice and detailed write up: http://drupal.org/node/141187
I wonder how they tackled these performance issues. Barry and Moshe are both involved in the CCK working group, and good things will eventually come out of these experiences...


I have posted the patch to

gordon's picture
gordon - Sun, 2008-04-13 13:12

I have posted the patch to correct this over at d.o

See http://drupal.org/node/246084

--
Gordon Heydon
Lead Drupal Developer - Lonely Planet