Posted by moshe weitzman on July 4, 2007 at 2:15am
So, what are the challenges for porting CCK to Drupal6? Seems like it will be easier than the D5 port. What new possibilities are opened up for us by 6? What is gonna be harder? We have schema api, batches, $node->build_mode, fapi3, ...
Comments
Schema api is going to be
Schema api is going to be really nice. Dopry and I actually started doing something like this at DrupalCon as an idea of a step on the way to bringing CCK to core. At that time I started a patch to incorporate our version of schema api into CCK and it eliminated a ton of code in CCK, and that's a very good thing.
Another thing I think we want to explore is using more of FAPI in the way we handle widgets and their processes. I haven't myself made any attempt to see what this will look like, but I'm thinking there is going to be some potential to rework widgets a bit to eliminate some CCK code and let FAPI handle more of that processing.
Basically, what I hope is that we see CCK get smaller and smaller and more and more bits of it make their way into core:)
Looks like we crossed-posted
Looks like we crossed-posted :-)
Meanwhile, I see that http://drupal.org/node/121620 was committed, which sure encourages us to open widget refactoring - that is, CCK 2.0 :-)
I heard that Earl is planning to merge Views D6 port and Views 2.0 development (and that Moshe is a litle worried about that, because of OG's dependency on Views...).
What strategy should CCK adopt here ?
I would really like to be
I would really like to be able to make some API changes now, and it does seem that doing that in the 6.x version makes some sense. It would be nice to port both a 1.0 and a 2.0 branch, but realistically, who has time to do that?
I'm thinking that we're talking about things that might make big changes in CCK core, but not so big changes in field modules, so it might not be too much to expect CCK field modules to pick up on the changes. Schema API means CCK field modules will also need to change the way they define their columns, which means they're already going to have to adapt to some API changes, so this might be a good time to change other things if we need to.
And it's real easy for end users to see the difference between 5.x and 6.x versions. I'm not sure they always pick up on differences between 5.1 and 5.2 versions.
This all means that the 5.x to 6.x port may take longer than others, but if Views is making major changes, that's going to be the situation anyway (CCK is going to have to adapt to the Views changes, too).
dedicated effort
i think embarking on CCK 2.0 during the port is fine if someone is dedicating major time to it. if the port takes more than 2 months, the whole drupal comunity is gonna get really cranky. we can easily refactor and port in 2 months, but i think it ought to be the primary focus of karen or yched or dopry in order to assure succes against the timeframe.
i worry about views 2.0 at same time as the port for the same reason. if earl dedicates himself to it, then no problem. but if other priorities compete too strongly, Views won't be ready when d6 is in RC and then sites can't test the RC and we get stuck.
Well, now that I've started
Well, now that I've started looking at how many changes will be needed for Schema API (and thinking about the fact that we will also need to incorporate Views 2.0 changes), I'm re-thinking this. Maybe we should not try to do other API changes as a part of the port. I'm definitely willing and planning to commit some time to get the port done as quickly as possible, and hopefully yched and maybe dopry can also commit time to it, but maybe we'd better just start with porting the existing code with no API changes. I'm open to suggestions, depending on how much time the others can spend on this.
Yes, porting should be
Yes, porting should be easier than D5 port. AFAICT, the most challenging bits will probably be FAPI3 conversion and ensuring every form works OK (CCK has nice and multi-level generated forms...). And of course E_ALL compliance :-D.
About new D6 features :
- Schema API is an overcool thing that will sure allow us to simplify some parts (although maybe not that much - the 'core' ddl part of cck was only 20-30 lines). Will probably require some thinking / refactoring, though.
- batch API should allow us to handle mass node deletion (eg content type deletion) in a more thorough way (call 'delete' op on each field of each node). AAMOF, that was the original reason I embarked in this 'batch' thing. There has also been some issues reported when db scheme changes (share a field, add a field, turn a field multiple) and you have a HUGE number of nodes - we could maybe have to batch stuff there.
- ajax forms can be cool at places, but the main feature (js 'add new values' button on node edit forms) is still not possible with current CCK architecture - that's 2.0 stuff
Got to run, more thoughts later :-)
node_load improvements
any thoughts on barry's proposal to eliminate the field cache in favor of a smarter node_load() that leverages schema API? see http://groups.drupal.org/node/3697. we'd obviously need some benchmarks.
sure
I did not get a chance to actually look at that code yet, but this sounds like a big improvement - the current 'one query 'per field load / save' is definitely frustrating. We'll have to look into this when migrating the db part.
I started some basic migration tasks yesterday (info files, schema for 'system' tables, first round of FAPI3 changes), and committed them to HEAD - I did not actually try to run the code yet :-)
Karen, if you have some code from the earlier ddl effort in the 'dynamic tables' area, it could be interesting to post it in a sandbox ?
@yched, I'm trying to find
@yched, I'm trying to find that code, seems to be lost :( Anyway, I'll take a look at what you've done so far and see if I have any more ideas...
Possible Structural Changes
What structural changes should be made in CCK in 6.x? I can think of some obvious ones:
1) Convert to Schema API method of handling database creation/changes.
2) Incorporate more of FAPI into widget processing
But this might also be a time to think of other ways to improve the API, going back to Dries earlier challenge to look over the CCK code and really think about the way everything is done and how it could be improved.
I'd also like to look at the end user experience and try to improve it if we can. One thing I've been thinking about is the way end users display a field in a template. I really hate the $node[field_name][0]['view'] construct -- I keep encouraging people to use content_format() instead because it works the same from version to version and doesn't tie us to a specific node construct, and it is too hard for non-programmers to grok -- and yched I know you keep encouraging people to do it the other way, so we haven't seen eye to eye on that :) There are problems with content_format(), too because it requires you to understand the concept of a 'delta' and what that is and it doesn't know anything about the new context values we've added to the display, so I propose we rework content_format() so that it defaults to accept just two simple arguments -- the name of the field and the page context (content_format('my_field', 'teaser') -- and it then spits out all the html needed, grabbing the value from the node, doing the foreach() loop, etc. That would be much easier for an end user to use in a theme. We can still have optional arguments for the format name, delta value, etc that can be used by programmers in more complex ways.
Or maybe we leave content_format() alone and add a new function for themers called content_display('field_name', 'context').
It looks to me like you are
It looks to me like you are talking about including the same sort of functionality as the patch I've just suggested (http://groups.drupal.org/node/4891) - though the patch only addresses one very very small aspect of your suggestion. Is that right? - At least what you are suggesting sounds like it would make it easier to get to where I needed for the editable view fields?
widget - field interface
It would be cool if we could separate the field and widgets a bit better - in 5.x the field modules had to re-implement existing widgets, because each widget was only working with the specified list of fields and no more - so e.g. content taxonomy introduces a new field type for which the options widgets are re-implemented.
Instead it would be great if every widget could be used with every field - so we could reuse code much better. I know that's not that easy to do, as e.g. filefields require a different data structure than others. A kind of interface system like it is common in object oriented languages would be useful - I'm sure we can find a rather simple solution for that problem.
Perhaps we find one in barcelona..
Already done
That's actually what has already been done -- you can now use any widget on any field, so long as the widget is structured like the core widgets. As an example, nodereference and userreference use optionwidgets for their select lists and they use the text module textfield instead instead of creating their own autocompletes.
And you can use widgets within widgets, which is sort of what nodereference and userreference are doing. In my new version of the Date module, my plan is that widgets are going to be several layers deep -- the top level will be a 'date_combo' widget that combines two date widgets (for the from and to dates), and a timezone widget into a single combo widget.
See the tail end of the discussion at http://drupal.org/node/157176 for more info. This will need more cleanup and testing, but it is basically working. I tried to put a lot of documentation into the core widget code to help make it more clear how things work. I hope to talk about this at DrupalCon and welcome more ideas for improving the system.
awesome!
this is even more flexible as I imagined!
thanks, for your awesome work!
Schema API question
I've been looking through the new Schema API code and it will be really great in CCK. The only problem I see right now is I can't figure out how we can dynamically create new tables that have no .schema file. I can see lots of ways to alter those tables if a .schema file exists, but no way to create a new schema out of thin air. We obviously will be doing that a lot in CCK. Anyone know how to do it?
Think I found the answer --
Think I found the answer -- we need to create our own hook_content_type_schema() or hook_field_schema() or something like that within the cck .schema file to dynamically alter the schema before we return it.
The initial game plan for
The initial game plan for the port is to leave Schema API out until we have a functional port of all the other parts of 6.x, except that we'll use Schema API for the static tables, node_field and node_field_instance. Yched, Dopry, Bjaspan, and I have discussed this via email, and because the Schema API changes are going to be significant that seems to be the best way to get started.
An initial issue for this is opened at http://drupal.org/node/157176. Note that HEAD will be broken and unusable until we get this finished -- not that it wasn't broken and unusable before we started :)
fields not modules
I think that fields of cck not should be modules.
In this mode they are loaded every time.
If this become include of cck they are loaded only when there is need of that
Port existing CCK codebase to Drupal6?
I am afraid that with much development going into refactoring and new features for the port of CCK to Drupal 6, it will take longer than necessary to have -any- CCK in Drupal 6.
Therefore, concurrently to the efforts outlined in this thread, I would like to start porting the DRUPAL-5 branch of CCK to Drupal 6, using Converting 5.x modules to 6.x as a guideline. Are you aware of such work already started, or would you even recommend against it?
Or is this exactly what KarenS has started?
Olav
D6 port is progressing well
The D6 port needs more work but is progressing well. It would take at least as long to start over and port the existing code with no API changes and it would create twice as much code to support (and there is barely enough time to support one version), so I wouldn't suggest doing that. In addition to the link above where the main port was started, yched is porting the Schema API changes in http://drupal.org/node/167945.
I plan to work more on this soon, and hopefully to get some help with the port at DrupalCon, so I think we'll have a working D6 version fairly soon.
pitching in
Olav - we could really use your skills working on this upgrade, and not some other version that won't be supported.. Everyone wants (and nearly demands) that CCK be ready at same time as D6 release.
contributed fields
Additionnally, CCK's value also lies in the contributed field modules (Imagefield, date, fielfield, EmbedMedia...)
Having a "direct 5 to 6 port" version requiring its own set of contributed fields would make little sense...
Already done
Edited moved my comment back to where it was supposed to go :)
progress
I see a lot of commits flying by in the various cck modules with comments like Two big D6 port changes: which has me excited to start testing.
For me cck is about the only thing holding be back from building (small, tester) sites using Drupal6. It would be nice to know if CCK for 6 is ready for testing or still a few weeks away.
Should we follow http://drupal.org/node/157176 for more updates about when you will be ready for testers?
--
Knaddisons Denver Life | mmm Chipotle Log | The Big Spanish Tour
knaddison blog | Morris Animal Foundation
Working on it
I'm committed to getting it out as quickly as I can and have been working pretty steadily on it. Lots of things are done, some of the later changes broke some earlier things that I need to go back and fix again. I don't know what to say about when it will be ready except that I want to get it done as quickly as possible, if for no reason other than that I have other things I need to do :)
And yes, that issue is probably where I'll post when we need testers.
Anyway, a very brief status:
1) Widgets reworked to use FAPI and to be nestable and re-usable is done, subject to some fine tuning and deeper testing. After exploring several different ways of passing around the field info widgets need, we're now creating an array called '#field_info' in the node form that contains the field settings arrays for all fields on the form.
2) Schema API is being used in many places and really reduced and simplified the code. There are a couple other places where we might do more with it (mainly in the big kahuna content_alter_db_field() function -- the biggest blackest hole in CCK).
3) Moving multiple field handling back into the content module is done, including an AHAH 'add more' button for adding multiple values, and the ability to set multiple to either be the unlimited AHAH add more button or a fixed number of instances. I'm no jquery expert, so I hope someone who is will see if there are ways to improve that.
4) There is the beginning of an API in the content_crud.inc file now -- it doesn't have all the features/functions that are needed, but does provide a way to do basic crud processing of fields. That is brand new and I'm still debugging it, so that is very much subject to change. The API didn't necessarily have to go in before release, but I found that consolidating all that logic really cleaned up the code and made it easier to debug things, so I went ahead and started it.
5) I've just started reworking content copy to use the API instead of drupal_execute to do its magic, which should help with things like install profiles (and also gives me another way to test the API). And I have to do something else with it anyway because the D6 FAPI changes broke it in lots of ways.
6) We're now storing module and column info in the field settings database which will finally make it possible to clean up after modules that have been disabled (in the current version as soon as a module is disabled you lose any way to get information needed to clean up the database). We also needed a way to do batch processing to clean up data when fields are not being used any more and in D6 we have that, too. These kinds of things require schema changes and database updates, so they can't be postponed until later and have to be done.
This port has major changes under the covers in CCK, so it is a bit slow going.
Ready for beta testers
The HEAD version of CCK is probably ready for some early beta testing. It's still buggy and definitely not production-ready, and there are more things that still need to be added, but it would be good to have other people starting to try the code out and look for bugs.
I've also started a manual for developers who need to update their modules at http://drupal.org/node/191796.