No field update

The idea behind separating Field and Instance was very simple: Instance contains all the things that are just lightweight settings, and Field contains all the things that affect the schema and are thus "heavyweight" to change. You have now identified a third category, however. Field also contains those things that are shared between multiple instances, and some of those things may prove to be lightweight and easily changeable (e.g. allowed values and default values). As you say, we would prefer not to impose arbitrary and unnecessary constraints on what can be updated after-the-fact from core.

As a reminder, the design for field updates is to provide a field_update_field() function which first performs whatever changes it can/knows how to and second delegates the remaining operations to a hook. The set of changes field_update_field() can perform itself might be empty, but I had suggested (back before we decided on per-field only storage) that conversion of shared and cardinality might be in that set. Of course, lightweight field settings could also be in that set. Actually, in this model, the hook part will have to come first---we can't make some lightweight changes, invoke the hook for the heavy changes, and discover that no contrib module can handle the heavyweight change. We'll have to clearly specify exactly who is responsible for what to prevent contrib field_update hooks and field_update_field from stepping on each other.

So, perhaps we need to update our definition: Field contains everything that is sharable between instances (always including all schema-affecting properties), and Instance contains everything that is unique per instance. field_update_field() then exists in core, and defers to a hook function for all heavyweight changes.

Anyway, here are my responses to your specific questions:

• What about allowed values list? Must be able to change it any time, must be able to share it (implies it must be a field rather than instance setting or otherwise sharable).

Three thoughts:

1. I agree with "must be able to change it" but "must be able to share it" is not obvious to me. I can imagine a single field with integer values attached to two different bundles that wants either different allowed values or different display labels for the same allowed values per bundle.

2. We discussed at the code sprint that Value List should probably be its own entity, separate from Field or Instance. Then an instance could either refer to an Allowed Value List or possibly someday also a Recommend Value List (think of a Windows-style Combo dropdown/text box). This gives you change-ability, share-ability, and update-ability within core. No mess, no fuss.

3. As the previous commenters suggested, a Value List sounds an awful lot like a taxonomy vocabulary. However, taxonomy "values" are all integers, whereas CCK value lists currently also allow strings. In principal (I don't know if this is currently possible), a field type module could support a Value List of some other type in which the List itself would contain a string representation but the database would contain some other native format (e.g. non-timestamp dates). Using taxonomy would preclude this.

For now, the simple thing is to leave it at a Field property and allow core to change it. I have no moral objection to this.

* What about allowed values php? Must be able to change it? Would be better if it just refers to a PHP function defined somewhere else, but we don't require that now. What about typos and bugs that need to be fixed?

Didn't we recently remove the PHP Input Filter from core? I think allowed values PHP is a great thing to banish to a contrib module. Could we get away with this?

* What about changes that have no effect on the schema? Why should we prevent them when they won't hurt anything, seems unnecessarily harsh.

See above.

* What about changes that change the field type, clearly should be a third party contrib solution.

Yes.

* What about changes that affect the schema but don't change the field type (i.e. change the size of a varchar)?

This is splitting hairs. Changing the schema is changing the field type. In this particular case, the field module should just make the varchar be length 255 and impose restrictions in the business logic. The whole point of varchar is that doing it that way imposes no wasted database storage overhead.

* What is the optimized way to store values? Would it be best to always use a 255 length on a varchar even if we don't need it so it can be changed later?

Yes.

* Should we have other field types for varchar and various sizes of text? Should we provide a few standard varchar sizes instead of leaving it open?

In some other post in this group you suggested that a "text" field should not be able to be either a varchar or a text column, that we should provide separate field types for those. The same question would apply for text and text:big, as you say. I think I think that I agree, these should be separate field types. I'm not certain.

* What about number fields? Should we provide a single large container for all sizes or limit it to some preset sizes, or to any size?

If here you are referring to int:tiny vs. int:big, float:normal vs. float:big, then my answer is "whatever we decide for the question about text columns, do that for numbers too." :-)

OK, stating 'one field type per db columns definition' means:
Text field type is split into varchar(255), tinytext, mediumtext, normaltext, longtext (or more probably a reasonable subset of the '*text' ones)
Integer is split into (a reasonable subset of) tinyint, smallint, mediumint, normalint, bigint.
Probably no biggie.

I'm less sure of what 'maximum size' means for float and decimal :
Float : well, we currently just use the 'float' schema column type, without any size option. We could probably keep on with this - if we want to keep a float field type at all ?

Numeric (decimal) has scale and precision.
- Do we settle on a single 'Numeric' field type with 'maximum' (= 'reasonably high') scale and precision - is there such a thing as a one-fits-all
- Do we pick and hardcode a couple values for the (scale, precision) pair to build a couple 'Numeric' field types ?

So one downside of standardizing on field types is that we will have a much longer list of field options and that list will need to be explained to end users who don't know anything about databases -- how do we tell users when to use varchar vs mediumtext vs longtext? That doesn't matter for the API, but will be an important factor in the UI, since we want to make a UI that is easy to use without knowing anything about how data is stored in the database. I guess the UI could be responsible for trying to explain all that, but it does add an extra burden to the UI.

And what do we do for numeric types? There are a lot of options and we have people already using it that may have created fields that use some combo we wouldn't have picked as an option.

I wouldn't mind getting rid of the float type. It's there because it was in the original code and people have float data that has to be supported and there is no good, reliable way to convert that float data to decimal data. We can remove the 'float' option in the UI so no new float values can be created and just support the existing method in the API.

Buried in the original notes was an idea that we add a flag to the field settings to indicate which settings affect the schema. One option is to continue to handle fields as we do now and use that flag to identify field attributes that can't be altered. So then our API will check if a requested change alters either the field type or any of those field attributes that affect the schema. If so, see if there is a contrib module that can handle the change. If none of the changes affect the schema, the API can process the change.

We could still add some code in the background to store values in optimal ways, like standardizing on the 255 length for a varchar, but keep that all behind the scenes.

I generally agree with you Barry, just a couple clarifications/observations:

• Other contrib modules have 'field' settings that may not affect the schema. Date module has a 'granularity' setting and I'm sure there are others, so it is well worth while to figure this out to be sure those harmless changes can be made.

• How will field_update() know if it is being asked to change something that affects the schema? That's where I think we may need a flag in field_info() or something like that to separate field settings that affect the schema from those that don't. Or maybe we break the field settings into two parts: field_schema_settings() and field_settings() for the settings that do and don't affect the schema.

And is 'multiple' (or I guess it's now 'cardinality') an allowed or non-allowed change now? With per-field storage, it wouldn't affect the schema to change it.