D7 -> D8 Roadmap

You are viewing a wiki page. You are welcome to join the group and then edit it. Be bold!

Date in Eight Issues

Related Core Issues

Summary of Date in Eight Initiative

Date is a contributed module that is used on the majority of sizeable Drupal sites. While there is quite a bit of functionality in the Date module, the most important and basic aspects help to make Drupal more powerful (such as use cases relating to editing and formatting date for publishing and events). While on the surface this functionality may seem simple, it’s actually quite complicated when one considers time of day, localization and the many formatting/interface challenges.

Fortunately, the most important functionality in the Date module has evolved to a very stable state. The Date module is primarily maintained by Karen Stevenson (KarenS) from Lullabot, who is is an experienced Drupal contributor and a maintainer of the CCK and Calendar modules that are also installed on most sizeable Drupal web sites. Karen has also made important contributions to Drupal 7, including being part of the original Fields in Core sprint, which help lead to moving CCK into core for Drupal 7.

There are several potential benefits from moving the most important aspects of Date into Drupal 8 core, including:

  • User Experience: The date-related functionality in Drupal 7 core is fairly limited, and the user experience will be improved by avoiding the need install the Date module to allow for more flexible formatting and UI features.

  • Contributor Experience: The basic elements of Date are important to many contributed modules, so we can deliver a more streamlined and robust tool set for contributors. And we can provide a consistent base for their efforts.

  • Other Drupal 8 Core Features: In addition to improving the current core experience, there are several several other Drupal 8 core initiatives that would benefit from the Date in 8 initiative, in particular:

  • Multilingual: The efforts to further improve localization and multilingual functionality in Drupal core would benefit from improved date configuration and formatting functionality;

  • Entities: Improved date functionality in core would help to unify entity properties and fields through a consistent date field storage format, date editor widget/UI, and date display formatter;

  • CMI: The Configuration Management Initiative (CMI) would benefit from date variables that are CMI compliant;

  • WSCCI: The Web Services initiative would benefit from consistent JSON-LD serialization of all date fields; and

  • VDC: The recently announced Views in Drupal Core (VDC) initiative would benefit from seeing the Views Date functionality also moved into core.

In light of the stability of Date and above benefits, Karen has been working on efforts to move several components of the Date module into core for Drupal 8, with help from a team she has brought together to assist. In particular, the proposed roadmap for the Date in 8 initiative involves 5 key elements:

  1. Date Component in Core: Integrating a new Date class in Core which provides a wrapper around the PHP DateTime object in order to deliver better flexibility, functionality, formatting and error handling. The component accepts input in various forms, including string dates, string expressions (like 12AM today), timestamps, an array of date part values (year, month, day), or even another DateTime object, creating a DateTime object from any of them. Unlike PHP’s basic DateTime class, which does no translation of the formatted display of a date, the component includes integration with the IntlDateFormatter, when available, to translate and adapt the output for calendars other than the Gregorian calendar. The component also tightens up the otherwise lenient error handling of the DateTime class for better validation of input. With this component in core, contributed modules can extend it or rely on it as a basis for lots of additional date, time, and calendar functionality.

  2. Date API in Core: The Date API includes a number of helper functions that we’d like to get into core. The most important of these are consistent methods for creating lists of translated and untranslated date values, like day names and month names. The translated values are needed for display and the untranslated values are used as array keys and in css class names. Having a central API for this will help both core and contrib. And it is the only way that core will ever be able to support things like alternative calendar systems, which require not just translation of date parts but alternative values for ‘days’ or ‘months’. There are miscellaneous other helper functions that might be needed to support core functionality that will be added as needed. While much of the current Date functionality will remain in contrib (see below -- e.g., repeats, all day, migrate, iCal. etc.), the most commonly used functionality would become part of core. Time permitting, this will also incorporate Date Views functionality, given that VDC has become a formal Drupal 8 initiative.

  3. Date FAPI in Core: The contrib Date API includes code for Date FAPI elements, datetime elements that can be used on any form, not just on fields, e.g., Date select lists, Date JQuery Pop-Up Calendars, Date text fields, and Date Timezone. These elements would be integrated in core to allow for an improved interface for managing date fields of any kind. They will also be made HTML5-compatible.

  4. Date Field in Core: Once all the previous steps have been accomplished, core will have the basic functionality needed to support a Datetime field. The current contrib Date module can store dates in various ways (as integer timestamps, as database-native datetime fields, or as varchar ISO dates). We need to identify the best approach to storing values in the database for core (based both on database and overall load testing) and probably provide just that one storage method in core. Depending on the results of those tests, contrib may also be adapted to use that one method or might continue to provide alternative storage methods. Date fields in contrib have lots of other options, a start and optional end date, ways to limit the granularity of dates, options to choose different methods of timezone handling. Those choices will likely remain in contrib. The core date field will be a simple date field that has none of those options. The core date field needs to be sufficient for use as a timestamp, an event date, or something like a birthday field in a user profile. The plan is to provide just enough functionality to do that.

  5. Contrib: An ongoing task is clearly delineating which functionality will remain in contrib in light of the above, most likely in the form of existing contrib modules (e.g., repeats, all day, migrate, iCal. etc.). The goal for everything that goes into core is to be sure that it is robust enough to support all the kinds of things that contrib might need to do, but leaving it to contrib to actually do those things.

Karen has already posted a Drupal 8 patch for the core Date component and she will be working with her team to move the above components into core by feature freeze (particularly incorporating the Date API and Date field in core). Thereafter, efforts will be focused on other core initiatives that would need to be addressed by code freeze (e.g., VDC integration, entity standardization, localization, etc.). Karen and her team will be providing updates on drupal.org and those who want to get inovled can:

Comments

Allow NULL in start & end?

arlinsandbulte's picture

Currently, a major wtf in Date is that optional end dates really are not. Date requires (forces) the end date to be populated if the start date is populated.
But, if the start date is optional, both can be NULL.

I propose allowing both start and/or end to allow NULL and not automatically be populated behind the scenes if they are left empty (NULL).

There's quite a lot of code

KarenS's picture

There's quite a lot of code that will not work with that right now, so this will involve some effort to go through the code and find all the places that are affected by this change.

And that brings up one more thing that we have never found a good solution for, the idea of 'present' or 'now' as either the start or end date. Like in a resume where you say you have been working at XYZ company from 2001 to 'present'. I really have no good ideas how to solve this and I'm not going to try to do it unless someone comes up with a viable and simple solution, just getting this on the table as something to work in if we can figure out a way.

yep, I know

arlinsandbulte's picture

There's quite a lot of code...

Just bringing it up for consideration, cause it IS a big wtf for a lot of devs & users alike.
I'm not a good judge of how much work it involves, and I don't know what our timeline constraints are. But, if we will be starting work in a sandbox, I hope we have enough freedom to break things and fix them along the way...
Also, all these ideas & features don't need to be addressed right away. But I do think they need to be considered in order to guide us down a path of future sustainability. Get done what needs to be done, but try to predict the future... I know that is hard.
(P.S. while I am not a good judge of effort, I do know this whole 'initiative' is a monumental effort).

About the 'Current' or 'Now' feature request

arlinsandbulte's picture

Yeah, that is a tough one. Here is the issue for that: [#1240526]
Using a text varchar database field might be of some use, but I'm not sure how.

Brainstorming:
Maybe add a 'current' flag similar to what we are considering for 'all day'. Then, add a cron hook to periodically change all database date values with that 'current' flag to the current date. That would help take care of queries & sorting. Then upon display, logic could show 'current' in place of the date value if the flag is set. This plan would probably need to limit 'current' type dates to at least day granularity...
This might also be implemented in an add-on module.

I think that using a cron job

hazah's picture

I think that using a cron job to periodically update date instances is a bit much. Abstractly, this isn't the solution, as DB's offer the 'NOW()' function. I think that the ultimate goal should be to map this function call to a possible value for the field. That is, when the field is queried for it's value, instead of doing a select on a column, the select should be on NOW(). This keeps everything in concert with the way things work now, and in general, this is how fields tend to work.

More flexible text widget

KarenS's picture

The following is copy of a post from http://drupal.org/node/501428#comment-6213380, by mgifford. Based on that, rather than remove the text widget it should be reworked to accept generic text like 'tomorrow'. That's an interesting idea worth pursuing.

The most accessible way to add in a date/time widget is to add in a single field that allows users to plunk in their values and then has PHP or jQuery resolve that into a machine readable fashion.

Rather than using a dropdown box where I fill in a day, month, year using select boxes (say we're talking about 10, July, 2012 as separate columns), it's just easier for everyone to just be able to type in "tomorrow". That can be enhanced using javascript to have a date picker widget, but there should be an option to type in date in any way that php's strtotime() or jQuery datejs' parseExact.

Machines can do this easier than people can.

Couple related links:
http://groups.drupal.org/node/19118
#504962: Provide accessible labels (was "Use fieldset and legend") for compound form elements

Another field type idea

KarenS's picture

Another idea I had is to change from the current way we define field types based on their storage type (datetime, iso, and timestamp) to differentiate between 'timestamps' (meaning individual values that are precise to the second) and 'dates'. It could look like this:

  • 'Timestamp' -- stores a unix timestamp in the database, suitable to be used for all the current core dates (node posted, node changed, etc), no concept of 'start' and 'end' dates, no concept of granularity -- always granular to the second, no question of timezone conversion -- always converted.

  • 'Date' -- stores an ISO date that can have a 'start' and an 'end', it can have variable granularity (year-only, date-only), timezone conversion can be optional, dates can be flagged as 'all-day' if desired.

  • 'Repeat' -- stores one or more RRULES that define a series of dates and is used to automatically populate a selected date field, either as a multiple value field on a single entity or in separate entities. The field would not store the dates directly, only the rules, and would generate and manage the dates it creates.

The 'timestamp' field could go into core because it is relatively simple and a good fit for the ways dates are used in core now, the 'date' field could remain in contrib because it is more complex, and there it could be further refined to do things like allow for flexible granularity. The 'repeat' field would definitely remain in contrib.

'Repeat' -- stores one or

FreeAndEasy's picture

'Repeat' -- stores one or more RRULES that define a series of dates and is used to automatically populate a selected date field, either as a multiple value field on a single entity or in separate entities. The field would not store the dates directly, only the rules, and would generate and manage the dates it creates.

THIS!

Please only save starting and end-dates in the DB and generate everything on the fly.
This has been a problem in numerous issues, here are two examples:
http://drupal.org/node/1779850
http://drupal.org/node/213847

The date-repeat functionality is almost unusable for real world use at the moment. I'm currently trying to build a simple eventcalendar with repeating events and my database is absolutely blowing up. More than 5000 entries in one date field table (next 3 years), and I haven't even entered a quarter of the data I need... Plus I have 8 Date Fields because Date-Repeat-Field-Api doesn't allow for mulitple values...

8x5000x4 = 160.000 rows in the database, nothankyou ;)

Plus I have to go back in three years and update the fields for the next years...

As you can see, its unusable in my case.

Please change it.

  1. Save the starting-/end-/ dates and exclusions and so on
  2. Generate the rest when actually needed.

A difficult problem

arlinsandbulte's picture

Yes, we are aware of this limitation to the date module.
But this is a difficult problem to solve in Drupal.

Generating repeat dates 'on the fly' as you suggest would likely result in even worse performance than today.

Also, repeated dates is outside the scope of this 'Date in Eight' initiative.
Date in Eight is all about getting the date api & date field into core.
Repeating dates will remain outside of core in a contributed module.

Why would that result in

FreeAndEasy's picture

Why would that result in worse performance? You don't always need to generate or display all possible dates of a repeat-date-field.
Take a calendar for example, you would only need to generate the repeat-date-fields for the current calendar view (1 week or month or whatever). Thats would result in a much better performance than querying a date table with thousands of rows.

This is a roadmap for things

KarenS's picture

This is a roadmap for things going into core. Repeats are not going into core, so this is not where we will discuss that.

Sounds resonable, but...

arlinsandbulte's picture

Sounds like a good compromise, but (at least in a perfect world) I would prefer to keep and maintain a single date type. This is especially true when newbies (or epxerts for that matter), start using one type and then later want to switch to another.

I wonder, somewhere I remember a conversation about entity PROPERTIES vs entity FIELDS. http://drupal.org/node/1021466
Posted, changed, etc. I think would be considered a property of the entity.
That might impact things, but I am not sure how.

I don't know where is the

Sylvain Lecoy's picture

I don't know where is the roadmap, however, fix this issue would be awesome: http://drupal.org/node/989366.

Look up ^^ You are posting

KarenS's picture

Look up ^^ You are posting this comment on the roadmap :)

Roadmap updated

KarenS's picture

I updated the roadmap to reflect a bunch of ideas coming out of all the sprints, camps, and cons I've been attending. A plan of attack is becoming clear, and it's something where we could do it one step at a time. At any point we stop getting things into core, we still will have made some improvements and things will still work. Whatever doesn't make it in will just live on in contrib.

@karenS any thought for other

Shahriar Gholami's picture

@karenS
any thought for other calendar systems?

I'd be glad to figure out a

KarenS's picture

I'd be glad to figure out a way to do that. I think that needs to go into contrib first while we figure out what it needs to look like and then have it out in the wild for testing before moving it to core.

I won't get this done myself, but I'm open to reviewing patches if someone wants to try.

I'm agree with you but even

Shahriar Gholami's picture

I'm agree with you but even in contrib it need some support from core
recently i spent some time on this but there is big problem, and it's the Datetime object
for example a date like this xxxx/2/31 is invalid in gregorian and Datetime constructor ties to fix it and changes it to something like xxxx/3/3
i didn't find any way to go around this
the only way i find till now is the same old one, saving as gregorian showing as other calendar systems
I created a sandbox project you may wanna take a look at it
http://drupal.org/sandbox/ShrGholami/1705170

Remarks about the ISO field

Alan D.'s picture

Copied from http://groups.drupal.org/node/221254

I think it would be best to limit this to a very simple sub-set of the supported extended formats stored in UTC. Maybe only

-{0,1}Y{1+}-MM-DDTHH:MM:SS

While not supported currently with the Date module, timezone information does have meaning on dates, 2012-01-01 in America is nearly an entire day behind 2012-01-01 in Australia. So maybe the field support can be reduced down to an ISO-like field and an ISO-like field with timezone support? This drops time support, but I don't think that core should support Time fields.

Negatives
DB bloat

Positives
Native db varchar indexing (and searches?) without the need for any date functions
Simpler code (?)

Date API

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: