A hopefully sane solution to dynamic string translation

I was thinking through dynamic variable translation with Konstantin Kaefer yesterday, and some good ideas come to me after I gone off IRC. This could solve variables, content types, menus, taxonomies, blocks, user profiles, as long as we add dynamic string based translation to these (which will be possible to turn off by type :).

1. We cannot reuse the locale functionality, because it is designed for "static English string to static foreign language string translation" from the ground up: http://groups.drupal.org/node/1827

2. We cannot use a simple t() like function either, as we need special widgets with validation and submit functions for some values (like the site logo or default user picture).

3. We cannot reuse the admin interfaces (menu, taxonomy, content type setup, settings pages) as localizer does, because many sites will not give administrator, menu editor, taxonomy editor, content type admin or site setting admin permissions to translators. BUT we need to reuse some forms from there.

4. Dries would like to see a simple reusable API anyway :)

So, we need to somehow get together what is good in t() and what is missing from t(), as well as provide a specialized UI for translation of these strings, a one-stop-shop for dynamic interface translation for the user.

Let's start with the API, it is easier :)

Note that we could possibly provide a generic dynamic translation layer for variables, menus, taxonomies, user profile options and blocks with this system.

But how do we make it extensible? Contrib might have more stuff to translate on top of these basic core Drupal objects. Well, the first parameter of dt() is a 'domain' (in the sense of how GNU gettext uses this term), identifying what domain the value is in to translate. Contrib modules can reuse the domains used by core (when adding more variables for example), or use new domains themselves.

But how do we make this user friendly? Well, we should not throw the whole list of translatable stuff at the user at once, so I say we can either provide a filter based on domains or tabs for domains on the translation interface (in core we would have at most 5 domains). Well, these tabs needs to have user friendly titles, so 'variable' is not suitable there. Here we connect to how extensibility can be implemented.

This provides a human readable list of names for domains provided by the actual module.

But what if Jose would like to add taxonomy term and menu item translation with standalone taxonomy terms and menu items, not dynamic string translation? Well, we have our domains handy, so we can easily provide an interface to select what stuff actually gets translated by this API. The user can be provided a selection list, where he or she can check off some of the domains he or she is not interested in translating with dynamic string translation (and translate with some other means, not included in Drupal core).

OK, this way the system is both extensible and restrictable. We can assemble translation forms for dt() values based on the default textfield, or the type of form item requested in the string parameter or the form chunk provided in the FAPI array. By default, most values are simply textfields, some are textareas, and there are a few exceptions, where custom upload widgets and validation is needed. Hopefully the above API is scalable enough to those needs.

Now for the harder part: how do we implement the above API? Well, my API examples above actually mix in two things: providing a domain name, a domain key and a default value AND providing information about the widget which should be presented to edit the value. We need both of these to present a UI to the user to edit the value, but not to grab a previously stored value from the database and use it on the site. It would be a show stopper if developers should include the UI widget information at all places, where the value is used. So we should decouple the UI definition from the dt() using API. This could be done with a new hook:

This is all nice and good (or prove me wrong if not :), but there are three unresolved issues still:

1. How do we collect the translation base values for dt()? Just as we go along, like t() or variable_get/_set() does? We won't have variable values for translation whose default values are acceptable for the base language... Come out with your good ideas!

2. How do we store the values translated? Interestingly we previously had a good storage strategy for variables but no UI, now we have the UI, but an extended API applicable to more than just variables, so storage is a question again. We can model our tables on locale modules tables: we need a source and a target table, source storing the domain and the key with the default value, while target storing values for the translations. We could even (oh the horror) look into reusing the locale table altogether, as we have most things needed already there (but adding a flag to signify dynamic vs. static strings). That way we could even get PO import and export for the dt() values as sweet topping :) We also need to somehow work out an in-memory cache of the values, especially variables, used on every page (we already have an in-memory cache for variables in our i18n SVN repository, but not for other values).

3. What about database level translation, so we can do SELECTs whose results are directly in the requested language? For that to work, we would need to be able to join the translation table to all possible object tables (which would require these objects to all be in some table, having a unique id, whose type is the same as our translation table id for every object). I am not sure we are about to solve this. If all the above are solved, that is still a huge step. Anyway, if you have ideas in this department, do not stop yourself from shouting up!

Finally I hope that my writeup was clear enough to start some active discussion and get a good solution into our i18n SVN repository and then to Drupal 6 as soon as possible. Come with your criticism and ideas!