Move interface strings entirely out of Drupal code files?

Forgot to mention that Jose already has a proposal for moving longer texts to individual files: http://drupal.org/node/365934 - that does solve some of the big text chunk related issues but tens of strings can still easily appear in a hook_menu or hook_form_alter or a form function.

I must say I like Drupal's approach to english strings. When I worked on (my own) system that used keys, I suffered royally and I remember that many times I just told myself "gosh, f*ck it" and hardcoded the strings in. So I think one of the disadvantages might be users that will skip this whole step. Right now, they mostly just embed the strings in t()

tuag (the user advocate guy) has a whole set of articles on how we could enable people to improve UX in projects by doing this, notably http://www.tuag.ca/articles/core/section1/string-practice and http://www.tuag.ca/articles/core/section1/string-manager. I've asked for a summary/comments here, so we'll hopefully get more from there, but in the meantime, these are very good reads. Thanks James Walker for the pointer (https://twitter.com/#!/walkah/status/78501799778193409).

The performance issue is reason to be concerned. The reason that this works well for large Java or .NET or desktop apps is because they aren't PHP ;) I've seen similar approaches to many Rails projects as well, but the difference with all of the above examples and Rails as well, is that they don't bootstrap from scratch for every request. Once the strings are parsed and a stored statically in a data structure, they don't need to be re-parsed and re-processed for the next request.

I am sure we could find a way to use caching (and in turn memcache) to make string lookups fast, but this will definitely impact smaller sites on shared hosting.

I am concerned that this would greatly raise the barrier to entry (both for reading and writing code), and also, as Jakub suggests, lead people to ignore the internationalization system all together when writing modules.

Is there a way to combine keys with default English strings in code? In other words, something closer to variable_get(). Is there a way to provide a tool (coder?) that would auto-generate appropriate keys for you once you have a 1st pass of the module?

For example:

has a NULL 3rd parameter, so an attempt to translate it might only use the hash of the string as the key, but if you write this:

Then you have a meaningful key that allows translations to be insensitive to changes in the English text. If the site is running English-only, then the speed would be the same as now.

vote for II->b
such system make the strings mostly short (id is much shorter than full text) and give us possibility for changing original strings without code change

Interesting discussion. I would add another advantage and another disadvantage:

Advantage:
- English-speaking developers would be able to immediately see when they have created something that does not translate well, possibly resulting in more robust modules that work better in all languages. They are somewhat protected now from things that break when run through the translation system if they only work on English language sites.

Disadvantage:
- I fear a gigantic mess of name collisions/confusion when developers don't realize they have created a key that some other module has used for some other purpose.

Of the other advantages and disadvantages mentioned above, my biggest concerns are readability of the code and performance.

We are actually suffering a similar problem in configuration management. We want to remove the need to specify defaults in variable_get() (or whatever its replacement is) because it can create a lot of really weird bugs and its hard keeping those up to date with each other throughout code. However what this means is that we will have to have the defaults specified in the files where we are storing configuration, and thus when you start using a variable you will need to go into that file and add it by hand somehow. We may also be able to come up with some tools to manage this, but ultimately it is the same problem you're facing in terms of the DX hassle. We have internally decided to go this route, but I wouldn't be surprised if we get a lot of flack for the decision and have to re-evaluate after feedback from the community.

I already developed custom php sites with i18n in mind, and the easier solution is clearly using keys. It helps decoupling the work of the developer and of the UX designer/translator (complete different jobs) as the translator doesn't have to handle code, and the coder doesn't have to do the work of the designer/translator (writing and translating). This system is already well in place other technologies as mentionned above.
Workflow problems can be solved with the idea suggested by bojanz, i.e. a module listing keys and allowing input for translations. This workflow already exists in D6 in the translation_table module.
Name collisions is a false problem, we don't have much functions/variables names collisions today and it's the same behavior.
Code readability is ok as long as your provide descriptive keys, really. It even allows lighter php files.

The performance may be the only real concern, I don't have suggestions here.

Completely agree with User advocate, and I would choose the UPPERCASE_AND_UNDERSCORE snyax for the same reasons than you. This would allow and enforce UX people to write meaningful form fields labels and description for example (really, we developers are bad at it, because it's not our job)

1. Wondering whether this could be a two-step process/transition; e.g.
   D8:
   

   <?php
  t('user.user_login_form.username.label', 'Username', array(...));
?>
   
   D9:
   
   <?php
  t('user.user_login_form.username.label', array(...));
?>

2. Wondering whether this context could replace and incorporate the existing context; e.g.,
   Current:
   

   <?php
  t('View', array(...), array('context' => 'verb'));
  t('View', array(...), array('context' => 'noun'));
?>
   
   New:
   
   <?php
  t('node.admin.content.list.link.view', array(...));
  t('views.menu.link.view', array(...));

  function node_string_info() {
    $strings['node.admin.content.list.link.view'] = array(
      'text' => 'View',
      'context' => 'verb',
    );
    return $strings;
  }
  function views_string_info() {
    $strings['views.menu.link.view'] = array(
      'text' => 'View',
      'context' => 'noun',
    );
    return $strings;
  }
?>

3. I don't understand why source strings would have to live in the database. Monolingual English sites could still run without the major overhead of retrieving non-localized strings from the database. Given a pattern similar to 2) above, a current call to t() would stay the same and would merely have one additional internal call to fetch the source from the info hook.
   That said, to make that possible without a string info cache, it'd have to be:
   

   <?php
  function t($module, $id, $args = array());

  t('user', 'user_login_form.username.label', array(...));
?>

We are not going to make the life of every Drupal developer utterly miserable for the perceived possible comfort of those relatively few who work with multilanguage sites.

Edit: i unsubscribed from this thread so this was my last reply.

Isn't this why we added the "context" argument in the t() function? I'm probably thinking of something different...

Haven't really dealt with translating Drupal websites into other languages much, so I can't really chime in that much here... But as a developer who's used other platforms that required string registries, I can say they are a pain to deal with. The experience I had was quite similar to the one meba mentioned earlier.

I decided to look how this is solved in Joomla. Joomla uses keys stored in {languagecode}.ini files. I do not know if we can compare quality of Joomla vs. Drupal extensions but from the 5 extensions I have checked, I can clearly see that using JText() class to output translations is definitely not used as often as t().

Not too fond of having to pass magic constants to t(). It would be possible with debug_backtrace, but it wouldn't exactly be considered kosher to use it for that purpose.

This would be easier to do if PHP had a preprocessor.

Regarding:
"This is being done by lots of systems in the industry. If you just look at how Java property files work or .NET resource files (which originate from decades old Windows programming standards). " and "Most software uses something like the proposed scheme."

Our t() is modeled over gettext, which is the base of translations in Linux. Gettext first release was in 95, back when Windows still didn't grasp the concept of 'one binary, many languages'.

Regarding the comments that developers shouldn't be doing UX while coding:
Well, excuse me, but it happens to be part of the job a lot of times. I don't know where you work, but in my experience the UXers do 'the frontend', which means the stuff that the clients end users (often their customers) see, and leave out all the administration pages to manage said frontend, to us Drupal people. Not to speak of the occasions where the client "just need this little thing" and don't want to involve too many people.

dropcube's comment is at http://buytaert.net/multilingual-support-in-drupal-8#comment-38976 .

As I'm one of the lead of Drupal translators here in Czech Republic, I'm sometimes angry when I see how are the developers unresponsible. I remember that I translated one release of Übercart to have almost full 100% translation. (Except bunch of words in Drupal 6 which can't be translated since it have completely different meanings in different contexts, unfortunetaly with the Drupal 7 and the lack of usage of the context is the situation almost same.) So I had the translation done, spending couple of days on it, and was proud of it, but within the week there was another (minor) release and 300 strings were changed. WTF. And almost all changes were in the following style:

• "Some form label:" => "Some form label"
• "You have error in !input." => "You have error in @input."
• "Hello @user." => "Hello %user."
• "My orders" => "My Orders"

I was very angry when I spotted these kind of changes and that was time when I will change the t() for string constants based translation. Fortunately some translators tools like Poedit has the translations memory and automatic translation feature which could performs the fuzzy translation. If we implement the similar functionality to the l.d.o, it will save a big part of translators effort which is wasted with fixing changes like this.

This my experience (or rant if you wish) will spoke for the "moving interface strings entirely out of Drupal code files", however I'm against that idea.

Constant based translation will be another barrier for the most of the developers as someone mentioned earlier. I'm giving -1 to that. I see the way in the better translators tool. Automatic translation isn't too hard to implement (at least for european languages I have no experience with Chinese or similar), I have been thinking about that for year, but the time is still against me.

I would like to see some standard for the context, and might be we could force to use at least module name as the context. It will help for the most cases, but e.g. for Ubercart there will be still problem with words like Order in the context of "ecommmerce" and "sorting". So probably only some forced and standardized marking with appropriate contexts (no matter if hierarchical or not) should work. Another idea is e.g. if we will pass the module name to the t() we could define the default context or contexts in the module's info file... just thinking.

Someone in this thread also called for the non-english based translation, I don't. But I call for the equivalent of the string formatter function like t() or format_plural(), to cleanup and format strings but without the translation processing. When I'm developing some custom modules for the local Czech clients, its idiotic to force Drupal developer to write the strings in the english and translate them through the Drupals translators interface. I'm often using just sprintf... But it has not full formatting potential of the t() and format_plural(). What about nt() - not translate, or might be local_t() and local_fomat_plural()? Or passing some option to t()? But since no untranslated strings are allowed in the Drupal core and contrib (I agree!), this may live in the contrib anyway, I just feel bad to create a module just because of one or two functions, but is probably the best and only way...

Desired usage:

Sorry, the last paragraph is a kind of offtopic, it just came to my mind.

Summary:

• Lets do evolution, not revolution. I feel like we are trying to replace whole system which more or less fits for 95% use cases with a system which will theoretically solve all current conflicts and problems (the last 5%), but will be less friendly for developers and totally different. It seems like replacing bunch of problems (but which are known and we know how to live with them) with another set of problems (most of them unforeseen now).
• Current problems of the translation system could be solved 1) with forced usage of context together with good standard, how to create good context name and 2) with fuzzy automatic translation system which will automatically generate suggestions for the untranslated strings and allows to quickly fix the changes.
• Site level string overrides could be done, "there is a module for that" (Drupal TM) http://drupal.org/project/stringoverrides. Global site strings overrides could be done via translators interface, big changes could be done directly in the theme layer. Might be just proper tool for importing/exporting the local string overrides is missing.

Despite I don't agree with the idea of "moving interface strings entirely out of Drupal code files?", thank you Gábor for opening and promoting the discussion about it. The Drupal interface translation system needs to be improved for sure.

While, as wojtha points out, it is annoying that minor text changes means that the translator has to update the string, it is a feature, not a bug. Consider a module with a text field with the following changes in the #description:

v1.0: "User to mall daily roports to."
v1.1: "User to mail daily reports to. "
v1.2: "User to mail daily reports to. Alternatively an e-mail address."

Evidently a new feature was introduced in v1.2. With t(), this will require the eyeball of the translator for each version, which will be annoying going from 1.0 to 1.1, but a Good Thing for version 1.2.

If the key is the logical mymodule.form.reports.user, how will the translator discover that the meaning of the text has changed? Are we to require versioned keys ala mymodule.form.reports.user.v2? I'll wager good money that that's a requrement that'll slip through the cracks often, resulting in old out of date translations.

After having read many of the comments, here are some more thoughts.

• That each string has a unique id doesn't mean you always need to write it. Most of the short strings around are either menu texts, form texts, etc.. These can get an automatic id from some string parser: menu.item.my/menu/path.title, form.my_form_id.field_name.description etc...

So you'd only need to add a explicit string id for random texts in the module that are not part of a known structure.

• Moving long texts out of the code can actually increase performance and readability. Performance because we may save some thousands of lines to be parsed for each page request, we only load the strings needed for each page. About readability, I don't see how a 20 lines help text wrapping each line with t() and '<p>' is more 'readable' than the same text on its own plain text file. So about performance I don't mean it will be faster, I just mean we cannot make the straight assumption it will be slower.

• About having the defaults in the code (for short texts) it sounds good to me if that helps. We just need to kill the assumption that default will be always English. Still we could require defaults to be English for contributing the module.

When you are developing a module for a non English site, the economics of translating the default texts to English in order to contribute it doesn't pay back at the moment because then you need to go through the full translation/update workflow for every text. But if we've got a key for every text, just replacing the default strings by English ones and moving the non-English ones to a separate file would make the module 'contributable' while keeping your original module working on your site without more work.

So it sounds good if all the contributed code needs to have string defaults in English. As long as all texts have keys, translating it and contributing and still keeping it working for your original language would be way easier.

Also having string keys can make code developed in a different language readable for English speakers. See the difference:

So please, let's make some important difference: Being able to write non English strings in code doesn't mean we need to open contributions to non English texts. It just means it will be much less work for me to translate and contribute the module while still using it for my original site.

The current system makes code written for a different language both not translatable and non readable (for English speakers).

The real big problem is not exactly having the text in the code. It is not having a language independent key to refer to it (override, translate, etc). So just adding keys (still having source strings there) would fix most of it.

We don't need to force everybody to move texts out of the code. But we can enable that option by having strings with a mandatory key and an optional default in code.

So, the whole point is: we need to enforce unique string keys. Whatever else doesn't matter that much.

Before I started to work with Drupal I was involved in some projects that used gforge (http://gforge.org/). Its code uses the key string stranslation method that Gabor is proposing.

From a developer standpoint it was a pain. When working in a project you have to be constantly updating the translation file. Some times things broke and the keys ended up appearing in the website everywhere.

When I switched to Drupal I felt that the t() method was more friendly. It has its glitches, but the other method will have too.

There is one important use case that made me consider a keyword-based system:
Text that is supposed to be specified by the user / "the client", and not be hardcoded.

Typical situation: The client wants a custom-desiged page with a fancy javascript widget, a list of latest whatever, and some text boxes with intro and description text. The developer decides this is not a node type, it is not a view, it is not a panel, but a custom hook_menu() page callback.

The client is still undecided about the text to put, so the developer can either
- hardcode something, and offer to change it later.
- fetch the text(s) with variable_get(), and provide a custom admin form to customize the text. The variable has to be added to i18nvars.
- Use t() in combination with strings_override module.
- Or, if the site does not have any English, the developer can simply hardcode a t() string, and the client can use the translation tools to customize it.

I think we do not need to make a hard decision here.
- We can keep the t() system for the biggest part of the interface, especially for most of the strings in contrib and core.
- We add tools similar to string_override, but better integrated in the overall architecture. These allow to customize the English original, and to make case-specific overrides for any translation.
- As discussed above, we might add an option for non-English original strings. However, those non-English strings need to be cleaned up before they go to contrib.
- In addition, we add a keyword-based system, for those cases where the developer has only a very rough idea of the intended string. This would be similar to variable_get() + i18nvars, but we get the admin form for free, integrated in a translation browser with search. This solution can grow in contrib in D7, and go into core in D8.

I strongly agree with the key approach having worked so much with multilingual web sites and application. A default language make it much harder for multilingual.

One thing I would add to the proposition is the ability to pass an array to the function to fill some placeholders. Consider this error message in two languages.

2 search results were found.

La recherche a générée 2 résultats.

The ideal way of achieving this is with an array.

English string : "{0} search results were found."
French string: "La recherche a générée {0} résultats."

and the function call:

t('search.results', [$results->count]);

What about changing the t function slightly?

This would allow string keys, but not require them. If a key is provided, find the translation by key. Otherwise, search by the passed string.
This would be the best of both worlds. I apologize if this has already been suggested. There's a lot to read here.

As this thread showed up on Twitter, it got me thinking.

While I'm still a big fan of t(), I am aware of it's limitations and issues. And looking at the state of D8 lately, I'm not sure that we haven't out-grown it.

Grabbing some of the ideas already mentioned:

Take this:

The first argument is the key, the second a fallback value. Note the wording here, "fallback", not "default". It is defined as a fallback in "developer English", which, in my experience, tends to be terse and jargon filled.

The idea of the fallback is to provide exactly that while developing, and providing something string extraction tools can use as a comment to jog the memory of the developer when doing the proper "translation" to the default English. In addition, it provides a human readable string in the code, so you don't have to mentally translate 'username.label.login_form.user' when scanning the code.

And we'll not even frown upon

As the fallback is internal, and not for public consummation, like a code comment would be. What we will frown upon, is contrib modules without the default translation into English that'll hide the internal strings.

The key is the real thing used for lookups, but it's done recursively, like so:

username.label.login_form.user
username.label.login_form
username.label
username

Which allows core to have a translation for username, and my_auth_module can then use:

for it's login form and reuse cores translation, while still allowing for a translator to target its form specifically.

It encourages reuse:

username.label.login_form.user
username.label.user_edit.user
username.header.user_list.user
username.label.node_edit_form.node

can all fall back to username, whlie

username.label.irc_login.irc_gateway can override it to "IRC Nick".

As for the translation files themselves, it could be handy if they supported pointing one translation to another 'show.button.image_details.image' to 'view' for instance, and "soft" specific translations, so that it can define a translation for 'username.label.login_form.my_auth_module' that'll only be used if a more general translation doesn't exist. Or maybe instead being able to promote a specific translation to a general if there is not, I'm not entirely sure on that one.

Some sets of patterns will need to be discussed, of course. I came up with these while pondering:

<short string>.<type>.<form/page>.<module>

Where:
short string: username, view, google_app_id, help, ..
type: label, button, description, header
form/page: form_id, router name
module: well, you get it...

And perhaps

entity.<type>.<what>

Where:
<type>: view, node, user, order, whathavewe
<what>: name, plural, perhaps

And

help.<module>.<topic>

And yes, lowercase separated by periods. That's not up for discussion.

It'll be a bit more difficult to implement that the current t() function, and performance is probably going to be a challenge, but it'll bring some advantages over both t() and the simpler "just keys being looked up" approach.

So, does this move the discussion along?