Posted by brmassa on May 28, 2007 at 8:41am
Guys,
Im developing a module called Live Translation. Its divided in three submodules:
- Live Translation: all end users will update all strings automatically. no more po/pot files.
- LT Extractor: similar to POTX > extract all translatable strings from all projects that your site hosts. It also provides a dynamic XML with updates, so users can download it.
- LT WebTranslation: provides a online interface to all users translate all strings. It uses OG to manage who can translate, sho can create a suggestion and who cannot help.
I really believe that decentralized translations (relying on each module developer) doesnt work. Providing a central repository to all translations and opening to all community is much more efficient.
the module is still beta since it scans modules instead projects. im gonna start the integration with project* modules soon.
regards,
massa
Comments
collaboration?
I tried to read all your README and introductions, but it is not explained why did you use a custom baked XML distribution format. Even Launchpad Rosetta which you point out is using Gettext PO imports and exports. I don't see why a custom XML format helps here. You need to develop your own extractor, your own importer and exporter and server component as well as an editor. Meanwhile, Drupal 6 goes on supporting Gettext files even better then Drupal 5 did and the POTX extractor will be updated for new constructs in Drupal 6.
Gettext files have the definite advantage that they have strong desktop support, so if you need to fix a few things yourself, or you prefer desktop editors to work on the train for example, you can still do so. KDE even has "OS level" support for gettext files, so you can get an instant overview of translation status on your file system (look: http://kbabel.kde.org/img/previewKonq.png). That is powerful!
It is important to point out that web interfaces for translators, and even automatic updates are the way to go for more involvement in Drupal translation but it should not be started with a complete clean slate as far as I see.
Anyway, let's say you have a strong reason to use your custom XML format (and not gettext or some of the existing XML formats already developed in this field like llXML, TMX or XLIFF), why do you fork potx module? Do you see a possibility to actually collaborate with the community to reuse what is already done? What about submitting patches to support different output formats (or a generic array format, which you can transform to anything) in POTX module, so you can use that as an API to generate your XML files? Forking the code of a module which is well maintained and follows Drupal core development closely does not seem to be a good idea for collaboration to me.
We agree on the goals, just not the tools you employ to get there!
collaboration!
Gábor,
heheh i think you got the wrong idea. In fact, you also gave me an idea.
the module is intended to be used by a server, like drupal.org (except the Live Translation itself, but, meanwhile, will be offered togheter). The communication between the server and the clients (drupal sites) is made thru XML, BUT (now the part you gave me a light) can be made thru PO "protocol". i can change this easely. (i didnt know about llXML, TMX or XLIFF: thats why i created one)
in fact, it doesnt matter much, since this operation is invisible to user. the user only press "update strings" link and everything goes automatically. If only one string was updated/translated since the last time the user updated, only one string will be downloaded. thats why i told "no more PO files". everything comes from the server, on demand. as i just told, it can be thru a XML or POT file.
Im planning (and building right now) a PO importer/exporter (its in the module issues for couple of days). Translations can be imported so the work will not start from scracth.
POTX is a module that every DEVELOPER must have. LT is module that every USER must have (and at least one server, like drupal.org). The approach is not the same. i customized the POTX functions because i didnt figured out how retrieve the strings. i dont need a physical file as a result. i need the strings array only in order to store into a table.
about web interface: im working to create a very helpful interface. Like Rosetta, it now gives some tips about new lines and "dont translate !variables". i use Kbabel to translate offline and i know who good a offline program is. WIth the PO importer/exporter, its going to be much faster, since you can download only untranslated strings from each module.
again, you gave me the idea to change all XML approach to PO approach. it, in fact, will make the module smaller.
best regards,
massa
great!
Drupal itself has a PO exporter/importer, and those can also be made more generic if you wish. Let's look into what can you reuse from what is already there first. It should save some money for your sponsor hopefully. There are still a few days left until the Drupal 6 code freeze, so if Drupal core code itself needs some generalization to support better import and export, we should act now. It would be good to be able to build this on existing functionality for Drupal 6 (I understand it is not possible with Drupal 5, but it would be nice to look forward).
Maybe you look at POTX module as a developer module but it can be thought of as an API module too. Maybe it does not support generation of arrays instead of files yet, but if more people would like to build on it, it should be made so. Important rule: Do not think of existing, in-development code as untouchable! It is important to reuse what is already there to distribute development effort, not copy and fork the code. So when POTX gets updated your module will automatically reuse its new features, like support for new localization constructs in Drupal 6, and your work will not get immediately obsolte and unmaintainable. Let's look into how you can reuse that and get POTX module modified to support you in that.
POTX and extractor
Gàbor,
i must confess: i thought POTX was the same as extractor.php from template. im doing now the total integration with POTX now: all string extraction operations will be made by POTX. i did wanted to use these functions since the begining.
the LT module goes to www.SERVER.org/update_strings to update the strings. POTX only returns a new file. i need POTX returns the content only. so i can print it as www.SERVER.org/update_strings page. for now, i have to copy potx_select_form_submit() and _potx_write_files() functions to customize.
Drupal 6: i really believe that automatic updates are much more efficient than downloading individual-po-files-that-depends-on-each-developer. so, if we believe that the translation server can be ready before D6 release date, i think its quite useful to implement the LT into locale. LT itself (not the extractor or web translation) is a very small module that only import "PO files" now form a given server.
Im customizing the PO importer and exporter because the tables are different. LT Web Translations uses equivalent tables but has a new ingredients: update time and user. If locales po importer/exporter could isolate all SQL instructions to specific functions in order to reuse most code as possible, it would be nice. Since all import/export process is quite concatenated (on function calls the next instead a big function calling every small piece), if i have to customize the first function, everything should be rewriten as well.
well.. im gonna work today on using POTX and PO files. by the end of the end, im going to publish the new version and discuss more details.
regards,
massa
ADDED: since you can alter the final Drupal 6, here is a tip: _locale_export_po() involves SQL searching, po content building and files recording. it would be much more useful if the second part, po content building, would be a separate function. POTX and LT would be able to generate PO files with custom strings and with/without file generation.
refactoring
Well, many people think if they set up a site, it should not change suddenly. Like the Hungarian translation team debated different translations for 'story' and 'theme' and 'watchdog' and so on. I for one don't want my site suddenly have different strings because some translators thought it sounds better. So it is important to distinguish between the "automatically do everything for me" and the "I want to control what is done" way and somehow support all colors of the rainbow inbetween. Drupal 6 now automatically imports translations of modules from the locale file system as you enable them (in install time and later), which is kind of inbetween what we had in Drupal 5 and what you have. We did implement this with keeping existing translations in the database, when importing automatically, so what you expect to be there keeps being there. That's a gesture to the control freaks, while trying to be as automated as possible.
Let's see how small and useful you can get this LT module, and we will talk about maybe integrating it in Drupal 6 or at least better supporting it as a contrib module.
Wouldn't it be possible to have these in a small table joined to the locales_target table? I would do it that way, but I did not look into your code to see what other implications might it have.
We are very busy getting many new stuff into Drupal 6, but "luckily" there is no work around these functions, so feel free to submit patches which we can quickly test and get into the system. Refactoring these parts would be good for the future anyway. I did a great deal of work for Drupal 6 to reorganize, rename and clean up some functions in locale.inc so it should be much cleaner there. Refactoring them for reusability would be fantastic.
Modifications done!
Gàbor,
Lets go:
1* About automatic update: the module will be as much automatic as importing po files when installing a new module. On LT, "off", "only new strings" and "ovewirte all strings" options are already functional.
2* LT integrated on Locale: its possible. i can redesign the module to be a patch to locale once all tests been done. (only the user part. the extractor and web translation will continue to be a independent module)
3* Using locale tables: i thought about it. no way to use locale tables. i need to create new ones. Web Translation records all "suggested" translations also, user id and update date.
4* Changing Drupal 6 import/export locale: well, since neither POTX nor Live Translation have a D6 version, im going to provide a patch to D5. i cleaned up many things and its more refined than ever. I created a micro-API, in order to allow creation new importers and exporters (llXML, TMX or XLIFF for example) on the future. POTX will be benefited too. Once you approve, i can work on porting. But... where can i post the patch?
5* im going to commit all changes to LT project in the next few hours. (well... im going to include the patch against D5 as well)
regards,
massa
patches against Drupal
Drupal 5 does not accept API modifications, just bug fixes. Your modifications are API modifications, so they should go to Drupal 6. (Actually, Drupal 5 has a different maintainer, I am only one of the maintainers of Drupal 6). Submit "task" issues against Drupal 6 here: http://drupal.org/node/add/project_issue/drupal/task
Gábor, i know about this.
Gábor,
i know about this. what i meant is for now, since POTX doesnt have yet a D6 version, im gonna publish a patch to D5 as a preliminary test. as soon its starts to run well, i convert everything to D6.
thinking better... im gonna convert right now all module to D6. it will take some time however.
regards,
massa
Updated Status
Guys,
I did some work on this module:
1* I finally split the User side (Live Translation) and Server side (LT Server).
2* Im hosting a server on http://Drupal.Titanatlas.com. Its now operatinal (its only a beta so errors may occur).
3* By the next few days im gonna contact all Translation Leaders on Drupal.org to also work on this, importing PO files and coordinating the process.
4* If you have a PO file with the translation of a contrib module, post it on http://Drupal.Titanatlas.com forum. Im going to commit it as soon as possible.
5* Here in Brazil, from Thursday to Sunday, it is a holiday and im going to travel. I promisse to answer everything as soon i come back.
regards,
massa