Contributors mailing list archives
Translations handling in OCA repositoriesby
When a commit gets any repository, Travis CI performs an stage that installs Odoo, clones repositories, installs the corresponding modules, and then exports the POT file from Odoo and push to Transifex all resources (POT files) and translations (PO file). This way, we assure that all the modifications made on strings (and possibly in PO files included in the PRs) are dumped to Transifex. This is performed basically by the script https://github.com/OCA/maintainer-quality-tools/blob/master/travis/travis_transifex.py
First problem we faced in the past is that there's no way to obtain something like "all the modified files", but the script has to check translation per translation if there has been any change, taking the PO file from Transifex and from GitHub, and comparing both at string level. This makes the script extremely slow as there are a lot of requests, and we also have to deal with the rate limits applied to both GitHub (5000 requests per hour) and Transifex (5 per second, 100 per minute for GET, 2 per second and 25 per minute for POST, and 5 per second, 50 per min for PUT - There's also a general limit of 6000 requests per hour - http://docs.transifex.com/api/#api-rate-limits). There's also no reliable way to check these rates (we didn't get consistent error codes - maybe nowadays this has changed), so the solution we took was to have some rest time between operations. The script needs with this the whole weekend to be completed for all the OCA repositories.
- There are several commits that have been made in the same repository coinciding with the v10 taskforce effort.
- The module I translated already contains a partial translation in the repository.
- The Transifex > GitHub script hasn't been launched in between.
I have tried several approaches to solve this without success: deleting empty entries, trying to remove files, detecting changes... One limitation in Git about the absence of timestamp metadata in files (check official statement about that in http://stackoverflow.com/a/1964508/6309) has also made this harder. Another limitation is about timestamps in Transifex, that are naive (not timezone aware).
But finally I have reached one satisfactory solution marking translation files with the timestamp of the last commit that modifies them (converted to UTC), and let Transifex client to skip these files. You can see it in this commit: https://github.com/OCA/maintainer-quality-tools/commit/4cad6e57feaabfb805a9c7972ac8b8c53924d9b9.
The only limitation for now is that you can't change a translation in GitHub and also in Transifex in the same synchronization window, which should be hard to reach if the rule is always to translate on Transifex.
I hope this explanation serves for anyone interested in the process. Any suggestion in it is welcome.
Open Source Integrators, Daniel Reis
Juan José Scarafía
Camptocamp France SAS, Alexandre Fayolle