Create Task
Maniphest T13519

More consistency in injection of translation artifacts inside project repositories
Open, Needs TriagePublic
Actions

Description

Current status

Right now translation handling shows some inconsistency depending on the type

desktop/json/appdata translations

The content of those po files is extracted and injected every night into the respective desktop/json/appdata files.

Issues:

  • it produces a bit of work when merging or rebasing content from different branches (for example when updating a work branch).
  • it makes editing of those files more complex, for both humans and machines (see the discussion about adding release notes to appdata files in T10972 ).

other translation artifacts

Those include: regular po and ts (generated from po) files loaded at run-time, translated documentation, js scripts for translation support, translated data files (for edu programs).

They are added to the tarball at release time by releaseme or other release scripts.

Issues:

  • release process a bit more complex;
  • easily forgotten when testing. I18n-related cmake macros allow for loading the translations during compile time, but it needs to be explicitly enabled.

New scenario

Periodically copy all translation artifacts inside each repository (at least daily) under po/.
Server hooks should be added to prevent anyone else but the inject script to change those files.

Benefits

desktop/json/appdata translations

The translations for those files will be injected into the respective files during compilation times, as other projects do, and they won't be installed.

other translation artifacts

They will be copied according the expected structure under po/, as releaseme and other release tools would do.

Issues

Large binary artifacts like data files and documentation images may make the git repositories bigger. Should those be excluded from the injection? Or maybe the LFS should be enforced for them.

Work needed

  • change the ki18n_install macro to support the inject the translations of appdata/json/desktop files at compile time, and exclude them from the installation. This will require a bump of the required Frameworks version for all applications who need to use it. Maybe this may wait for Qt6.
  • figure out the impact of large files in git;
  • figure out whether the two parts of the problem can be implemented separately.

Related Objects

ltoscano created this task.Aug 19 2020, 6:01 PM
rempt added a subscriber: rempt.Aug 22 2020, 11:08 AM
rempt added a comment.Aug 22 2020, 11:11 AM

Note: any procedure that downloads anything at compile time is broken by design. The git repo for any project should be complete in itself and contain everything that is needed to build a binary releasable artefact. Every time anyone builds a project from a repo, the result should be the same as a build from a source release archive.

pshinjo added a subscriber: pshinjo.Aug 23 2020, 8:56 PM

Periodical injection of PO files into source trees can make developers to avoid cross-checking git and SVN when there is a bug related to translation. Let's take an example of Krita bug https://bugs.kde.org/show_bug.cgi?id=408481. Although it was not Korean translation's fault, but if it really was then developers don't have to check Korean translation SVN tree and possibly bisect the revision where mistranslation had been introduced.

rempt added a comment.Aug 24 2020, 8:40 AM
In T13519#238038, @pshinjo wrote:

Periodical injection of PO files into source trees can make developers to avoid cross-checking git and SVN when there is a bug related to translation. Let's take an example of Krita bug https://bugs.kde.org/show_bug.cgi?id=408481. Although it was not Korean translation's fault, but if it really was then developers don't have to check Korean translation SVN tree and possibly bisect the revision where mistranslation had been introduced.

You got that wrong. Not having the translations in the project's git repo means that developers NEVER see the translations. And fixing that Krita bug was done without checking any translations either: in fact there was no issue with any translation whatsovever.

ognarb added a subscriber: ognarb.Aug 24 2020, 10:00 AM

From my point of view, using that for the all the kde repositories would make a lot of sense and would also help distro that provides nightly build like openSUSE Krypton to also package the app translations with their KDE packages. Another advantage is that developers can more easily test if the translations are working on their app or not.

Storing the translations in the repo is already done with the StaticMessage.sh scripts in various website repositories. Currently, the StaticMessage.sh also post-process the po files, but I wouldn't mind transforming the po files to markdown/yaml/... at deploy time in the binary factory instead of using scripty for that.

Concerning the repo size, something I found out is that the po files doesn't increase the git repo size as much I as first thought. The images will be more problematic but I think git LFS should solve most of the problems here.

In terms of the bump of the required framework version, I fear that there is no way around and it might be easier to force all apps to use a recent version of the frameworks v5 than forcing all apps to update to Qt6/kF6 at the same time.

huftis added a subscriber: huftis.Aug 24 2020, 6:20 PM
In T13519#238083, @ognarb wrote:

Storing the translations in the repo is already done with the StaticMessage.sh scripts in various website repositories. Currently, the StaticMessage.sh also post-process the po files, but I wouldn't mind transforming the po files to markdown/yaml/... at deploy time in the binary factory instead of using scripty for that.

Yes, that sounds like the better solution. The PO files should really be considered as source files, and then it makes sense that any files generated from them (.mo/.xml/.yaml/.desktop files) should automatically be generated at compile time.

The desktop files would then be .desktop.in files, and intltool (intltool-merge) could be used to generate a .desktop file from the .desktop.in file and the set of .po files – just the way it’s done for normal (non-KDE) applications which store the PO files in the source repository.

pino added a subscriber: pino.Aug 24 2020, 7:00 PM
In T13519#238118, @huftis wrote:

The desktop files would then be .desktop.in files, and intltool (intltool-merge) could be used to generate a .desktop file from the .desktop.in file and the set of .po files – just the way it’s done for normal (non-KDE) applications which store the PO files in the source repository.

intltool is not needed, gettext can do that for desktop files and for any kind of XML files (AppData included).

JSON files still require something else though.

rempt added a comment.Aug 24 2020, 7:18 PM
In T13519#238118, @huftis wrote:
In T13519#238083, @ognarb wrote:

Storing the translations in the repo is already done with the StaticMessage.sh scripts in various website repositories. Currently, the StaticMessage.sh also post-process the po files, but I wouldn't mind transforming the po files to markdown/yaml/... at deploy time in the binary factory instead of using scripty for that.

Yes, that sounds like the better solution. The PO files should really be considered as source files, and then it makes sense that any files generated from them (.mo/.xml/.yaml/.desktop files) should automatically be generated at compile time.

The desktop files would then be .desktop.in files, and intltool (intltool-merge) could be used to generate a .desktop file from the .desktop.in file and the set of .po files – just the way it’s done for normal (non-KDE) applications which store the PO files in the source repository.

Yes, that would be absolutely ideal, in my opinion. Repo has all the sources, build puts it together, but no downloads from anywhere during the build process.