Better (online) localization
Open, Needs TriagePublic

Description

Description

Most likely, I am knocking at an open door, but I think it's worth mentioning here. KDE definitely needs a better system for managing localization. The mailing list topic is still hot about the issue, as the exact requirements are not clear yet. Personally I don't mind if its Weblate, Pootle, Damned Lies or Pontoon, anything would be an improvement.

To give an example of the high technical overhead there are at time of writing 57 subpages on contributing, a notable amount of which involve dealing with configuring tools for GUI translation. This current process provides a high contribution entry barrier compared to online localization tools where contributors simply create an account, enter a suggested string, and a designated person(s) for that language approves/denies w/ comment. For an example of how simple contributing localization could be, please checkout the Weblate demo: https://hosted.weblate.org/

Also a related issues, which is even worse: KDE heavily depends on Qt, but Qt's localization is horrible. I would like to see an effort from KDE's side to make translators life easier in this regard.

Current issues:

  • technologies used include dated stuff like SVN
  • apparently integration with existing systems (like identitfy) is sub-par
  • documentation is not easy to find

What it will take

  • Improve localization system and documentation

How we know we succeeded

  • More people start translating
  • Translation is an easy way to start getting involved

Relevant links

I am willing to put work into this

  • Balázs Meskó (@meskobalazs) - testing, feedback, Hungarian l10n maintenance
  • Scott Anecito (@sanecito) - testing, feedback, Japanese l10n maintenance
  • Alexander Zhigalin (@zhigalin) - testing
  • Guo Yunhe (@guoyunhe) - testing, feedback, Simplified Chinese l10n maintenance
  • Ovidiu-Florin BOGDAN (@obogdan) - deploying & maintaining translation platform, testing, feedback, Romanian l10n maintenance

I am interested

There are a very large number of changes, so older changes are hidden. Show Older Changes
meskobalazs updated the task description. (Show Details)Jun 11 2019, 7:03 AM
meskobalazs updated the task description. (Show Details)Jun 11 2019, 7:09 AM
pshinjo added a subscriber: pshinjo.

Shall we merge T11092 onto this or vice versa? I think those two have the same goal.

Fuchs removed a subscriber: Fuchs.Jun 13 2019, 6:39 PM
GB_2 updated the task description. (Show Details)Jun 13 2019, 6:39 PM
GB_2 updated the task description. (Show Details)

For the record: the Qt translation system is not generally used for KDE software. Even if a program does not use KI18n, the translation file is converted to the gettext format.

meven updated the task description. (Show Details)Jun 14 2019, 8:47 AM
meven added a subscriber: meven.

For the record: the Qt translation system is not generally used for KDE software. Even if a program does not use KI18n, the translation file is converted to the gettext format.

It might mean the localization of Qt itself, which I also find confusing for beginners or casual contributors.

meskobalazs added a comment.EditedJun 14 2019, 3:07 PM

It might mean the localization of Qt itself, which I also find confusing for beginners or casual contributors.

That's what I meant. Frankly, the translators shouldn't need to know to use advanced Git features and Gerrit to send in their work. If the KDE project could provide a simple way to automate sending in the translations, that would be terrific.

ngraham added a subscriber: sanecito.
ngraham added a subscriber: ngraham.

This Task now has two duplicates. Looks like the localization infrastructure is a bit of a point point for some.

This is not a new information, and it has been discussed for a good while. It's just that so far no one proposed or was able to implement a solution that fulfill all requirements raised and discussed on the translator's list.

It might mean the localization of Qt itself, which I also find confusing for beginners or casual contributors.

That's what I meant. Frankly, the translators shouldn't need to know to use advanced Git features and Gerrit to send in their work. If the KDE project could provide a simple way to automate sending in the translations, that would be terrific.

I agree that the current workflow for translating Qt is even more complex than ours, but on the other hand this is unfortunately out of scope for the KDE community. It is a totally valid discussion for the Qt community.

sanecito added a comment.EditedJun 25 2019, 7:32 PM

Thanks @ngraham for catching my miss about seeing there's already a task.

This is not a new information, and it has been discussed for a good while. It's just that so far no one proposed or was able to implement a solution that fulfill all requirements raised and discussed on the translator's list.

Even so, just so others who haven't attempted localization before I'll just copy paste my description from my duplicate task as I believe it provides context and shows just how painful it is:

Currently it is extremely painful to jump in and help translate KDE due to the existing complexity for even those with a technical computer science background. There are 57 subpages alone on the topic. While some of these subpages are things like mailing lists, etc. a good majority of them involve tools, GUI, and stuff like SVN.

Compare the complexity of reading let's say, 30 of those pages, installing and configuring the tools, overhead of dealing with SVN/commits to name just a few compared to something like this Weblate demo where you just create an account, click 'Projects' or 'Languages', click the item of note, and click 'Translate'. The first option which is what KDE has presently is not only intimidating to non-computer science background folk and can scare away potential localizers, but there's the overhead cost of manually dealing with SVN. Using something like Weblate is user friendly; the workflow is arguably more simple and productive which lets you focus on the localization itself.

As a localizer, I've used various web based tools like Pootle, Transifex, Zanata, and Weblate. I believe Weblate philosophically best aligns with KDE given it's libre, and Weblate is the most reliable and allows for self hosting. Weblate also has several useful features like new strings notifications to automatically, easily keep translators up-to-date on new work to be done.

For more context about tool choice, the translation team of The Monero Project of which I am a part, has talked to Pootle (what was initially chosen), and after trying to get some bugs resolved w/ Pootle found out Pootle's maintainers are trying to hand off Pootle. Zanata is just bad, their provided instance has been down for 1+ week at a time w/ no communication. Transifex is non-libre. I can't speak much to Damned Lies or Pontoon, but Damned Lies is more complex to use and Pontoon is arguably built for Mozilla, not for general use.

I haven't been part of the mailing list or wherever these conversations have taken place. Is there a link that can be referenced of said requirements (I'm not part of the mailing list, but I guess I will join now)? Having web translation is something that I'm quite passionate about so anybody can easily translate Plasma Mobile, etc, and I don't quite see what requirement(s) say, Weblate, fails to meet.

There other requirements in terms of "don't break the existing workflow for teams who want to translate directly from SVN" (because there are advantages from having the entire tree at your disposal, and using the offline tools).

The archives of the list are here: https://marc.info/?l=kde-i18n-doc
There is a recent discussion about this in the last months (look for the threads titles "Use Pontoon for l10n").

sanecito updated the task description. (Show Details)Jun 25 2019, 8:26 PM
lydia added a subscriber: lydia.Jul 19 2019, 10:29 AM

The proposal is still focused a bit too much on the solution. What is it that we actually want to achieve? Make our software available to more people speaking different languages?

I had a nice conversation about this with @aacid at the Valencia Sprint.

What transpired is that the current tools although showing their age are mature and translators are accustomed to them, so they don't really need a new tooling.
That being said the documentation is not in great shape and makes it hard for new translators to get involved.
The situation is fine for "major" (read Europeans) languages that are well represented in our translator community.
What is lacking is an easy way for translators to add their work for languages where we lack in translators especially.
Languages like Japanese and Korean are less than 70 % translated, Arabic, Hindi, Bengali and African languages are worse, despite those are spoken in countries representing millions of potential users and community members.
You can have an overview of our translation status in https://l10n.kde.org/stats/gui/stable-kf5/team/
And compare this to the list of languages by number of speaker https://en.wikipedia.org/wiki/List_of_languages_by_total_number_of_speakers

That is where a new simpler online tool might work well.
That is what I feel should be the stake here.

So I believe you are right @lydia , the end goal is simply put to make our software available to more people speaking different languages

Those are really useful points and stats. I think they should go in the task description.

lydia added a comment.Aug 3 2019, 4:03 PM

Are you going to adapt the description based on the comments so we can take it into account for voting? When done please move it to the ready for voting column.

lydia assigned this task to meskobalazs.Aug 3 2019, 4:07 PM
lydia updated the task description. (Show Details)Aug 3 2019, 4:16 PM
lydia added a comment.Aug 13 2019, 8:50 PM

Quick reminder that there are two days left before the voting starts. Please make any changes you still want to make soon and let's move it to the ready for voting column.

Languages like Japanese and Korean are less than 70 % translated, Arabic, Hindi, Bengali and African languages are worse, despite those are spoken in countries representing millions of potential users and community members.

Hi.
Since Arabic was mentioned, I have to react :)
Actually, Arabic status in KDE is the same as in GNOME, and in almost every other open-source application. Mainly because: translators don't want to learn proper software (like Lokalize, which is pretty simple), or becuase they don't give a damn about correct language. Most of the apps at Google Play are badly translated into Arabic, and even machine translated. Things like typos or consistency or formatting doesn't matter for them. What matters is that the text is in Arabic letters, even if it most of it is transliterated or written in a slang or whatever.
This is the same even in proprietary software and games.

Now regarding this propsal, I am still against using anything like Transifex, Crowdin or Pootle, Weblate. These are hard to maintain and makes it difficult to ensure consistency, etc etc (I've written my opinion on the mailing list several times). Things like Damned Lies that GNOME uses is the best imho because it have the good UI/UX and a good level of maintaining and communication between team members. And let's be honest, offline translation with a proper application (with TM, Glossary, etc) is better than an online website which will be laggy for almost anything (I'm pretty sure because of how LO's Pootle is laggy). Add to that people who will jump between files, or translate just the simple “open save are you sure” sentences in all files, and create a whole mess with that.

Thank you.

sanecito added a comment.EditedAug 21 2019, 4:44 PM

These are hard to maintain and makes it difficult to ensure consistency, etc etc (I've written my opinion on the mailing list several times).

Can you elaborate on this any here in the comments since this is where the discussion about the next three major KDE initiatives is taking place? Have you maintained say, a Weblate instance? Maintaining Weblate via upgrades is relatively simple and largely scripted: https://docs.weblate.org/en/latest/admin/upgrade.html#generic-upgrade-instructions. It should be easy to maintain consistency; people make suggestions and designated people either approve or deny with comment.

And let's be honest, offline translation with a proper application (with TM, Glossary, etc) is better than an online website which will be laggy for almost anything (I'm pretty sure because of how LO's Pootle is laggy).

Monero's self hosted Pootle instance when we used it was not laggy (although we now prefer Weblate over Pootle and have migrated). Weblate also has many if not all the features of offline translations tools such as a Glossary (https://docs.weblate.org/en/latest/user/translating.html#managing-glossaries). Do you have any lag issues so severe with say, the Weblate demo (https://hosted.weblate.org/projects/) that would make dealing with the SVN overhead preferable to clicking a "Suggest" button as a translator?

Things like Damned Lies that GNOME uses is the best imho because it have the good UI/UX and a good level of maintaining and communication between team members.

Damned Lies is simply a UX for communication. It does not solve the high SVN/read several dozen wiki pages to get started roadblock. But yes, sure if Damned Lies would lower the barrier to engaging comms, that's something that can be considered. Given people created three separate tickets for online localization tools though and not mailing list tools, I would argue the primary focus in the localization space should be the localization tool, not localization comm tools.

Add to that people who will jump between files, or translate just the simple “open save are you sure” sentences in all files, and create a whole mess with that.

This is assuming the extremely negative. Tools like Weblate provide suggestions based on similar strings elsewhere for consistency. Thus you are likely to see consistency for simple strings like "Open", "Save", etc. I find it hard to see how people would create a mess with such basic strings, especially if the designated language leads are approving/denying suggestions.

The question I would pose to anyone who is against online translation tools is this: Do you honestly believe linguistically technical speakers should be capped from contributing to KDE because of having to read 57 subpages alone on familiarizing them with getting setup and SVN? As opposed to just creating an account and contributing via an online tool?

People can continue to use their SVN workflows, but as a FOSS, we really need to enable means for people to easily contribute rather than opt for hurting the project by keeping status quos with high entry barriers.

sanecito updated the task description. (Show Details)Aug 21 2019, 4:55 PM

Moved to voting taskboard as it's been a week since lydia's comments concerning voting starting on Aug 15th and there have been no updates to the task. I'd rather not have this miss out on voting so I've moved it over. If anyone wants to make edits please do so, but make sure it gets moved back to the ready for voting label.

I've checked Weblate in more details now, as well as the documentation. hmmm, not very bad. Proper permissions, fast browsing, checks, good UI, good editing box, etc etc..
We tried once to translate KDE docs by pootle, but it was a horrible thing at that time becuase of bad SVN integration. By that we had to have each single file translated with anything so that it can be uploaded and pushed to the server, or we can't push anything (not sure why, my teammate was doing that back then). I've removed all those files now.

I also translate LibreOffice and the translation Pootle instance is really laggy. (Huge project, diffrent versions as well as docs, plus many languages).
I don't have much experiance with Weblate, but I have with Transifex and Crowdin. Transifex have a really bad UI (laggy a bit, small fonts, bad placeholder implementation, etc). Crowdin is good, really.
Tbh, I just prefer offline translation. If online can prove it's better, why not.

By the way, when new comers want to translate, they find my email (somehow) and just contacts me. I then send a PO file and they translate it. Once they're done, we check it, and I push it. Simple as that, plus learning a translation app (Lokalize, poEdit, Virtaal).

sanecito added a comment.EditedAug 21 2019, 7:40 PM

I also translate LibreOffice and the translation Pootle instance is really laggy. (Huge project, diffrent versions as well as docs, plus many languages).

Yep, while Monero is nowhere near the scope of LibreOffice, we had annoying issues like inaccurate untranslated/translated string counts when using Pootle. When we pressed Pootle repeatedly they expressed interest in handing off maintainer-ship to us. Coupled with there being no release for the past 2 years, and next to no commits on the 2.8.x and master branches in those 2 years either, I think it's safe to say Pootle is not the best tool to use. Conversely, Weblate development is pretty active with a new release every couple of months. I'm confident that if there are performance issues because of memory leaks, etc given the talent pool behind KDE and Weblate we'd collectively be able to work through the issues. :)

Tbh, I just prefer offline translation. If online can prove it's better, why not.

Totally understandable. Being able to still translate offline was a concern that was also raised by another person in this thread. I think it should be possible for people to make updates via their desktop SVN tools and for people to make updates via an online tool like Weblate. Allowing online and offline prevents us from isolating any one group of contributors.

By the way, when new comers want to translate, they find my email (somehow) and just contacts me. I then send a PO file and they translate it. Once they're done, we check it, and I push it. Simple as that, plus learning a translation app (Lokalize, poEdit, Virtaal).

That's great that people are able to find you and still contribute! Alas, not everyone can find you (I didn't when trying to figure out how to contribute Japanese translations to KDE and opted to contribute to FOSS with simpler workflows). Having an easy to use online tool that doesn't require intervention (beyond designated maintainers' approve/deny) would free you up from requests by people so you yourself can spend more time contributing languages like Arabic so KDE can be better used (and also not have a single point of failure by using you as a relay to make SVN changes). :)

@meskobalazs I'm also interested in testing this one, please add me to the list

I was the one who started that "Use Pontoon for l10n" thread in the list. The KDE Malayalam team was dormant for a lot of years. I was interested in revamping it and was looking for a better way to engage people to it. I tried Pontoon, Weblate, Pootle. I've detailed my findings here : https://gitlab.com/snippets/1838918

I only had access to an old Debian Jessy server and got Pootle working. Imported some components of KDE to it and started localizing. I made an intermediary git repo to merge and manage the different branches of localization. The setup is documented in detail here : https://github.com/subins2000/kde-pootle

The KDE Malayalam team Pootle instance is currently active here : https://kde.smc.org.in/ml. Pootle has support for downloading PO and then re-upload. We were able to get new people into localization easily as a web interface was easy to use and the ones who preferred PO would download & upload later. The translation memory across all the components made it further easy to localize in the web interface.

The problem with Weblate was importing took so much time. There's no redis worker support like Pootle in weblate. Maybe if I'm able to get a better server, I can try Weblate or Pontoon. I did chat with Pontoon devs, they're interested in Pontoon becoming a project that can be deployed for non Mozilla projects. The current Pontoon can be modified easily into having a non Mozilla login and just a simple registration/login form. When I experimented Pontoon and tried to import KDE into it, memory overflowed. Pontoon is resource intensive and configuring it to use svn was also a bit complicated IIRC.

meskobalazs updated the task description. (Show Details)Aug 23 2019, 12:48 PM
meskobalazs updated the task description. (Show Details)Aug 23 2019, 12:55 PM
lydia added a comment.Aug 24 2019, 3:39 PM

Vote invite were sent to everyone subscribed to the KDE community mailing list and everyone with a developer account. Any contributor who didn't receive one please subscribe to the mailing list to not miss future announcements and send me a quick email (lydia@kde.org) and I'll send you a vote invite.

guoyunhe added a subscriber: guoyunhe.EditedAug 26 2019, 7:20 PM

Simplified Chinese team moved to Crowdin workflow since 2017. We have seen a boost of contributors in the last two years, from around 5 to a list of 30+.

The scripts we used is here https://github.com/KDE-China/crowdin

Maybe we can use open source self hosted Weblate. Anyway is better than SVN. After 2016, here isn't anyone from Simplified Chinese team getting developer accounts after 2016. People without SVN access has to sent the file to mail list. Most of them give up in the end...

I think many contributors have asked when can we get an online translation system. But the answer is always: no, we won't do it for now.

I submitted one Qt's translation last year. And it takes a whole year to be fixed in Qt 5.14, which we still haven't seen...

guoyunhe updated the task description. (Show Details)Aug 26 2019, 7:23 PM

[spam comment removed by sysadmin]

obogdan updated the task description. (Show Details)Aug 29 2019, 8:22 PM
obogdan added a subscriber: obogdan.
aacid added a comment.Sep 1 2019, 3:56 PM

I think many contributors have asked when can we get an online translation system. But the answer is always: no, we won't do it for now.

That's a lie, that's not the answer.

lydia added a comment.Sep 9 2019, 8:23 AM

Unfortunately this goal wasn't selected as part of this round. If there is a critical mass of people who still want to push it forward please do! If support like financing a sprint is needed please reach out to the board.

Can we get a server to start testing with Weblate? It only takes one or two weeks to set up a test instance of Weblate + scripts for synchronization.

If KDE can't provide a test Weblate server, I can front one on my own dime. A server on Linode that meets the 2 core + 2 GB recommended specs goes for $20/month.

The other thing we'd need to know is who we could contact to connect the test Weblate instance to whatever integration/pre-prod infrastrucutre KDE has to make sure the seamless SVN integration bits all check out. In particular I believe some KDE sysadmin would need to provide some faceless account that would handle the automated pushes/pulls between integration/pre-prod Summit and Weblate.

If KDE can't provide a test Weblate server, I can front one on my own dime. A server on Linode that meets the 2 core + 2 GB recommended specs goes for $20/month.

The other thing we'd need to know is who we could contact to connect the test Weblate instance to whatever integration/pre-prod infrastrucutre KDE has to make sure the seamless SVN integration bits all check out. In particular I believe some KDE sysadmin would need to provide some faceless account that would handle the automated pushes/pulls between integration/pre-prod Summit and Weblate.

I am sure they can since this is a very critical use case.

For testing stage, we can probably start with SVN --> Weblate. This direction doesn't need SVN account. After that, we can first use a developer account to test Weblate --> SVN. When we are ready to take off, ask for a faceless account.

I mentioned it elsewhere, but before doing this, it should be clear how much the changes made to the underlying repository (file rename, file changes which do not pass through the online tool) impact the online tool.
Also, I'd personally invest more time into preparation work which would reduce the amount of changes that may break the strings, like for example:

  • flattening the directory structure removing the modules (so <lang>/messages/<repository>/<po_file> instead of <lang>/messages/<module>/<po_file>
  • a mechanism to detect file renames and magically handles them
  • move everyone to POSummit, so that there is just one branch to take care of;

With less underlying moving part, implementing a web interface would be easier.

flattening directory structure will be a very good idea. since here a lot of file moving from one folder to another. Weblate doesn't support folder structure. But we can also use a script to map current folder structure to a flatten structure.

for file renaming, we can have a list file of all files we uploaded to Weblate. The script will check the list file every time. If some new po files or renamed po files are not in the list. If it is not in the list, we upload the po file to Weblate so we don't lose translations. If some file in the list is not existing on SVN, we request delete it from Weblate, so here will not be duplicated files.

so we can do it without changing current structure on SVN. but flattening it will be a bonous.

ppeter added a subscriber: ppeter.EditedSep 9 2019, 12:08 PM

So I made a RFC for this changes, and I hope it is helpful for migrating to Weblate. Hope it successful! :)

 ! Subject   : Migrate Weblate to SVN [RFC draft]
 ! Date      : Sep 09 2019
 ! RFC       : ppeter (Yi-Jyun Pan)
   Submitter
 ! Approved  : DEPRECATED
 ! Metadata
   $LastUpdate: 2019-09-09 15:07:55 UTC
   Revision 1.

--- Content
Translate flow:

   Translating  ->  Reviewing       -> Submit (-> Git)
   (Anyone [1])     (The specified reviewers)
                    ex. KDE Developer Account Owner
[1]: We should restrict people who can translate to
     ensure the quality if available.

FAQ 1: If a translation team has a translation platform...
 -> (1) Convert to Weblate.
    (2) Tell to project administrators for removing
        their languages in Weblate

FAQ 2: If someone wanna use old way...
 -> It is still available to upload manually to Weblate,
    but if the affected scale is too much,
    you still can upload translations directly to
    the translation repository.
    
    Translators have to talk with Project Administrators before
    uploading to repository, and the Project Administrators MUST TO
    RE-SYNC REPOSITORY IMMEDIATELY after discussed, submitted and
    approved to prevent conflicts.

FAQ 3: What methods to login to Weblate?
I have three plan:
  - Use the native login method provided by Weblate.
  - Use KDE developer account to login.
  - Or even provide the two way to login.
  
FAQ 4: How to let all translators know this change?
So, it is possible to tag all translators in Phabricators,
but we can't really sure that everyone know that (for example,
spam or such thing) , so... it must to have a schedule, 
and let most translators be able to response those changes.

 (+- 15 days)    (+- 3 months)
 / 1 months  \ /   6 months                \ 
|-------------|-----------------------------|---------------->
(1)          (2)                           (3) 

(1) - Migrate some extragear translations to Weblate, still allow people to
      manage translations in SVN, but it will be announced
      that it is being deprecated.

(2) - Migrate most of components to Weblate and disallow people to translate
      some extragear translations that is migrated in (1). Now, it might be
      announced that it is deprecated, you should use Weblate now.

(3) - Now, SVN submit way might be disallowed, people just can
      submit to Weblate.
      
FAQ 5. Where to store the translations?
Git is better, and I think place them in cgit.kde.org is better enough,
if you want to make some big changes like CASE 2, you should to submit to
Phabricators.
ppeter added a comment.EditedSep 9 2019, 12:10 PM

I mentioned it elsewhere, but before doing this, it should be clear how much the changes made to the underlying repository (file rename, file changes which do not pass through the online tool) impact the online tool.
Also, I'd personally invest more time into preparation work which would reduce the amount of changes that may break the strings, like for example:

  • flattening the directory structure removing the modules (so <lang>/messages/<repository>/<po_file> instead of <lang>/messages/<module>/<po_file>
  • a mechanism to detect file renames and magically handles them
  • move everyone to POSummit, so that there is just one branch to take care of;

    With less underlying moving part, implementing a web interface would be easier.

We can make <lang>/messages/<repository>/<po_file>.po as <lang>/messages/<repository>/<module>_<po_file>.po.

If KDE can't provide a test Weblate server, I can front one on my own dime. A server on Linode that meets the 2 core + 2 GB recommended specs goes for $20/month.

The other thing we'd need to know is who we could contact to connect the test Weblate instance to whatever integration/pre-prod infrastrucutre KDE has to make sure the seamless SVN integration bits all check out. In particular I believe some KDE sysadmin would need to provide some faceless account that would handle the automated pushes/pulls between integration/pre-prod Summit and Weblate.

I think they might have resource for it, btw I also have a server for testing (8 core + 32GB RAM, and I expect that Weblate will use 1/16 of RAMs).

I mentioned it elsewhere, but before doing this, it should be clear how much the changes made to the underlying repository (file rename, file changes which do not pass through the online tool) impact the online tool.
Also, I'd personally invest more time into preparation work which would reduce the amount of changes that may break the strings, like for example:

  • flattening the directory structure removing the modules (so <lang>/messages/<repository>/<po_file> instead of <lang>/messages/<module>/<po_file>
  • a mechanism to detect file renames and magically handles them
  • move everyone to POSummit, so that there is just one branch to take care of;

    With less underlying moving part, implementing a web interface would be easier.

We can make <lang>/messages/<repository>/<po_file>.po as <lang>/messages/<repository>/<module>_<po_file>.po.

That would make things more difficult. In fact modules must disappear from there, because otherwise repositories moving among modules will increase the number of moves. We just manage to remove the module name from json and desktop files translations, and having them back won't simply happen.

The idea to have show which module a program belong to from l10n.kde.org (which is a reason why probably need Damned Lies anyway, regardless of whether an online tool is there or not).

ppeter added a comment.EditedSep 9 2019, 12:17 PM

I mentioned it elsewhere, but before doing this, it should be clear how much the changes made to the underlying repository (file rename, file changes which do not pass through the online tool) impact the online tool.
Also, I'd personally invest more time into preparation work which would reduce the amount of changes that may break the strings, like for example:

  • flattening the directory structure removing the modules (so <lang>/messages/<repository>/<po_file> instead of <lang>/messages/<module>/<po_file>
  • a mechanism to detect file renames and magically handles them
  • move everyone to POSummit, so that there is just one branch to take care of;

    With less underlying moving part, implementing a web interface would be easier.

We can make <lang>/messages/<repository>/<po_file>.po as <lang>/messages/<repository>/<module>_<po_file>.po.

That would make things more difficult. In fact modules must disappear from there, because otherwise repositories moving among modules will increase the number of moves. We just manage to remove the module name from json and desktop files translations, and having them back won't simply happen.

The idea to have show which module a program belong to from l10n.kde.org (which is a reason why probably need Damned Lies anyway, regardless of whether an online tool is there or not).

I think, instead of having a new Damned Lies, why not just preserve the old way to translate? I was the Damned Lies user and I didn't have a good experience on it.

ppeter added a comment.EditedSep 9 2019, 12:22 PM

And if it is approved, could I open a JIRA task to Qt for migrating their translation platform to KDE's weblate due to the reason @meskobalazs said? @ltoscano

I am also a Qt translator and I also wanna let Qt localization easier.

And if it is approved, could I open a JIRA task to Qt for migrating their translation platform to KDE's weblate due to the reason @meskobalazs said? @ltoscano

I am also a Qt translator and I also wanna let Qt localization easier.

I'm not sure what has been approved here :)

Moreover, the Qt community is a different story, and the KDE sites and tools are not the proper place to discuss how fix things in the Qt project, which has its own channels for discussion and workflows for decision. Which means that you can try to do anything that you want as any other contributor in the Qt project without asking here. You need to get the approval of the Qt project maintainers of course.

And no, even if we had a KDE online translation tool (whose timing is not yet clear at this point), the Qt project won't and should not use it.

I think, instead of having a new Damned Lies, why not just preserve the old way to translate? I was the Damned Lies user and I didn't have a good experience on it.

We need it as replacement of l10n.kde.org, whoae code is not aging well.
Which kind of user experience was bad for you? I'm only talking about the browsing and grouping of files, not uploading and reviewing (unless some team wants to use it).

aacid added a comment.Sep 9 2019, 1:49 PM
still allow people to manage translations in SVN, but it will be announced that it is being deprecated.

NO we are NOT deprecating SVN in any way, that is not acceptable.

ppeter added a comment.Sep 9 2019, 2:48 PM
still allow people to manage translations in SVN, but it will be announced that it is being deprecated.

NO we are NOT deprecating SVN in any way, that is not acceptable.

No... It means that submit PO file to SVN is unacceptable after migrated to Weblate to prevent the conflict issues, I don't really want to deprecate SVN completely.

still allow people to manage translations in SVN, but it will be announced that it is being deprecated.

NO we are NOT deprecating SVN in any way, that is not acceptable.

No... It means that submit PO file to SVN is unacceptable after migrated to Weblate to prevent the conflict issues, I don't really want to deprecate SVN completely.

It would mean deprecating SVN.

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

ppeter added a comment.Sep 9 2019, 3:06 PM
still allow people to manage translations in SVN, but it will be announced that it is being deprecated.

NO we are NOT deprecating SVN in any way, that is not acceptable.

No... It means that submit PO file to SVN is unacceptable after migrated to Weblate to prevent the conflict issues, I don't really want to deprecate SVN completely.

It would mean deprecating SVN.

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

Oh... then I don't think that use Weblate is the good choice anymore. Thanks for answering.

Do you have an alternative then to the current problem of reducing barrier to contribute @ppeter ? Forgive my ignorance, but it's possible for Weblate and Git PR's to co-exist so I don't see why it wouldn't be possible for Weblate and SVN to co-exist to appease both sides.

SVN submit and Weblate can work together. I have made a workflow with SVN+Crowdin. New translation submitted to SVN will also be uploaded to Crowdin. So both Lokalize and Weblate/Crowdin lovers can still contribute in their preferred way. This is totally possible.

Here is my script for Crowdin workflow, should be similar with Weblate: https://github.com/KDE-China/crowdin/blob/master/upload_translations

Everything is possible if you would give it a try. Everything is impossible if you only make predictions instead of actions...

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

Can you give a more detailed description of what kind of change you need to do with po files? When the context is clear, we can support it in the Weblate workflow.

ppeter added a comment.Sep 9 2019, 3:57 PM
still allow people to manage translations in SVN, but it will be announced that it is being deprecated.

NO we are NOT deprecating SVN in any way, that is not acceptable.

No... It means that submit PO file to SVN is unacceptable after migrated to Weblate to prevent the conflict issues, I don't really want to deprecate SVN completely.

It would mean deprecating SVN.

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

Oh... See https://phabricator.kde.org/T11070#198374 FAQ 2.

SVN submit and Weblate can work together. I have made a workflow with SVN+Crowdin. New translation submitted to SVN will also be uploaded to Crowdin. So both Lokalize and Weblate/Crowdin lovers can still contribute in their preferred way. This is totally possible.

Here is my script for Crowdin workflow, should be similar with Weblate: https://github.com/KDE-China/crowdin/blob/master/upload_translations

How does manage the case when a file is renamed but there are new proposed translations for it? Does crowdin recognize the rename and reassign the pending changes?

still allow people to manage translations in SVN, but it will be announced that it is being deprecated.

NO we are NOT deprecating SVN in any way, that is not acceptable.

No... It means that submit PO file to SVN is unacceptable after migrated to Weblate to prevent the conflict issues, I don't really want to deprecate SVN completely.

It would mean deprecating SVN.

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

Oh... See https://phabricator.kde.org/T11070#198374 FAQ 2.

That does not really address what happens in some cases (see my other answer).

BTW, FAQ 3 ha no choice: we must use Identity.
Same for FAQ4: the official channel where all teams must be (at least their coordinators) is the kde-i18n-doc mailing list.

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

Can you give a more detailed description of what kind of change you need to do with po files? When the context is clear, we can support it in the Weblate workflow.

Any kind of change in both strings (for example, fixing mistakes in tags which prevent the po file - from documentation - from being compiled) and headers (license details, missing tags, etc). And also renaming of files, and other operations that are probably not problematic from this workflow (copy with a different name and removal).

How does manage the case when a file is renamed but there are new proposed translations for it? Does crowdin recognize the rename and reassign the pending changes?

The process is like:

  1. Crowdin will create a new po file in database.
  2. Script will upload rename po file at night, uploading most translations. Changes during the gap (<24 hours), are not copied to new file but still in database.
  3. Usually we just to go the new file, using "Perfect Match" suggestions to fill them. It takes usually just a minute, because it is very unlikely that you translated a thousand strings of a file and the file is renamed in the same day. Even if that is the case, you can still download the previous po file from Crowdin and upload it to new file name, which will not take longer than a minute.

One of the requirement is to allow people to still do changes on SVN. If I, as administrator, need to directly change a po file (for several languages), I must be able to do it, and the online translation tool should cope with that.

Can you give a more detailed description of what kind of change you need to do with po files? When the context is clear, we can support it in the Weblate workflow.

Any kind of change in both strings (for example, fixing mistakes in tags which prevent the po file - from documentation - from being compiled) and headers (license details, missing tags, etc). And also renaming of files, and other operations that are probably not problematic from this workflow (copy with a different name and removal).

In Crowdin, you can have multiple translation history, the latest one is used:

  1. This is a foobar (used)
  2. That foobar

If you changed a translation in SVN "This is NOT A foobar", when uploading translation daily, it will compare to the history, make sure it is not overriding new changes of online translation. Because we didn't found this string in history, it will override:

  1. This is NOT A foobar (used)
  2. This is a foobar
  3. That foobar

If then it is changed on Crowdin again, to fix a type, etc. "This is NOT a foobar":

  1. This is NOT a foobar (used)
  2. This is NOT A foobar
  3. This is a foobar
  4. That foobar

Next time when SVN uploading translations, it can find "This is NOT A foobar" in history, so it is old translation, do not upload and override. New online translations will be downloaded to SVN.

huftis added a subscriber: huftis.Sep 15 2019, 10:32 AM