Translation of jekyll or hugo based websites
Open, Needs TriagePublic

Description

I experimented with the po4a tool to extract the translation of an Markdown/HTML based websites automaticaly. https://jibecfed.fedorapeople.org/blog-hugo/en/2018/11/translation-automation-with-po4a/

This works quite well. ๐Ÿ˜„

I have multiple questions for the translation team:

  • It is possible to get the translation at compile time (in binary factory)?
  • It is better to have one pot file per page or one pot file per website?
  • Where can I find information about how scripty work?
ognarb created this task.Sep 19 2019, 3:11 PM

@ltoscano I created a merge request for translating the content of kate-editor.org https://invent.kde.org/websites/kate-editor-org/merge_requests/6

python3 translations.py extract extracts the string and create kate-editor-org.pot
python3 translations.py generate-translations expects translations/kate-editor-org.$LANG.po and create *.md/*.html files

This only translate the content, not the menu elements, footer, ...

Ok so I suppose the po files need to be located at locale/$lang/LC_MESSAGES, do I need to make a bash script instead?

No, I think that's the structure required by sphinx. Make sure that import_po_dirs returns the structure that you need.
Please note that the use case of docs-krita-org involves multiple pot files. If you just need one (but again, it depends on the size), this is a simpler case:
https://cgit.kde.org/plasma-browser-integration.git/tree/StaticMessages.sh

ognarb added a comment.EditedSep 19 2019, 5:24 PM

Now I see that you mean, I added an import function. python3 translations.py import directory/with/po/files

If it helps, do all the experiments you want on kate-editor.org.
The web-server will re-pull the git all 5 minutes after some change.

@ltoscano I merged the change in the kate-editor.org repository. The https://kate-editor.org/about-kate/ page should now be translatable. Can the Localization team do the necessary scripty change?

Extract: python3 translations.py extract path/to/pot/files
Import: python3 translations.py import path/to/po/files
Generate translated markdown: python3 translations.py generate-translations

As I mentioned, you need to create a StaticMessages.sh file in your repository, which expose the two twin functions (export_pot_dir/import_pot_dirs if you deal with multiple pot files or export_pot_file/import_pot_files if you have just one), following the example of websites/docs-krita-org, plasma-browser-integration or other projects (kdeconnect-android, for example).
The only thing I can do on the translation side is to enable the extraction for that repository, but you need to write that script.

@ltoscano I added the StaticMessages.sh script. It should now works ;)

Translations with Hugo and po4a works ๐ŸŽ‰ See https://kate-editor.org/about-kate.uk/ or https://kate-editor.org/about-kate.pl/

I still have a problem: translating the front matter https://gohugo.io/content-management/front-matter/ is something not as easy to translate as I think without understanding yaml and knowing the context.

Here nothing needs to be translated since this is referencing a template:

---
layout: index
---

Here only the title content needs to be translated and not the keys

---
title: Features
author: Christoph Cullmann
date: 2010-07-09T08:40:19+00:00
---

to

---
title: Features in the translated language
author: Christoph Cullmann
date: 2010-07-09T08:40:19+00:00
---

One possibility would be to try to extract those strings separately, with a manual extraction and create a list of keys that don't need to be translated. In the meantime can someone fix: https://websvn.kde.org/trunk/l10n-kf5/nl/messages/www/kate-editor-org.po?view=markup?

ognarb added a comment.Oct 7 2019, 2:27 PM

I added a language picker in kate-editor.org and I have bad news, hugo doesn't support partial translated website. Either the entire website is translated or it's not and a lot of dead link are created, because Hugo doesn't provide any fallback to the default language.

This has as effect that in the homepage kate-editor.org, if another languages than English is selected we don't see the blog posts and all the link in the menu are dead.

And Jekyll doesn't provide an API to create a language picker :(

So my next bet will be to see if I can create a sphinx theme what doesn't look like a documentation website.

Ok I was able to work around some of the limitations of Hugo. I added a redirection in javascript, the user experience is not very nice since we need to wait at least 5 seconds before being redirected. For example https://kate-editor.org/uk/build-it/

I don't think I can accelerate the redirection much more. :( So maybe making the error message more helpful to the user could be a solution.

The problem with the blog posts not displayed on the translated page. I could try creating a Hugo paginator from scratch, but I'm not sure it's possible.

Too bad that partially translated stuff doesn't work :/
I only read a bit in the above thread on discourse: is there actually a real bug report that requests support for this?

Hmm, beside that, can one not simply fix that by using a 404 handler that does the redirect? I did that once for the AbsInt page, just rewriting the url to the generic english page if the original url ended up in 404.

@cullmann thanks for the hint, I come up with this 404 handler written using mod_rewrite.

<Directory "/srv/www/kate-editor-org">
        RewriteEngine On
        RewriteBase /
        # Don't redirect if file, symlink or directoy exist
        RewriteCond %{REQUEST_FILENAME} -s [OR]
        RewriteCond %{REQUEST_FILENAME} -l [OR]
        RewriteCond %{REQUEST_FILENAME} -d
        RewriteRule ^.*$ - [NC,L]

        # Redirect /lg/xxxxxx -> /xxxxxx
        RewriteRule "^[a-z]{2}/(.*)$" "$1" [L,R=301]

This was tested with Apache2.4 on OpenSuse Tumbleweed.

kate-editor.org should have the module enabled and you should be able to just add some .htaccess with such rules to the hugo static files dir.

Christoph is correct here, our systems permit the use of .htaccess files, so you should be able to use the Apache Rewrite module without anything being needed on our side
(Note that RewriteBase will probably not be required)

Hugo already generate a .htaccess in public, so this configuration need to be added in the apache conf :(

Is there no way of getting Hugo to append given content to the .htaccess it already generates?

Ok, the .htaccess was actually not generated by Hugo but located in the static directory. I pushed to the server let's see if its work. ;)

ognarb removed a subscriber: Sysadmin.Oct 13 2019, 7:38 PM
ognarb added a comment.Jan 9 2020, 9:45 PM

The tooling improved a bit for https://kde.org/announcements/releases. The next step would be to create a python package with the script so that we don't need to duplicate the code for each repo.

ltoscano moved this task from Backlog to In progress on the Localization board.Jun 22 2020, 5:08 PM
yaron added a subscriber: yaron.Nov 15 2020, 8:40 AM