Port Parley away from Kross
Open, Needs TriagePublic

Description

Parley uses Kross for its plugin interface, most importantly for its Wiktionary Python plugin.

Related Objects

StatusAssignedTask
OpenNone
Opencordlandwehr
alex added a subscriber: alex.Aug 19 2021, 4:50 PM

@cordlandwehr Does the Wiktionary even work for you? I only get errors while trying them out.

And I can't figure out how the automatic translations from leo and google are supposed to work. But TBH I find it also questionable that API requests are made without asking the user first.

@alex IIRC all those plugins have to be enabled by the user before doing any automatic downloading. But there are a few separate extensions:

  • script for moving phrases from one lesson to another (trivial script, could easily be implemented in C++ IMO)
  • script for downloading audio files for words from Wiktionary (uses mwclient python library and is probably hard to reimplement)
  • script for translating words/phrases via Leo (looks like a simple to rewrite script)
  • script for translating words/phrases via Google Translate (looks like a simple to rewrite script)

Yet, I have the same experience and did not yet succeed to make them work for me, which is quite sad, because is released code :/
However, I think that all of them are reasonable features but it will need somebody who fixes/rewrite them without Kross. At the moment, my suggestion would be to disable them at least in master because they do not work. If they then do not get ported to anything until the first KF6 based Parley release, we have to remove them.

alex added a comment.EditedAug 22 2021, 10:23 AM

script for moving phrases from one lesson to another (trivial script, could easily be implemented in C++ IMO)

Though that is an example script. But one could argue that it is the most prominent of scripts that users might consider useful.

However, I think that all of them are reasonable features but it will need somebody who fixes/rewrite them without Kross. At the moment, my suggestion would be to disable them at least in master because they do not work. If they then do not get ported to anything until the first KF6 based Parley release, we have to remove them.

Quite honestly I get the feeling that there are lots of other code smells/issues/overdue maintenance stuff in parley. I don't think that would even be ones first priority.

However disabling them would be almost the same as removing, with the exception that we carry on the dependency stuff. It is not like they are lost for ever if we remove them - there is still the git history. Or one could even keep thy python scripts around as a reference for the google translate/leo webscraping.

IIRC all those plugins have to be enabled by the user before doing any automatic downloading. But there are a few separate extensions:

But when one enters vocabulary the requests are made in the background, there is no explicit button or context menu entry for getting the translations.
Edit: The desktop wiles state that they should be enabled by default, X-KDE-PluginInfo-EnabledByDefault=true

See https://invent.kde.org/education/parley/-/merge_requests/13, not sure how I feel about the other part of the example script. IMHO it does not really fit in the editor context.

cc @aacid @hriesenbeck I guess removing a feature will always be controversial in KDE. However the scripting is for large parts just defunct and adds only more technical debt.

I checked with a distro package of Parley and must change my previous claim slightly, it seems that for me only the audio file download from Wiktionary is completely broken for me, the others seem to work there; I expect that I have something wrongly built with my Kross interpreters...
However, I think the question is, which of these scripts shall be ported to real C++ implementations? IMO the whole scripting engine of Parley never turned out to be a widely used feature and I suggest to keep complexity low by just rewriting the really useful bits in C++, which will then ultimately make the Kross dependency obsolete.
For the translation scripts I think it is easy to do, just by judging of the complexity of the scripts. For the audio download I just noticed that we actually have a C++ based implementation for accessing mediawiki api (https://invent.kde.org/libraries/libmediawiki). Yet, I did not check if this meets all needs. But it means, there could be a way also to replace the mwclient Python library.
Needless to say that I fully agree with your assessment of the technical debts in the code base :/ Just by testing the scripts I hit about 3 different crashes :( Will look if I can at least fix them in the release branch and also will try to spend some time in the most pressing codebase modernize issues. But no promises ;) Yet, that is probably outside if this issue.

alex added a comment.Aug 22 2021, 5:33 PM

Did you get the automatic translation using leo/google working?

For the audio download I just noticed that we actually have a C++ based implementation for accessing mediawiki api

It is marked as porting aid in https://invent.kde.org/libraries/libmediawiki/-/blob/master/metainfo.yaml#L7

alex added a comment.Aug 22 2021, 6:32 PM

Yet, that is probably outside if this issue.

Though the scripting part makes up a few thousand lines, which is directly related :D

Yeah, looking forward to remove all that lines of deprecated code :) If you are not already working on it, I can take over the rewrite of the translator scripts into a more user friendly and also Kross-less solution.

alex added a comment.Aug 29 2021, 10:00 AM

I was mostly asking for you to response if you got them even to work properly :)

Especially because the google API endpoint seems pretty dead to me: Just try opening https://translate.google.com/translate_dict?q=completely&langpair=English%7CDanish

Also the leo stuff did not work for me, even when hardcoding one of the example URLs (scraping the website using curl still works)

@alex I know, but then looked into the code and was shocked how much too complicated the implementation looks for this mostly simple feature.
But the the topic, I confirm that I also cannot make any of the translation APIs work, they all look dead. For google there seems to be the V2 API (https://cloud.google.com/translate/docs/reference/rest/v2/translate), yet that requires an API key. For Leo I could not find any recent API information. And probably it can simply replaced with DeepL, but there we also need an API key.
Yet, I think that the translation support is a really cool feature when integrated properly for creating vocabulary sets. So I think it's worth the time to see if it can be reenabled with more recent APIs.

alex added a comment.Aug 29 2021, 10:34 AM

Are you aware of translate-shell? That would be an entirely different approach, but at least in the KRunner plugin that uses it (https://github.com/naraesk/krunner-translator) the results seem pretty good.

IMHO implementing this would be the way to go, then we have it as an optional runtime dependency and don't need to do webscrapting ourselves.
If we were to do that the Kross stuff would become immediately obsolete :)

I like this :) Is it possible to access exactly this krunner plugin via dbus or would it rather be the way to load the plugin the same way krunner is doing it?

alex added a comment.EditedAug 29 2021, 11:59 AM

It would just be calling the command with the QProcess API.

Just try out :

LC_ALL=C trans -t de programming -show-alternatives=n -show-original=n -show-original-phonetics=n -show-languages=n -show-original-dictionary=n -show-dictionary=y

Then the first line will be the direct translation and the dictionary contains some alternatives. From looking at the parley it seems like the scripts are supposed to support multiple suggestions. Then we could do this too

https://invent.kde.org/education/parley/-/commits/work/translateshell_is_awesome Is a draft, mostly messing around with the arguments & such. But it should be a decent proof of concept.

script for downloading audio files for words from Wiktionary (uses mwclient python library and is probably hard to reimplement)

Translateshell can also speak the stuff or download the audio. See also https://github.com/soimort/translate-shell/issues/402 for the Wiktionary related stuff

alex added a comment.Aug 29 2021, 12:00 PM

Also the scripts are currently executed synchronous, which is a really bad idea considering we are doing web requests :D

You know, you rock :D thanks for the PoC snippet. I already started some refactoring to get the translation part into a unit-testible self-contained async class. The idea is to create an adapter class to translatorshell with a decent to use API.

cordlandwehr moved this task from Backlog to In Progress on the KF6 board.Aug 30 2021, 3:43 PM