Make Input Methods Just Work
Open, Needs TriagePublic

Description

Description

IM setup in Plasma is confusing because users are supposed to hand-pick the right components in the IM stack. Searching for documentation often lands you in outdated tutorials suggesting Fcitx and SCIM (both of which seem unmaintained).

In the case of Japanese, I found by trial and error that the combination of ibus and mozc works decently, and kimtoy could be used to autostart ibus-daemon at login time, but it also replaces the ibus applet with one that didn't work for me. I also discovered kimpanel, which provides yet another UI for switching between input methods.

Adding to the confusion, distros offer custom IM configuration tools that often don't integrate well with Plasma. For instance, Fedora has imsettings.

Proposed changes

  • Provide a merged tray icon for keyboard layouts and current input method. Having both is very confusing, and users don't know the difference between layouts and input methods, they just want to switch between "type in English" or "type in Thai".
  • Remove any mentions of legacy input methods: Fcitx, SCIM, XIM, etc.
  • Work with IBus maintainers to provide a Plasma applet equivalent to ibus-ui-gtk3
  • Deprecate kimtoy, or improve the tray UI and make the configuration UI into a kcm
  • Update the Keyboard Layouts kcm to allow launching the external IM configuration UI (ibus-setup?)
  • Work with distros to ensure all the required packages are either pre-installed or can be installed with one click when needed (via PackageKit / Discover)
  • Maybe: turn on IBus by default for all users, with pre-configured methods such as emoji and unicode input
  • Maybe: integrate IBus Typing Booster (suggests word completions and next words from a dictionary)

Relevant links

How we know we succeeded

Users stop asking how to setup input methods on forums and bug trackers. Tutorials become obsolete because the steps to configure a new IM are obvious and discoverable.

I am interested

I am willing to put work into this

  • Eike Hein (@hein)
  • YOUR NAME HERE

Should we have input methods as a goal or maybe something a bit more broad such as Accessibility?

hein added a comment.EditedTue, Jun 11, 11:03 AM

I submitted the same goal last year:

https://phabricator.kde.org/T6838

I also mentored the GSoC project that implemented a lot of this.

So naturally, my full support. This is incredibly necessary work.

Some the details of the proposal are a bit wrong though, although the brunt of it has the right goals. Here's my talk that lays out the todo:

https://youtu.be/wE0KDabPfsQ

! In T11054#187995, @hein wrote:
Some the details of the proposal are a bit wrong though, although the brunt of it has the right goals. Here's my talk that lays out the todo:

https://youtu.be/wE0KDabPfsQ

As the comment on Youtube points out the slides are not readable. Can you upload them somewhere and link here or in the video, so one can follow your talk better. I would also like to get an overview on current state on X11 in particular.

On Wayland there is the text-input and input-methods protocols. Mostly pushed by Purism at the moment because of their need in Librem 5 smartphones (for virtual keyboard). If these protocols lost focus in regards to traditional input-methods for internationalization and accessibility is yet to be seen when they are finally implemented somewhere or by a thorough inspection. For us there is a patch of the text-input protocol D16735, but without the input-methods protocol it does not make much sense to merge it yet.

GNOME uses some internal stuff, wlroots on the other side has implemented both protocols. There is also the question how this can be integrated with XWayland: https://gitlab.freedesktop.org/xorg/xserver/issues/820

In general I propose to concentrate on working on a user-friendly solution primarily on Wayland and only if possible try to rescue concepts over to X11.

Should we have input methods as a goal or maybe something a bit more broad such as Accessibility?

IMO Input methods could be part of accessibility. Think about Voice- , Braille-, OSK and similar. This could include input for different languages as well.

hein added a comment.Tue, Jun 11, 2:35 PM

As the comment on Youtube points out the slides are not readable. Can you upload them somewhere and link here or in the video, so one can follow your talk better. I would also like to get an overview on current state on X11 in particular.

Talk slides: www.eikehein.com/kde/Input_Methods_in_Plasma_5.pdf

I've also contributed a bit to the Wayland protocols by working with dorota and the Sway people.

In T11054#187995, @hein wrote:

I submitted the same goal last year:

https://phabricator.kde.org/T6838

Your proposal was framed and justified so much better than mine!

Feel free to edit this task and merge your ideas into it (do I need to grant you access or can anyone edit? This is my first time using Phabricator).

hein updated the task description. (Show Details)Wed, Jun 12, 9:25 AM

I would object a stronger integration with IBus (or any of the single input method). Even within the "common" CJK input method framework, Chinese/Japanese and Korean input methods have different requirements, which is a common source of frustration why Korean IM is sometimes not working out-of-the-box.

  • Chinese/Japanese input requires dictionaries to translate phonetic input into Hanzi/Kana, while Korean input is not heavily depending on the usage of dictionary for Hangul input (only Hanja requires it)
  • Chinese/Japanese input uses "explicit" commit of preedit text and when user moves focus away from a control it is okay to remove the preedit text, while Korean input uses "implicit" commit of preedit text and user expects automatic commit of preedit text when user moves focus away. This behavior is expected from Korean users from the beginning of the computer era, and it is not changeable. Period.
  • Chinese/Japanese IM includes several extra features regarding text input, while Korean IM doesn't have eyecandy features and users don't expect that much from an IM.

IBus once had (and maybe still?) a notorious bug for Korean users (and bad image as such) so called "trailing character bug"/"끝글자 버그", where preedit text is not committed properly and follows to another text area even after changing the focus. The asynchronous event loop design of IBus made the Korean input problem complicated, and it was counted as one of the motivation of developing another IM framework Nimf [1] (although I haven't personally evaluated it). While the IBus bug was said to be addressed, since my first task after installation is always removing IBus and installing something else to avoid that annoying bug, I couldn't confirm whether the bug is persistent yet. Some applications are also known to having such a bug, when it modifies preedit text handling behavior, not handling preedit text properly or interferes the events generated by IM (premature IM reset or such). This needs to be addressed by stable set of requirements and expected behaviors.

Also we need to make sure that common shortcuts used by applications should not collide with the IM's global shortcuts such as Ctrl+Space or Shift+Space. If that happens, users should be informed about the possibility and change either the app or the IM. This also applies to xkb options: when both Plasma and IM wants to access/modify xkb settings it should not collide againast each other.

Finally, cross-distro packages (AppImage, Snap, Flatpak) is another source of problem. Due to sandboxed design and limited integration with what is installed in the system, IMs are not available to the applications installed by those packages. This definitely is another source of user's confusion.

[1] https://gitlab.com/nimf-i18n/nimf/

pshinjo updated the task description. (Show Details)Thu, Jun 13, 12:02 PM
ngraham updated the task description. (Show Details)Thu, Jun 13, 1:40 PM

I would object a stronger integration with IBus (or any of the single input method). Even within the "common" CJK input method framework, Chinese/Japanese and Korean input methods have different requirements, which is a common source of frustration why Korean IM is sometimes not working out-of-the-box.

How do Gnome users cope with these issues?

Development appears to have stalled in older IM frameworks, and thus they don't have to don't support Wayland and desktop portals. Keeping them working for a few more years might be feasible, but we might not want to focus our future efforts on them.

pshinjo added a comment.EditedFri, Jun 14, 11:45 AM

I would object a stronger integration with IBus (or any of the single input method). Even within the "common" CJK input method framework, Chinese/Japanese and Korean input methods have different requirements, which is a common source of frustration why Korean IM is sometimes not working out-of-the-box.

How do Gnome users cope with these issues?

Development appears to have stalled in older IM frameworks, and thus they don't have to don't support Wayland and desktop portals. Keeping them working for a few more years might be feasible, but we might not want to focus our future efforts on them.

Some tried to fix how Mutter on Wayland behaves [4], some avoid the bug by not using IBus at all, and switch to another IM framework. IBus and Korean didn't worked well for a long time and users have reported issues dating back from 10 years ago [1]. Even earlier Qt 5 IBus module had critical bug where all the spacing of words are garbled [2]. It was also pointed out that addressing the bugs within the asynchronous architecture of IBus is not possible [3 - please look inside the comments]. The root cause of these bugs sometimes could be traced to IMs not properly handling certain user's input habits (when they change focus? did they try to explicitly commit the preedit text by pressing non-character keys? etc.), but as everybody knows, these basics are impossible to change, and more importantly, if the IM's behavior changes between versions when user tries to do the same, they will try to change IM instead.

This is also the reason why certain users want to avoid Wayland at all, as their favorite IM framework doesn't support Wayland and/or IM framework available in the Wayland is not adequate for their workflow.

[1] https://github.com/ibus/ibus/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aclosed++korean
[2] https://github.com/ibus/ibus/issues/1742
[3] https://github.com/ibus/ibus/issues/1980
[4] https://gitlab.gnome.org/GNOME/mutter/issues/152

hein added a comment.EditedFri, Jun 14, 1:07 PM

The work Geon and I did in GSoC had an fcitc5 backend with plans to add a second one.

I'll say firmly though that I don't like the "let's not solve our integration problems because we need to support all the things" position. If ibus has a bug, we should try to get the bug in ibus fixed. One of the problems with the Input Method community is this pattern of "there's a minor thing about XYZ that we don't like, so instead of working together we'll do our own thing". That way nothing ever gets great in the end! There's not enough manpower in this area to spread it so thinly.

In T11054#188801, @hein wrote:

The work Geon and I did in GSoC had an fcitc5 backend with plans to add a second one.

I'll say firmly though that I don't like the "let's not solve our integration problems because we need to support all the things" position. If ibus has a bug, we should try to get the bug in ibus fixed. One of the problems with the Input Method community is this pattern of "there's a minor thing about XYZ that we don't like, so instead of working together we'll do our own thing". That way nothing ever gets great in the end! There's not enough manpower in this area to spread it so thinly.

I do agree on "we need to solve integration problem" and having a reasonable default implementation. I might be biased against IBus due to its problematic history and people calling out "IBus is buggy". What I don't agree is making IBus or any other framework as a strong dependency so users can't change their IM framework at all. I interpreted the proposed changes as eliminating this possibility.

lydia updated the task description. (Show Details)Sat, Jun 15, 7:10 AM
lydia added a subscriber: lydia.Sat, Jun 15, 7:26 AM

I think it'd be a good idea to shamelessly borrow content from Eike's 2017 goal proposal and integrate it here ;-)

In T11054#188801, @hein wrote:

"there's a minor thing about XYZ that we don't like, so instead of working together we'll do our own thing". That way nothing ever gets great in the end! There's not enough manpower in this area to spread it so thinly.

+ a bazillion

I would even go so far as to say this is a general problem in the FOSS world: not enough cooperation for tools that aim to solve the same problem in roughly the same way. When the available development resources are limited, all this does is produce half a dozen buggy incomplete products that are all very similar to one another, rather than one or two really good ones. It's fine (and great) to have multiple products that aim to solve a problem in fundamentally different ways or for different target users, but otherwise It's better to cooperate than fork or work separately IMO.

I think it'd be a good idea to shamelessly borrow content from Eike's 2017 goal proposal and integrate it here ;-)

Yes, let's! @hein