Defining output-input usage patterns
Open, NormalPublic

Description

We (Dorota from Purism, @mart, me) had some discussions at Plasma Mobile Sprint today about the question on how KWin should react to different input and output device classes. I'm looking for more feedback on our ideas.

Current state of affairs

  • On Plasma Mobile we install a kwinrc config file, which controls KWin's behavior, for example that the window placement strategy should always be "Maximizing".
  • In KWin we have the TabletModeManager class. Its purpose is to inform clients via DBus about the current usage mode of convertibles. Kirigami uses this information for example to change the size of visual elements to allow easier touch control.

Goals

  • Define a concept of determining and controlling the window placement and other window manager internal characteristic depending on current hardware and configuration.
  • Support multiple characteristics in the same session with different hardware devices (convergence).
  • Communicate the selected characteristic to interested clients.

Solution proposal

KWin internal

  • Define in KWin usage patterns describing the preferred way of arranging content and interacting with it. For example: Desktop, Phone, Tablet, TV, VR, ...
  • Set usage pattern per output.
  • Determine which usage pattern should be set on which output by evaluating information about the size of the output, the overall availability of input methods and manual system/user configuration.
  • Change window placement strategy, decoration visibility and other properties accordingly (should still be configurable, but per usage pattern).

Informing clients

  • Define a Wayland protocol to inform interested clients per surface, when this surface changes output. Set enums describing the current usage pattern on the output. Enums could be:
    • Directly named Desktop, Phone,... like the usage patterns
    • Enums named Mouse, Touch, GameController, VRGoggles,... resembling real hardware.
  • Add new names in new protocol versions.
  • Solution 1: Send a single enum
    • If client only supports older version, send the best fitting supported older enum as fallback.
  • Solution 2: Send a priority list of enum values
    • Fallback is integrated directly. Clients should just ignore values in the list if they can't interpret it.
    • By reading out multiple values, clients are able in theory to determine how best react to the usage pattern.
  • Solution 3: Send two enum values, one for input usage pattern and one for output usage pattern.
    • Combinations like Phone + Touch / Phone + Keyboard or VRGoggles + Gloves / VRGoggles + KnuckleControllers are easily possible
    • Might be redundant in certain cases: For example difference between Phone + Touch / Tablet + Touch is negligible client-side (in this example not with CSD).
romangg created this task.Fri, Feb 8, 5:15 PM
romangg triaged this task as Normal priority.
romangg updated the task description. (Show Details)Fri, Feb 8, 5:33 PM

Change window placement strategy, decoration visibility and other properties accordingly

I'd quite like to improve our kwin scripting for initial placement.

Ideally we want it to run between XdgShellSurface being committed and us sending our initial configure.
Kwin scripts would give plasma mobile maximum flexibility, and help with all the kwin tiling scripts, and allow us to slowly clear out the horrible placement.cpp class.

Define a Wayland protocol to inform interested clients per surface, when this surface changes output.

wl_surface already has a protocol to informs clients when their surface changes output. It supports a surface being on N at once.
It should work right now.

From the sounds of it we just want to decorate either the wl_output/xdg_output/kde_output with some extra semantic values. Should be doable.

Define a Wayland protocol to inform interested clients per surface, when this surface changes output.

wl_surface already has a protocol to informs clients when their surface changes output. It supports a surface being on N at once.
It should work right now.

From the sounds of it we just want to decorate either the wl_output/xdg_output/kde_output with some extra semantic values. Should be doable.

Yea, sounds spot on. What do you think how these semantic values should be defined? I.e. solution 1, 2, 3 above or something else?

That doesn't sound quite right.

Change window placement strategy, decoration visibility and other properties accordingly

I'd quite like to improve our kwin scripting for initial placement.

Ideally we want it to run between XdgShellSurface being committed and us sending our initial configure.
Kwin scripts would give plasma mobile maximum flexibility, and help with all the kwin tiling scripts, and allow us to slowly clear out the horrible placement.cpp class.

Define a Wayland protocol to inform interested clients per surface, when this surface changes output.

wl_surface already has a protocol to informs clients when their surface changes output. It supports a surface being on N at once.
It should work right now.

From the sounds of it we just want to decorate either the wl_output/xdg_output/kde_output with some extra semantic values. Should be doable.

That's not quite right, there are many cases when the usage patterns change, but the output doesn't. Some examples:

  • tablet has a keyboard attached,
  • convertible switches modes,
  • a gamer picks up a controller and moves from the keyboard chair to a couch farther away.

I don't think the modes should be strongly attached only to display, there's more going on than that.