Was there any coordination with @graesslin? I'd love to hear his opinion on this :)

I think this is one of the most important tasks for screen reader users. I assume we'll have to contact the KWin developers. Ideally after doing a bit of research how other window managers solve this.

In T7034#164237, @gladhorn wrote:

I think this is one of the most important tasks for screen reader users. I assume we'll have to contact the KWin developers. Ideally after doing a bit of research how other window managers solve this.

Indeed this is very important. without beeing able to switch an task an desktop is useless.

The common way is to use Alt-Tab to switch windows. Currently Orca does not react to alt-tab while alt is held, but once a new window gains focus, it is read.
So for me this would mostly about enabling the reading of window titles while alt is pressed.
My understanding is that there is luckily a centralized component in KWin which deals with this. kwin/tabbox seems like the relevant files.

Now the question is if we get away with just sending a few at-spi events or if more work is needed.

Next to the window switching, there is also desktop switching (virtual desktops) which I consider secondary (but it may be fixed with the same change).

My current thinking is:
KWin shows up as accessible application with no children.
Create an accessible window inside that application. This window has no relation to any other object but will probably make things less confusing for screen readers. It can also become the active window for screen reader purposes when interacting with KWin.

When the tab switcher is active, populate that window with a list of children, send focus events for them according to which one is currently active.

Howdy,

I asked about this to Joanie (the orca maintaier). she was last week on an trip.
https://mail.gnome.org/archives/orca-list/2018-October/msg00270.html
lets wait for her answer.

cheers chrys

In T7034#165364, @gladhorn wrote:

Next to the window switching, there is also desktop switching (virtual desktops) which I consider secondary (but it may be fixed with the same change).

Agree

In T7034#165373, @gladhorn wrote:

My current thinking is:
KWin shows up as accessible application with no children.
Create an accessible window inside that application. This window has no relation to any other object but will probably make things less confusing for screen readers. It can also become the active window for screen reader purposes when interacting with KWin.

When the tab switcher is active, populate that window with a list of children, send focus events for them according to which one is currently active.

Your current thinking sounds sane to me. :) Hopefully it will JustWork(tm). If it doesn't, let me know and I'll see if I either need to add something in Orca or if something is missing from your initial implementation.

And thanks for doing this!!

It can also become the active window for screen reader purposes when interacting with KWin.

When I looked there wasn't a nice way to do this with the Qt API - and where I got gave up.

If you can make a window for screen reader purposes without changing the stacking order we don't need a secondary window we can do it straight on the existing task switcher.

My only concern is that if I implement a (virtual) window for accessibility only for the task switcher, Orca would probably read the window before reading the focus inside it. But that may be fine.
"task switcher" -> "firefox window". I'll try writing it in the near future and let others play with it. In the worst case we can then use Orca scripts, if people find announcing the tab switcher window to be a nuisance.

Deactivating the clients when the window switcher is active (and on the other side let the window switcher take focus) was discussed here: D13758

In short: this should be no problem on Wayland, but might lead to regressions / "side effects" on X/Xwayland. Making it easier for accessibility would be another argument to look into it again more closely.

My plan was not to change anything regarding x11/wayland at all. It can be done purely on the accessibility level - faking it completely.

In T7034#165782, @gladhorn wrote:

My plan was not to change anything regarding x11/wayland at all. It can be done purely on the accessibility level - faking it completely.

this sounds like an good idea.

I now have a very hackish proof of concept. It shows KWin as application on the accessibility dbus session. It has one child (a virtual main window) with children for the "clients" as KWin calls the windows. There are no updates sent yet, so Orca will not read anything. In addition I'm leaking memory, so this needs a day of cleanup. But I think in principle it will work.

In T7034#165782, @gladhorn wrote:

My plan was not to change anything regarding x11/wayland at all. It can be done purely on the accessibility level - faking it completely.

What I meant is that if we can do it in a generic way without faking, it would be probably better. But let's see how the faking solution works out.

Yes, I'd love it if we do give KWin focus - it would make things just work for accessibility, so in that sense it's the right solution. I just had the feeling that it would be close to impossible to convince the KWin maintainers to go that way.
I already have a proof of concept that informs of the Windows that are open - I just need to figure out how to best send updates and do a proper model for memory management of the stuff I wrote, currently I wouldn't be surprised if it crashes and leaks, I just wanted to see that it's doable at all...

I have been a fool :) I just found that KWin actually does create accessible interfaces for the window switcher since it uses QML and we added accessibility information in some form... What is missing is only pretending that the task switcher window is the active window and maybe sending focus events. That should make this task quite a bit smaller (and I get to throw away my proof of concept code and start over).

OK, I seem to have reached something surprisingly simple: D16638 and D16664 give me a tab box that works with Orca.
The requestActivate is needed to get the focus into the window, the window manager hint has to go (are there side-effects?) and after that pressing alt-tab announces a window containing labels for all other windows. The role should be adjusted, but compared to everything else that should be rather trivial.

In T7034#166345, @gladhorn wrote:

OK, I seem to have reached something surprisingly simple: D16638 and D16664 give me a tab box that works with Orca.
The requestActivate is needed to get the focus into the window, the window manager hint has to go (are there side-effects?) and after that pressing alt-tab announces a window containing labels for all other windows. The role should be adjusted, but compared to everything else that should be rather trivial.

so you have an working Task switcher then? woee awsome!

Yes, it looks like this will work.

Is this finished now?

Yes, I hope so :) it would be great to have anyone else report success, but I think it's done.