Create Task
Maniphest T12863

Use PipeWire for screen casting
Open, Needs TriagePublic
Actions

Description

Currently, we use a proprietary protocol for sharing GBM buffers, which are then imported in xdg-desktop-portal-kde. The main problem with the current approach is that it's not suitable for capturing individual windows and we have to copy buffers in xdg-desktop-portal-kde.

A better way to implement screen casting would be to introduce an API which xdg-desktop-portal-kde could use to request a PipeWire stream for the particular screen or window. Once, KWin is a PipeWire server, we can drop the GBM thing.

PipeWire allows buffers to be shared using DMA-Buf and memfd. memfd can be especially useful for NVIDIA users. Obviously, this would mean higher CPU usage, but it's better than nothing.

Protocol-wise, the API for requesting PipeWire streams can be implemented using D-Bus. For reference, here's Mutter's D-Bus API https://gitlab.gnome.org/GNOME/mutter/-/blob/master/src/org.gnome.Mutter.ScreenCast.xml

Related Objects

zzag created this task.Mar 26 2020, 12:01 PM
zzag updated the task description. (Show Details)Apr 2 2020, 1:50 PM
apol added a subscriber: apol.Apr 2 2020, 2:10 PM

Obviously, this would mean higher CPU usage, but it's better than nothing.

Higher CPU usage than what? no casting?

cblack added a subscriber: cblack.Apr 2 2020, 2:28 PM
zzag added a comment.Apr 14 2020, 8:35 PM
In T12863#225186, @apol wrote:

Obviously, this would mean higher CPU usage, but it's better than nothing.

Higher CPU usage than what? no casting?

Yes, I wanted to emphasize that having high CPU usage is better than no screen casting at all.

davidedmundson added a subscriber: davidedmundson.Apr 15 2020, 8:10 AM

Let me add in some background context.

The original pipewire export is out of process purely because MF wanted to avoid the blit of the buffer to not slow down compositing. Perfectly sensible.

Now that pipewire can handle DMA-Buf handles we no longer need to blit, which means that reason goes away.

Vlad's proposal is that we can effectively copy/paste the xdg-desktop-portal code into kwin and take the handles directly.

This will simplify code and be faster.

apol added a comment.May 25 2020, 1:05 PM

The biggest problem with sharing the dmabuf buffers it that its memory management becomes quite complex (and I'm unsure it's really doable).
PipeWire has mechanisms to create the buffers it's going to need, juggling this with passing the buffer from the app/output and making sure it stays relevant feels messy and error-prone.
Copying from dmabuf->dmabuf shouldn't be very expensive though, as it shouldn't go through the buses (AFAIK, that is).

Here's a compendium of all patches I've been working on to that end:
https://invent.kde.org/libraries/plasma-wayland-protocols/-/merge_requests/1
https://invent.kde.org/plasma/kwayland-server/-/merge_requests/2
https://invent.kde.org/plasma/kwin/-/merge_requests/10
https://invent.kde.org/plasma/xdg-desktop-portal-kde/-/merge_requests/1
https://invent.kde.org/frameworks/kwayland/-/merge_requests/1

davidedmundson moved this task from Backlog to Done on the KWin board.Aug 10 2020, 5:56 PM