[RFC] Half pixel correction
AbandonedPublic

Authored by zzag on Jun 6 2018, 4:44 PM.

Details

Reviewers
davidedmundson
fredrik
graesslin
Group Reviewers
KWin
Summary

Borders of decoration are stored in a texture atlas with 1 pixel gap.
Depending on some circumstances, texture bleeding could happen, e.g.
when zooming screen(see 360549), or when scaling windows.

This diff addresses the problem above by adding the half pixel
correction. The biggest drawback of the half pixel correction is that
it doesn't work in the case when textures have mipmaps. AFAIK,
the decoration texture atlas and window pixmap do not have mipmaps.

Half pixel correction happens only for decoration and content window
quads. If shadow quads bleed, that's fine, same with the cross fade
quads.

BUG: 360549

Test Plan
  • Zoomed screen
  • No seams

Diff Detail

Repository
R108 KWin
Branch
half-pixel-correction
Lint
No Linters Available
Unit
No Unit Test Coverage
zzag created this revision.Jun 6 2018, 4:44 PM
Restricted Application added a project: KWin. · View Herald TranscriptJun 6 2018, 4:44 PM
Restricted Application added a subscriber: kwin. · View Herald Transcript
zzag requested review of this revision.Jun 6 2018, 4:44 PM
fvogt added a subscriber: fvogt.Jun 6 2018, 4:49 PM

Maybe you could add an explanation of what the HPC actually does and why it only affects OpenGL?

scene.cpp
901

I'd call this pixelCenter instead.

zzag added a comment.Jun 7 2018, 4:24 PM

Maybe you could add an explanation of what the HPC actually does and why it only affects OpenGL?

Someone can put many small subtextures into one big texture so called a texture atlas. By doing that way, he/she/it can avoid calling multiple times glBindTexture. But there is a problem when using linear filter. If we scale the atlas, neighboring texels will be sampled. Sampling neighbors near edges of the subtextures can cause so known texture bleeding.
One way to fix the texture bleeding is by using an array texture. I think that's not possible to use array textures in KWin right now.
Another ugly solution is the half pixel correction. With that, we add/subtract "0.5"(half texel size) to/from texture coordinates to snap them to pixel centers. By doing that, only desired pixels will be sampled.
Yet another solution is to pad the subtextures. Depending on what decoration borders are (are they solid? are they gradients?), it may produce not quite the best results.

It affects only OpenGL because that's specific only to OpenGL, I believe we don't need HPC for XRender.


I might be wrong so I will be glad if someone correct me.

zzag added a comment.EditedJun 7 2018, 4:33 PM

Just for comparison.

Before

After

Array textures wouldn't help in this case, because all layers must have the same size.

scene.cpp
901

This check shouldn't be necessary. The Xrender backend doesn't use a texture atlas, so it can't use the quads generated by this function.

943

AFAICT this function is only used to generate content quads, so I don't think we want to do this correction here.

fvogt added a comment.Jun 8 2018, 9:44 AM

@zzag: I meant putting it into the description - I already know what this is about :D

It seems to me like this breaks rendering of some applications sometimes - sometimes yakuake appears blurry on unhide and every second redraw after that is also blurry.

zzag added a comment.Jun 8 2018, 10:39 AM

Array textures wouldn't help in this case, because all layers must have the same size.

Agree, in this particular case, it wouldn't help.

@zzag: I meant putting it into the description - I already know what this is about :D

Oh, sorry. :D

It seems to me like this breaks rendering of some applications sometimes - sometimes yakuake appears blurry on unhide and every second redraw after that is also blurry.

Yeah, I've noticed similar bug, but only when resizing system settings to a minimum possible size (with mouse). I'll dig what's causing that.

scene.cpp
943

The problem is that content doesn't fit window pixmap edge to edge. E.g. for Breeze, u0=4 and v0=29, so unless we do correction here, there will be bleeding.

zzag added a comment.EditedJun 8 2018, 10:44 AM
In D13382#275726, @zzag wrote:

Yeah, I've noticed similar bug, but only when resizing system settings to a minimum possible size (with mouse). I'll dig what's causing that.

It seems like it's caused by the Resize Window effect.

Update

The resize window effect sets PAINT_WINDOW_TRANSFORMED. If PAINT_WINDOW_TRANSFORMED is set, linear filter will be used.

davidedmundson added inline comments.Jun 8 2018, 12:33 PM
scene.cpp
963

I would have thought this would be:

(r.x + offset.x) * scale + halfPixel

as the size of the filter isn't affected by the scale of the source window.

zzag updated this revision to Diff 35825.Jun 8 2018, 12:34 PM

Don't scale half pixels

zzag added a comment.Jun 8 2018, 1:34 PM

Furhter considerations

Move half pixel correction stuff to scene_opengl.cpp, e.g.

WindowQuad withHalfPixelCorrection(const WindowQuad &quad)
{
    WindowQuad newQuad(quad.type());
    if (quad.uvAxisSwapped()) {
        newQuad[0] = WindowVertex(...);
        // ....
    } else {
        // ...
    }
    return newQuad;
}

and use it when filling leaves, e.g.

void SceneOpenGL2Window::performPaint(int mask, QRegion region, WindowPaintData data)
{
    // ...
    foreach (const WindowQuad &quad, data.quads) {
        switch (quad.type()) {
        case WindowQuadDecoration:
            quads[DecorationLeaf].append(withHalfPixelCorrection(quad));
            break;
        // ...
        }
    }
    // ...
}

And at this point, I'm out of ideas. Because the problem with the linear filter is still there (content/decoration quads are blurry when they don't have to be). The most straightforward way to fix the problem is to adjust vertices, but that's unacceptable solution.

Any other ideas how to fix the seams?

fvogt added a comment.Jun 13 2018, 5:40 PM

Hm. How does the linear filtering in combination with this patch cause the blurryness?

Also, I have the impression that this patch changes the appearance of windows just slightly, fonts appear sharper. Is that actually possible or is it just a placebo?

@zzag
I'm a bit lost as to what you're saying the status of this is. Are we saying this does cause bluriness?

If there are any drawbacks, we could avoid doing the half pixel correction on the Window Contents. They're not in an atlas so standard clamp to edge will work just fine.

zzag added a comment.Jun 13 2018, 7:15 PM

Hm. How does the linear filtering in combination with this patch cause the blurryness?

@zzag
I'm a bit lost as to what you're saying the status of this is. Are we saying this does cause bluriness?

My assumption why window content looks blurry: a part of a texture that is being mapped onto content rect is a little bit smaller(by 1px), so linear filter kicks in and "blurs" content.

If there are any drawbacks, we could avoid doing the half pixel correction on the Window Contents. They're not in an atlas so standard clamp to edge will work just fine.

So, here's window content without half pixel correction:

It looks like OpenGL scene clamps textures to edge by default. https://github.com/KDE/kwin/blob/master/plugins/scenes/opengl/scene_opengl.cpp#L1512

zzag added a comment.Jun 13 2018, 9:26 PM

@davidedmundson

Clamp to edge won't help on X11, because as WindowPixmap::contentsRect docs state

The geometry of the Client's content inside the pixmap. In case of a decorated Client the
pixmap also contains the decoration which is not rendered into this pixmap, though. This
contentsRect tells where inside the complete pixmap the real content is.

It looks like this on X11

Content of dolphin window texture on X11.

On Wayland, it seems to be a different case:

Content of kate window texture on Wayland.

So, half pixel correction makes sense only on Wayland, right?

zzag added a comment.Jun 13 2018, 9:30 PM

Also, I have the impression that this patch changes the appearance of windows just slightly, fonts appear sharper. Is that actually possible or is it just a placebo?

Are you using subpixel rendering?

fvogt added a comment.Jun 13 2018, 9:31 PM
In D13382#277983, @zzag wrote:

Also, I have the impression that this patch changes the appearance of windows just slightly, fonts appear sharper. Is that actually possible or is it just a placebo?

Are you using subpixel rendering?

Yes.

zzag added a comment.Jun 13 2018, 9:48 PM

Yes.

The worst case: the half pixel correction(to be precise the nearest neighbor filter) messed fonts.
The best case: your mind plays tricks on you. :D

Need to check how fonts are rendered with/without HPC.

If it indeed messes fonts, the only viable solution is padding, I guess.

zzag updated this revision to Diff 36123.EditedJun 13 2018, 10:01 PM

Do not correct window content quads

Half pixel correction causes blurriness with the linear filter. We can
avoid the correction of window content by using clamp to edge wrap mode.
Because on X11 window pixmap also "contains" borders, there will be some
bleeding. On Wayland, there is no such problem.

zzag updated this revision to Diff 36124.Jun 13 2018, 10:03 PM

Delete unrelated change

zzag added a comment.Jun 16 2018, 5:01 PM

I'll try to implement the approach with padding. Does Qt have something to clamp QImages to edge? Currently, I'm thinking about implementing QImage pad(const QImage &src, int padding).

zzag abandoned this revision.Jun 16 2018, 8:13 PM

Superseded by D13575.

My preferred long term solution to this problem would be to render the window untransformed to a texture, and transform that texture instead of the individual parts. This is something kwin will have to do anyway to implement wayland sub-surfaces in a conformant manner.

zzag added a comment.Jun 17 2018, 5:01 PM

My preferred long term solution to this problem would be to render the window untransformed to a texture, and transform that texture instead of the individual parts. This is something kwin will have to do anyway to implement wayland sub-surfaces in a conformant manner.

Yeah, it sounds better and would work for both X11 and Wayland. So, we don't need D13575, right?

In D13382#279309, @zzag wrote:

My preferred long term solution to this problem would be to render the window untransformed to a texture, and transform that texture instead of the individual parts. This is something kwin will have to do anyway to implement wayland sub-surfaces in a conformant manner.

Yeah, it sounds better and would work for both X11 and Wayland. So, we don't need D13575, right?

It depends on whether we want it as a temporary solution.