HDR Support in Krita
Open, WishlistPublic

Description

We want to make Krita support working on HDR images using HDR displays, instead of using tonemapping through OpenColorIO.

The following parts are needed:

• Make Krita’s canvas display an HDR image tonemapped to the range available to the HDR screens.(1000, 600 and 400 nits)
• For an HDR image it should possible to edit it tonemapped on an SDR screen without unreasonable clipping of the range.
• Improve tools that might not work well with HDR if tonemapping is not present:

◦ At least one color selector should have the ability to select color at any luminance step in the HDR-10 supported range of 0-10000 nits, or more specifically the PQ (EOTF 2084) values 0-1023. The color selector should display the colors as HDR not SDR.
◦ Gradients need to be calculated in HDR mode and have the HDR color range 
◦ filters should support either support HDR mode or be disabled when editing HDR content

• Simplify integration of Krita to FFMPEG to support seamless HW accelerated HEVC 10bit encoding when creating HDR10 animations in KBL platforms
• Add BT.2020 color gamut support and SMPTE 2084 for luminance storage and rendering
• Add/improve support for HDR file formats both using uncompressed (or lossless) high quality formats, and also compressed formats suitable for the web and emailing. OpenEXR, TIFF (more?). If possible also add AV1, JPEG XT, JPEG 2000, Extended JPEG or JPEG-HDR and Extended PNG

rempt created this task.EditedJul 23 2018, 7:46 AM
rempt triaged this task as Wishlist priority.
rempt updated the task description. (Show Details)

A patch that addes a hdr viewport to luminance-hdr: https://github.com/LuminanceHDR/LuminanceHDR

rempt added a comment.EditedJul 23 2018, 8:37 AM

Soemone else is trying to do the same thing: https://www.reddit.com/r/opengl/comments/8n0r3a/native_hdr_rendering_windows_10/

And example of using NV_DX_interop2:

rempt added a comment.Aug 28 2018, 7:45 AM

https://git.code.sf.net/p/mrviewer/code mrviewer-code can show the Desk.exr image in hdr with opengl/cg; no d3d

rempt added a comment.Aug 29 2018, 8:53 AM

Note: https://github.com/halogenica/WGL_NV_DX shows how to use opengl and direct3d in one window, but notes that this extension is no longer supported by recent drivers, and was always very shaky.

Update:

I am looking into three directions:

  1. Use Direct3D directly
  2. Use Direct3D through Angle (https://github.com/google/angle or https://github.com/Microsoft/angle).
  3. Use OpenGL directly

I have discarded a fourth approach.

Direct3D Directly

  • This is what VLC uses, for instance
  • And what is provided in the LuminanceHDR patch Fred provided me with
    • I never got LuminanceHDR to run correctly
    • I've integrated that patch in a test application and saw HDR pixels, but not the correct image
    • A side-note: it basically converts input into an 8 bit image, then the code stretches that back into a HDR range
  • For Krita, the problem would be that we would only be able to create a HDR view window using Direct3D, we wouldn't be able to make that into an actual editable canvas because we use the Qt OpenGL QPainter implementation to draw things like canvas decorations for tools. Porting Krita's canvas completely to pure Direct3D is just too much work.

Angle

  • Krita uses Angle on Windows for systems with an Intel GPU because most for most GPU's Intel's OpenGL drivers are very broken. We already struggled with when working on Krita sketch, and the situation has not improved.
  • Angle is packaged with Qt: https://wiki.qt.io/Qt_5_on_Windows_ANGLE_and_OpenGL
    • The version of Angle that is packaged with Qt only supports OpenGL ES 2, which does not support 16 or 32 bits floating point textures: when using Angle, Krita's support for HDR images is already broken. The exposure slider in the lut docker doesn't work because the relevant shaders that are part of opencolorio cannot be built (http://opencolorio.org/).
    • There is a newer version of Angle that does support HDR. We know this because Google Chrome uses Angle and can show HDR content. We also have access to the patch set that enables HDR in Google Chrome.

-> I need to get answers from Qt people about Angle and how hard it would be to update Qt to Google's Angle. Qt wants to use the Microsoft Angle, which probably doesn't support HDR, since it dates back to 2017.

OpenGL

  • Krita uses OpenGL natively for its gpu accelerated canvas, and has used that since 2005.
  • OpenColorIO which is what VFX pros use to manage HDR data and convert from scene refered to display refered uses OpenGL natively: we need this.
  • It is possible to show HDR content with pure OpenGL: https://sourceforge.net/projects/mrviewer/ is an application that can show the OpenEXR desk image correctly. (http://download.savannah.nongnu.org/releases/openexr/openexr-images-1.7.0.tar.gz)
  • I am working on a test application that uses Half (16f) format textures to show HDR content in opengl, but I haven't got that working yet.
  • Using OpenGL directly breaks the GUI though: it introduces the "dim grey UX elements" issue, where things like the menubar, toolbars, dockers etc. are drawn wrong

Direct3D + OpenGL

Additionally, there's a fourth direction that I've investigated, which is creating a direct3d surface and share that with OpenGL. However, the extension for that (WGL_NV_DX) is no longer supported according to the one example I've found of someone who got that working: https://github.com/halogenica/WGL_NV_DX

I'm at the moment working on 3.

rempt added a comment.Sep 10 2018, 1:55 PM

If Angle is enabled, OpenColorIO doesn't work:https://bugs.kde.org/show_bug.cgi?id=397977

After patching that problem out, the shaders don't compile.

Krita could not initialize the OpenGL canvas:

Failed to add fragment shader source from file: highq_downscale.frag - Cause: ERROR: 0:9: '' : No precision specified (sampler)
ERROR: 0:17: 'texture3D' : no matching overloaded function found
ERROR: 0:17: 'rgb' : field selection requires structure, vector, or interface block on left hand side
ERROR: 0:17: 'assign' : cannot convert from 'const float' to 'mediump 3-component vector of float'
ERROR: 0:28: '' : No precision specified (sampler)

In T9256#159693, @rempt wrote:

Angle

  • The version of Angle that is packaged with Qt only supports OpenGL ES 2, which does not support 16 or 32 bits floating point textures: when using Angle, Krita's support for HDR images is already broken.

Correction: That version does support OpenGL ES 3. The OpenGL canvas does fail when used on an OpenGL ES 2 context (not sure what features needs 3 though) and I specified GLSL ES 3.00 (only supported since OpenGL ES 3) for the shaders (see a19df1a8e96ea58f9304ae5402a2020a08b30582)

In T9256#159716, @rempt wrote:

If Angle is enabled, OpenColorIO doesn't work:https://bugs.kde.org/show_bug.cgi?id=397977

After patching that problem out, the shaders don't compile.

Krita could not initialize the OpenGL canvas:

Failed to add fragment shader source from file: highq_downscale.frag - Cause: ERROR: 0:9: '' : No precision specified (sampler)
ERROR: 0:17: 'texture3D' : no matching overloaded function found
ERROR: 0:17: 'rgb' : field selection requires structure, vector, or interface block on left hand side
ERROR: 0:17: 'assign' : cannot convert from 'const float' to 'mediump 3-component vector of float'
ERROR: 0:28: '' : No precision specified (sampler)

I hadn't touched OCIO when adding support for ANGLE, but these are some minor differences between desktop GLSL and GLSL ES. For example floating point stuff needs a precision qualifier (see sections 4.5.3 and 4.5.4 of https://www.khronos.org/registry/OpenGL/specs/es/3.0/GLSL_ES_Specification_3.00.pdf). The identifier texture3D probably needs to be defined as texture (related to D13485, also see https://gamedev.stackexchange.com/questions/138384/how-do-i-avoid-using-the-wrong-texture2d-function-in-glsl/138385#138385 and the linked post in there). The other issues shouldn't be too difficult either.

rempt added a comment.Oct 8 2018, 9:36 AM

More up to date opengl/direct3d interoperability extension: https://www.khronos.org/registry/OpenGL/extensions/NV/WGL_NV_DX_interop2.txt Intel says it supports this on Windows.

A blog by netflix on their hdr implementation: https://medium.com/netflix-techblog/enhancing-the-netflix-ui-experience-with-hdr-1e7506ad3e8

Work in progress patches on adding Float16 surface support to Qt's angle. The patch does not work atm. I'm saving it here just show the basic idea:

Patch for hdrtest:

Patch for qtbase

With these two patches the correct context seem to be created, but the the window becomes black. It might be related to internal Qt's caches... I didn't check yet.

It is worth noting that OCIO is undergoing tremendous changes at the moment, in particular on the GPU path. Autodesk has completely redesigned the chain which outputs 1:1 with CPU path.

In particular, https://github.com/imageworks/OpenColorIO/pull/539 is worth examining.

The current 2.0 changes are in master of OCIO proper.