GSoC 2020: Dynamic Fill Layers in Krita using SeExpr
Closed, ResolvedPublic

Description

Synopsis

Layers are one of the core concepts of digital painting. They allow artists to control different parts of their artwork at once, for instance, color, lighting, lineart, as well as texture. A key feature of them is their ability to be resized, composited, renamed, grouped or deleted independently of the rest of the document.

Patterns and textures are also essential components of an artist’s toolbox, allowing them to represent the intricacies of a physical material. They come in two forms: bitmap textures, which are images contained in e.g. PNG or OpenEXR files, or procedural textures, which are generated on the fly using their mathematical representation.

KDE’s Krita painting suite supports using patterns and textures through two types of layers, File or Fill Layers. However, neither of them let artists create dynamically generated content: File Layers are inherently static, and Fill Layers support only color fills (like Paint Layers) or basic pattern rendering.

The goal of this project is to let artists create dynamic content through a new, scriptable Fill Layer. To this effect, I propose to integrate Disney Animation’s SeExpr expression language (Walt Disney Animation Studios (n.d.)) into the Krita painting suite.

Deliverables

The delivered implementation should be expected to include:

  • support for dynamically generated content on a KisLayer
  • an UI for creating and editing the associated script
  • integration tests
  • user documentation

Project Schedule

Weeks are measured from Mon-Sun to align with GSoC’s Coding Period start date. All weeks after the Community Bonding period include writing a weekly report that will be published on Planet KDE and amyspark.me.

WeekTimeframePlans
May 4 -- May 29Research SeExpr layout and dependencies, document KRA format and proposed layer specification.
1Jun 1 -- Jun 5Define the new generator. Build SeExpr and try calling it from within Krita.
2Jun 8 -- Jun 12Test rendering of a script from a string. Write unit tests.
3Jun 15 -- Jun 19UI design. Push a MWE for entering a script.
4Jun 22 -- Jun 26UI design and implementation.
5Jun 29 -- Jul 3Write UX tests.
6Jul 6 -- Jul 10UI polish.
7Jul 13 -- Jul 17Integrate SeExpr into the Krita packaging process.
8Jul 20 -- Jul 24General testing and bug fixes.
9Jul 27 -- Jul 31Push another alpha build. Testing and bug fixes.
10Aug 3 -- Aug 7General testing and bug fixes.
11Aug 10 -- Aug 14Document the new layer in the manual.
12Aug 17 -- Aug 21Reserved week (just in case anything happens)
13Aug 24 -- Aug 28Reserved week (just in case anything happens)
Aug 31*Finish line!* 👍

The full proposal is available at this Google Docs link.

Related Objects

lsegovia created this task.May 5 2020, 5:08 PM
lsegovia triaged this task as Normal priority.
lsegovia updated the task description. (Show Details)May 5 2020, 5:14 PM
rempt added a subscriber: rempt.May 6 2020, 8:19 AM

Get in touch with Wolthera when it comes to documenting the .kra format: she has started doing that for the manual, so it might be useful to help fill out the specification she's set up.

Hey @lsegovia, @eoinoneill and I just spent a while testing out your SeExpr patch and I want to share some of our initial reactions.

We intentionally went into this completely blind, without having read any documentation or seeing many examples of SeExpr script, for the simple reason that this is how the vast majority of users (many of whom are not very technical) will first experience this feature.
While I don't want to be condescending, we have to make the base assumption that our users are generally less technical than we are. (Even if this feature is intended for the more technically inclined members of the Krita community!)

With that said, here's some of our thoughts;

1.) A direct GUI link to some kind documentation would be a massive help for first time users.

I know that you're not quite at the point in your project where you're thinking about documentation, but I think it would be nice to eventually have some kind of "Help/Guide/What is this?" button directly on the SeExpr interface that will bring confused users immediately to some kind of beginner's guide, tutorial, or other easy-to-understand documentation.
Even if it doesn't teach people every nook and cranny of the language, I think it'd be a great way to put users on the path to learning more.

2.) A good selection of presets would be really useful.

Not only will it help new users learn by example when the presets are simple, it will also help make the feature much more usable and powerful even for users who aren't at all technically inclined or interested in learning SeExpr.
The Yamaha DX-7 was one of the most popular synthesizers ever--not because it was easy to use (it was insanely unintuitive), but because it had a decent set of presets and good system for sharing. =]
Maybe we could have a list of presets on the right side of the interface.

3.) Can't we avoid manual updates by listening for changes to the SeExpr editor?

We should be able to automatically update the generated output by emitting an update signal whenever the user edits the script or tweaks a widget.
If we go this route, you can and should use a KisSignalCompressor to limit the number of update signals that are handled by the generator itself.

4.) Is it possible to achieve looping effects and patterns using SeExpr?

SeExpr might not be built for it out of the box, but it'd be really nice to have some ability to create looping textures and patterns.

5.) Some of the widgets could look a bit nicer. (I know you're still working on it.)

Overall, we both think it's very cool and, after looking up a bit of documentation, I can definitely see the utility of it in all kinds of situations.
Having the ability to do procedural image creation in Krita can open the door to all kinds of new things, and if we can make it easy to use for technical and non-technical people alike, then I think it'll be a fan favorite. =]
Great work so far!

@emmetoneill , thanks for the feedback!

1.) A direct GUI link to some kind documentation would be a massive help for first time users.

I think this would be great! Do you know previous work, in Krita or other apps, to have as a guide?

2.) A good selection of presets would be really useful.

Yes, @Deevad told me the same. SeExpr has a widget for this, example here, but it works with a hardcoded path. Work here is:

  • to reimplement this from scratch, integrated with our resource system
  • to get example scripts. Here we need help from artists, and @Deevad posted a thread asking for help last week at krita-artists.

More generally, we need to think how we'd represent (data-structure-wise) these presets inside our resource system.

3.) Can't we avoid manual updates by listening for changes to the SeExpr editor?

Yep! The generator listens to the button widget for the sake of ease of testing. (The example editor does this as well.)
Do you think it'd be better to connect it to the editor's signals instead? (with some delay, I'd say? cc @dkazakov because I don't know how the signal compressor would work here)

4.) Is it possible to achieve looping effects and patterns using SeExpr?

I take it you mean seamless patterns? I do map 1:1 SeExpr's canvas size with Krita's, but SeExpr doesn't have any limitations regarding that, afaik.

5.) Some of the widgets could look a bit nicer. (I know you're still working on it.)

The widgets you see are entirely Disney's work. I'll start working soon (tm) on translating them, but I've not reviewed any of them for design kinks. It'd be great if anyone screenshots them and notes their flaws down!

Hey everyone,

I've been working hard on making SeExpr (both base and widget library) behave much better with Krita.

So far, the changes on the SeExpr side are:

d9a7c387d1caeef675e1ff78cd8011003e703239 Do not require ExprControlCollection on ExprEditor creation
771f8b52315ff3eefa06dadeabc93792a08b3a7c Fix flickering of QTooltip help
bb7d81d1c379c9364bdcb778490ef08c6eeb9e4d Fix buffer underflow in CCurveScene::getCPixmap
43108496fd344170a3d748f96b13030fbeb5017c Clean Editable.h up
1e008feb7e3bed158ef64d04ea842942a94bc3b4 Consistence changes in UI
c2d6ba2aa171b65c35c97c45acbb4d3656b68856 Install only the necessary headers
ad7dfd28306bf6c2b6c2a211f8d838a984e35d62 Elide ExprControl labels
c38cf7a9d6e0a2dbf552b3e060b283d43535cd57 Drop last remaining bits of Platform.h overrides
75c41b41ea7f24d0710cf359c1fb5c2d3467fadc Improve, localize, and display expression errors
c69149d9b034890ec152d74d1fff47da4306bc25 Platform.h header cleanup

These changes allowed, among other things,

  • migration of the SeExpr dialog back to the .ui file (as with the rest of Krita)
  • to install only the necessary headers
  • to fully localize (when enabled) detected errors in SeExpr scripts

The next, and final, objective will be to integrate SeExpr scripts as bundle-able resources according to the 4.3 specification, not Agata's rewrite yet.

@rempt -- I could have as stretch goals:

  • Research multi-threaded renders: SeExpr provides a way to detect threadsafe expressions and share these among workers, and Agata's MR for Histogram is really instructive.
  • Move SeExpr logging to a interceptable approach. I had in mind logfault, which allows for a Qt shim on the Krita side.
  • Make SeExpr fully locale-proof. I added some hacks wherever sscanf and atof are used (they are sensitive to decimal point changes between locales), but I'd like to not deal with locales at all.
rempt added a comment.Jun 24 2020, 8:00 AM

Sounds good, though it might be good to have as a goal "port to master", too :-)

  • Research multi-threaded renders: SeExpr provides a way to detect threadsafe expressions and share these among workers, and Agata's MR for Histogram is really instructive.
  • Move SeExpr logging to a interceptable approach. I had in mind logfault, which allows for a Qt shim on the Krita side.
  • Make SeExpr fully locale-proof. I added some hacks wherever sscanf and atof are used (they are sensitive to decimal point changes between locales), but I'd like to not deal with locales at all.

Sounds good, though it might be good to have as a goal "port to master", too :-)

Port to master has been done as https://invent.kde.org/graphics/krita/-/merge_requests/380.
As for the locale-proof requirement, it is not possible at this moment. See bugs #24 and #26 in the scn library.

I keep thinking about converting KisGenerators to the strokes system, to enable parallelism at the layer level, yet it scares me how invasive the changes will be according to @dkazakov .

Multithreaded KisGenerators (i.e. conversion to the strokes system) was proposed in https://invent.kde.org/graphics/krita/-/merge_requests/412. Please let me know if you want an appimage for testing.

Once merged, SeExpr will need an additional dummy-fix: there exists the possibility of having non-threadable scripts. In this case, the generator should report it does not support this feature. Although the Disney-provided functions are fully threadable, I'd like to address this for safety's sake.

lsegovia added a comment.EditedJul 28 2020, 2:00 AM

The SeExpr generator has been merged in e4dbb4d80c50e330a7f9807c2ac7fd611bc8d0ab (master branch) and 510f039cc676dfe69d44a5bc2d8144cf9a0874a4 (4.3).

The SeExpr documentation and tutorial is available as a MR: https://invent.kde.org/documentation/docs-krita-org/-/merge_requests/145 .

The SeExpr documentation has been merged in 9bb62c8ea128d9c5a059438108a5c8492d10e3c6 (master branch) and a71ad098ba1c33708d60365edf66fb8219674644 (4.3).

Additional stretch goals were:

As all MRs are merged, the coding period is finished, and my report has been delivered, closing this as resolved. 🎉

lsegovia closed this task as Resolved.Aug 25 2020, 1:43 PM