Fixing resource management properly (includes tagging)
Open, HighPublic

Description

Resource Management

Purpose of the rewrite:

  • The current implementation is very buggy
  • The current implementation does not scale
  • The current implementation loads all resources on start-up
    • this is slow.
    • this takes a lot of memory: default pack is 120 mb of memory
    • even if the brushes get dropped after loading, the least amount of memory management is better because it takes less time.
    • some people have 20K amount of resources. These people don't want to have all of those loaded all the time. We wish to offer people grouping.

Grouping and Tagging

Resources should be groupable. Users need to be able to select which resources are visible in the user interface, group resources related to tasks together. Groups can be heterogenous. We have the following options to group resources:

  • Bundles
  • Library (Adobe brush library)
  • Tags:
    • The most flexible.
    • Usability is a bit buggier (adding/removing) Four types of tags:
      • Krita tags
      • Bundle tags. We have both bundles and tags, for each bundle an automatic tag is generated. Sometimes when the bundle is removed. Automatic Bundle Tag is troublesome because they have an ugly name.
        • We probably should stop automatic generating of tags for bundles. Bundles can be enabled and disabled in the resource manager.
        • We should allow search by bundle name in the resource browser search bar
        • We should recommend that bundle authors provide useful tags inside the bundle
      • Tags inside the bundle.
      • User tags
  • Resources can refer to each other. (tips, patterns)

We will consider bundles and adobe resource libraries as resource storage and not as user-visible grouping mechanisms. All grouping in Krita will be done with tags. Bundles and adobe resource libraries are read-only. (We will provide a way to generate and regenerate bundles.

Tags are considered to be a separate resource.

Translating tags

The tags Krita provides by default should be translatable. Tags contained in a bundle should also be translatable. We will provide a default set of useful tags that can be translated by the i18n community. Bundle authors can use these tags, or provide tags with translations. The bundle will provide a domain for translations: if a bundle contains a translated tag that is a duplicate of a tag provided by Krita or by another bundle, krita will use the translation only for the resources in the bundle.

Resource locations

Right now we have 2 places of resources that get loaded.

  • resources contained in the Krita installation: these are read-only
  • User resources.

There is a third place, namely .kra files, which can contain gamut masks and palettes. Maybe .kra files should be treated as temporary bundle storages.

These two folders are tricky to keep in sync and this gives a lot of trouble. We will package the default resources in the application binary through Qt's resource system (http://doc.qt.io/qt-5/resources.html). The bundled resources will be treated as a "template". On first run, the resources will be placed in the user's resource location. The user will be able to select one location where Krita will store resources.

  • There will be a button for restoring resource(s) to factory?
  • How to handle modified resource overriding.
    • Only copying files over when the brushes are new.
    • UUID?
    • Store database in bundle?
    • How to handle duplicated files with different UUID?
      • Scan function?
      • GUI duplicates?
      • Force duplicate brushes to be named differently?
  • update:
    • user didn't change anything
    • user changed stuff
    • User deleted stuff
    • Krita changed
    • Krita Deleted
    • Krita added stuff.

The user's local resource storage system will contain a database with metadata about installed resources (name, icon, history, location of the resource). The database should be sufficient to populate the resource selection models. Only when a resource is accessed should it be loaded.

It must be possible to locate the resource storage in a synchronized folder, like Dropbox, onedrive, icloud, google drive or nextcloud. The system should not have absolute paths to resources.

Note: should it be a goal to support network drives? If so, the database should not be in the storage location, because that would give us all the headaches of a single-user database masking as a multi-user database.

Synchronizing the database and the folder: this should be light-weight. We will check for:

  • filename: if we cannot find a file, or a new file appears, the database must be updated
  • creation/modification date. If the date differs, we need to run a checksum to determine whether there's been a real change.

This takes care of the situation where resources are changed with outside applications.

It should be possible to migrate a resource storage to a new system. Either through zipping up everything or an export/import mechanism.

What happens with resources on updating Krita

User made no changeUser changedUser Removed
Krita made no changeNo change
Krita changedCopy to userKeep user, no historyKeep remembered
Krita remove simpleRemoveKeep
Krita force removeRemove even when used a lotKeep
Krita addAdd resource

Simple Remove: Brush has been removed from the default set in a newer version of Krita.
Forced Remove: Brush has been determined to be buggy, and we want to be sure it or any derived brush gets removed.

We will implement a mechanism that checks how often and how recent a resource has been used: if Krita removes a brush from the default install but the user has used this resource often and recently, it will be kept and become a user-defined resource.

This means that a Krita update needs to have a list of resources removed since we implemented the new resource handling.

We might want to allow the user to review changed resources on udpating Krita. This should not be shown by default on updating, but as a flag in the resource history view. See next section.

What happens if the user adds an old compatibility bundle that contains resources that have the same "identity" as a new resource? The latest added resource will be the current version. Add a bundle with 2.9 resources that have the same uuid as current resources, and the current resources will be replaced, the old ones will become history states.

History of resources

We will implement a history management dialog that will show the history of resources: creating/installation, modification, removal, restore.

  • Make it possible to allow users to step backwards: recover earlier states of a resource
  • This needs restore to system or just clearing the history.
    • Add a collapse history state function which will delete everything but original and last state.
    • Handling states: a history slider
    • Locking states: users can flag a state for keeping, even if history will be cleared or collapsed
      • Which doesn't get removed.
      • Add favourite/bookmarked states
    • History can be collapsed automatically if there are more than X states: locked, favorite or bookmarked states do not count towards X.
    • Make default: a certain state that will be used as a default state to restore to.
  • Restart from a history?
  • Hide the resource and history cache from the user.

The history view should also have a resource inspector to view:

  • Metadata inside kpp files
    • GIMP and others delete the metadata, so that is awkward.
    • Users should be able to see the md5sum for resources. In fact, all the boring information should be visible to the user in some kind of distant window.

The resource inspector should also be available from the bundle editor.

Resources that contain resources

We will not consider bundles or libraries to be resources in their own right but as a storage mechanism.

Examples:

  1. Adobe brush libraries:
    • Treat the abr library as a bundle? Yes.
  2. Bundles
  3. Folders as a sort of bundle

Vector libraries are one resource per library, swatches are one resource per swatch file.

What is a resource, and what is not a resource

  • ASL (Adobe Style Library)
    • Is a resource without resourceserver.
  • Templates: though they should be
  • Workspaces and Sessions are technically resources.
  • Text/Paragraph/Shape styles: are not implemented yet. There is already a design for this.

Translating Resources

  • We currently translate brush presets and brush tips and other Gimp-based resources. We will stop doing this, because it makes it very difficult to support tutorials. However, we will show the translations as tooltips.
  • Translating templates (https://phabricator.kde.org/T8759).
    • Use keywords/vocabulary which can be translated.
    • And then when we create an image from the template it translates the keywords into proper language.
    • We can also do this when creating a new image

Resources that have no extensions

Not all resources have no sensible extension
Dmitry: Solve by database.

Dynamic Resources

These get created by Krita

  • They cannot be edited
  • They cannot be deleted.
  • Only gimp gradients can have foreground/background as part of the gradient definition.
  • Clipboard brush?
  • Clipboard pattern -- custom brush.

No resource state

If there's no resources at all, what should Krita do?

  • Create a new default brush?
    • If this situation happens, krita informs user that resource folder could not be accessed.
  • Crash?
  • No resources.

Answer: Just stick all resources into a qrc and let noone near them? Dmitry suggest using the paintopfactory. Decision made affects unit test.

Editing Bundles

If the user edits a resource that originates from a bundle we will not automatically save the new version in the bundle. This is hellish when user changes a bundle, and then shares it again, it may seem that the original bundle with copyright attached is kinda awful.

We _will_ make it possible for bundle authors to regenerate bundles from edited resources.

  • Make changes to brushes that are in bundles, having some kind of updating of the brushes inside the bundle with the changes brushes outside.
  • Using history to check if brushes have been changed.
  • Also works for removing and adding.

The current bundle creator/editor is sufficient gui-wise, though bugs need to be fixed.

  • Create a wizard that creates a bundle from a set of tags.
    • Awkward if there’s patterns + gradients.
    • Bundle creation is pretty easy due to automatic pattern and brush-tip resolution.

If the bundle format changes, it will be by extending the current format. No automatic conversion of older bundles is foreseen.

  • People really really want to get rid of their bundles: just disabling is not enough.

Get Hot New Stuff/share.krita.org

  • Clemens is still putting money in it, two fte are working on the server side
  • The protocol is pretty hard though. The work done in 2017 was not useful.
  • We still have no moderation: the design has not been adapted, so all share sites are one big database.

RSS feeds for bundles as an alternative? → Just researching what is possible in terms of registery. We could have an upload queue that we could moderate, and then publish as an rss feed and page on the website. The rss feed could be shown in the welcome page or the resoruce manager.

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
bcooksley changed the visibility from "All Users" to "Public (No Login Required)".Aug 12 2019, 10:13 AM
rempt added a comment.Oct 25 2019, 7:03 AM

Meeting notes:

woltherav added a comment.EditedOct 26 2019, 4:17 PM

Ok, I did some attempts at fixing resource views, but I only managed to make a few fixes.

  • ResourceLoaderRegistery doesn't specify resource-specific mimetypes. So the dialogs' filters are just showing everything.
  • Syncing doesn't delete resources, this is because of the bad sql query as posted in the MR.
  • Sometimes syncing will just keep on creating new resources for a given type(because the versioning scheme is messed up) In my case this is gradients.
  • The KisResourceGridProxyModel has too many rows, as you can see from the pattern and brush preset dockers. I am not sure even if stuff like the gradients shows the right amount of entries either, and the brushtips I cannot access at all.

Outside of that...

  • Patterns work with custom pattern, I fixed that :)
  • Patterns can be loaded from file, minus the issues with the proxymodel and the mimetypes as noted above.
  • Patterns workboth with the fill tool and the 'fill with pattern' command.
  • Patterns crash with the geometric tool. Crash is in QThreads, so I guess the pattern is handled confusingly.
  • Could not check Textured brushes, because the brush settings are too crashy.
  • Fill layers do not work with patterns, show nothing.
  • Gradients can be saved now.
  • I ensured that the pattern and gradients look a little better now, but the gradients in particular still look funny.
woltherav added a comment.EditedOct 31 2019, 3:51 PM
Resource TypeDoneSaving/Editing TODODeletion tested
PaintoppresetCan't test because settings pop-up is broken, import worksX
BrushtipsXCan't test because settings pop-up is brokendeletion crashes
GradientsXEditing gradients works a little funny, and thus versioning won't work right now, needs a lot more workX
PaletteWaiting for the Local storages stuff to be finishedX
PatternXnothing.X
WorkspacesXnothing.X
Vector Libraries-We cannot save, edit or import these to begin with.-
Window layoutsXnothing.Crashes on select
SessionsXnothingCan't test because of proxymodel bugs
Gamut MaskXnothingCan't test because of proxymodel bugs

Deletion, when it works, seems to revert next krita session(already noted in the bugs spreadsheet). Proxymodel bugs makes it hard to select a resource, and the crashes are a little too obscure for me to fix.

After poking at the proxygrid model again, I sincerely suggest we start using https://doc.qt.io/qt-5/qlistview.html with the flowmode enabled instead of making qgridview work for us :(

rempt added a comment.Nov 20 2019, 1:47 PM

Resource Rewrite Discussion

Krita Mini-Sprint 2019-10-22

Attendees:
  • Boudewijn Rempt
  • Wolthera van Hövell tot Westerflier
  • Dmitry Kazakov
  • Agata Cacko

State of the rewrite

  • Technical details this morning.
  • All pieces have been started; no new classes are needed, but many parts don't yet work.
    1. DONE
    2. copy over resources to the user's resources folder
    3. initial creation of the database schema
    4. initial filling of the database
    5. folder storage class
    6. tag loading from desktop files
    7. updating the cache db with changes done directly on the resources folder
    8. bundle storage class
    9. Database explorer UI
    10. models for integrating with the UI
    11. implement dirty resources mechanism
    12. Remove all hints that bundles are resources.
    13. implement metadata mechanism for resources --> the KisResourceModel class has many unimplemented methods atm
    14. Implement in-memory resources, that are deleted from the db when Krita starts (add to locator cache, set id on resource, add to database, update model)
    15. DOING
    16. Implement KoResourceServer as a shim for KoResourceModel
    17. fix loading the actual preset in the preset delegate
    18. TODO
    19. PLUGIN \[Agata\] ASL storage class (Adobe Layer Style). Needs KisLayerStyleResource class. Needs good unittest. And ASL contains gradients, patterns and layerstyles.
      • Complexity: Simple
      • Time needed: 2-3 days, refactoring might take longer because the rest of krita isn't prepared to see layers styles as a resource. Lots of UI work.
    20. PLUGIN \[Agata\] ABR storage class (Adobe Brush Library)
      • Complexity: Simple
      • Time needed: 2-3 days, probably no gui rework needed.
    21. UI \[wolthera\] adding/updating/removing resources from the UI (resource views)
      • Complexity: Testing and bugfixing (check all resource types, where they are removed and added etc.). Simple but takes a lot of time
      • Time needed: a lot, min. 2 weeks
    22. UI/DB \[Agata\] showing/selecting/adding/removing tags from the UI (resource views, tags) - current code is difficult, too complex, it might be better to not reuse it.
      • Complexity: EXTREME!!!
      • Time needed: Month
        • tags system: we can drop the wildcards and things, autocomplete is interesting, but multi-select is important. The connection between the tag library and the tag gui needs rewritten. The current gui is sort of okay.
        • Make KisFavoriteManager a model-view class for presets and tags
    23. DB \[later\] database migration and versioning (loop over version, find update *from* version, execute and so on until version is up to date)
      • Needs testing, Write regression tests for that, Manual for regression testing on every release, discuss grouping tests with Anna-m. Create a db of version X, fill it with test data, convert it to the current version (say, X+4), check that the database integrity is not impaired. Repeat for every version.
        • Complexity: Doesn't require coding skills, just system design skills. But is fiddly. And tedious
        • Time Needed: recurs every time the database schema changes
    24. DB \[boud\] MD5sums need to be added to the versioned resource table.
      • Complexity: simple
      • Time Needed: 1 hour
    25. DB->PLUGIN->CORE->UI \[boud\] Resources in the kra file vs resources in the database: Every kra file will become an extra storage item, added and removed when the file is opened and closed. KisDocumentStorage . Only resources that are not the latest version as determined by md5 are added to the document storage, if they are the latest version, the ones from the original storage is used.
      • Integrate with loading layer styles: the layer style needs to refer to the resources in KisResourceLocator.
      • integrate with loaded .kra files
      • integrate with loaded .svg files
      • Complexity: Medium
      • Time Needed: 3 weeks
    26. CORE \[later, dmitry\] Make resources reloadable
      • fix reloading the preset if dirty presets is switched off (KisPaintopBox::resourceSelected). The problem is that the preset itself no longer knows where it comes from or how it should be loaded.
      • Integrate with dirty-preset functionality
      • Complexity: Medium
      • Time Needed: 2 weeks
      • NOTE: Might be simpler, depends on memory management for resources, all resources are in shared pointer, unless cloned explictely, so modify preset in one window, presets are modified everywhere. Presets are always dirty, because we're not making clones *anymore* , we need to add, ensure there's a reload method in koresource, right now there's only a load method, and this might not work appropriately, due the new concept of bundles.
    27. UI \[later\] Would be nice: icons for workspaces, generated or screenshots.
      • Complexity:
      • Time Needed:
    28. UI/CORE \[Wolthera\] bundle creator
      • Main complexity is fixing the bundle generator. Then the UI design. Then compatibility with old bundle format. Rewrite bundles, rewrite GUI.
      • Complexity: Advanced.
      • Time Needed: Month
      • Resource manager should go!
    29. UI \[wolthera\] bundle selector
      • Complexity: Simple
      • Time Needed: Week?
      • popup button in every resource view to select active bundles
      • changing bundles list should update all the views
    30. UI \[wolthera\] Adding resources: allow D&D of the resources on resource views (or Krita main Window)
    31. CORE \[agata\] versioned resources: for read-only storages: need design, works for folder storage (Saving modified/versioned resources that come from read-only storages.) Needs solid tests.
      • Complexity: Extreme
      • Time Needed: depends...
    32. DB/CORE/... \[dmitry, later\] resources dependent on other resources (e.g. preset -> brush and pattern) This needs to be stored in the database as links between resource id's and saved to bundles when creating or updating bundles.
      • Database work: add relations between resources: create_resources_resources.sql
      • Refactor creating presets that use gradients, brush tips or patterns.
      • Preset with dependent resources should create a KisDocumentStorage for this preset (if the dependent resources are new). It is memory storage and removed on every Krita restart.
      • Should be implemented on a level of: KisResourceCacheDB
        • Complexity: Extreme
        • Time Needed:
  • UI/DB \[later\] Implement undelete of resources dialog (or put in resource browser)

KNOWN BUGS

  • The resource viewer as used in dockers are broken, while the view widgets work in the database browser.
  • KGM cannot be loaded
  • Gradient previews are square.
  • Bugs in systems boudewijn hasn't touched. Eg. drawing and using resources in other methods may or may not leak due to the changes to the system.
    • Presets/Brush tips/Masked brushes
    • Patterns
    • Gradients
    • Palettes
    • Workspaces???
  • Need hidden flag for storages (or do I have that already)

How to proceed

  • Database things need to be done first.
  • Agata:
    • ABR/ASL needs to be done by the same person, and they can be done at any moment.
    • Versioned resources.
    • Tags
  • Boud:
    • will handle md5sums
    • resource storage for kra file.
  • Dmitry:
    1. dmitry should fix difficult bugs, ivan should fix easy bugs.
    2. linked/dependant resources
    3. maybe reloadable resources
  • Wolthera will come to Deventer for mini sprint for the bundles
    • Adding/removing/updating resources from the UI.
    • dnd of presets on main krita window.
    • bundle selector
  • Later:
    • workspaces icons
    • resources reloadable is completely seperate
    • migrations last
    • undelete of resources

Sorting Resources

There are 2 todo's: sorting resources in the popup palette and sorting resources in the resource viewer.

Resource Viewer
  • We need to have D&D of resources in the resource viewer to change the order (as in the palette viewer)
  • There needs to be a unique resource priority order number in the resources table, unique *per* resource type, not globally
  • On moving the resource to a new location, it needs to be renumbered, and all resources with an index > the new number need to be renumbered. In practice, this is fiddly and bug-prone, but it is doable.
  • There cannot be different sort orders for different tags; within a tag, the order is always the global one. This is actually what already happens when using deevad's hack with alphabetically grouping resources.
Popup palette

This is a *different* problem from the order problem in the resource viewer.

There needs to be a per-tag definition of the resources shown in the palette, with position linked to individual slots.

This would be a new table linking tags and resources.

woltherav added a comment.EditedNov 20 2019, 3:16 PM

Bundles

Mini bundle meeting today...

Activating bundles:

Bundles active/inactive should be done with a checkbox thingy. That in turn should be sent to the storage model to deactivate the bundle there (with qt::checkstaterole).

  • The storagemodel should become a singleton for this so the resourcemodels each can be more easily informed about disabled bundles... (come to think of it, proly the tag model(s) too would like to know the storagemodel is updated...)
  • Bundle active/inactive widget should be created.
  • The Resource model needs to be reset whenever the storagemodel has been changed.

Importing bundles

Either we...

  1. Copy the bundle into the bundle folder and sync db.
  2. Or do adddocumentstorage.

First is preferable.

Creating bundles

  • Whenever users manipulate bundles, they are manipulating the bundle storage. It is not until 'save' is pressed that the bundle storage is commited to a file on disk.
  • Format should remain mostly the same. Tags have changed though.
  • When a bundlestorage is turned into a bundle file, the files put in it need to be disabled elsewhere and the bundle needs to be activated. For this, we need to remove the files from the folderstorage, and figure out versioning for resources in bundlestorages, as otherwise we'll have duplicate resources.
tymond added a subscriber: tymond.Nov 22 2019, 2:37 PM

From today's discussion:

  • @rempt : ResourceLocator should have a method to add a storage (blocks progress in ASL layer styles)
  • SVG Collections should stay one resource per collection for now
  • Window Layouts and Workspaces are only internally connected; they are different types of resources, so no subtype there.

For tagging notes, see T12044

  • @rempt : ResourceLocator should have a method to add a storage (blocks progress in ASL layer styles)

Not sure how relevant this is, but KisResourceLocator::addStorage now exists.