Improve Palette Docker
Closed, ResolvedPublic

Description

Kickstarter goal:
"Managing palettes and color swatches. We want to add tagging, searching, creating, saving and editing of palettes. Palettes are also limited to 8 bit sRGB colors, so we'll also create a new swatchbook format that allows for higher channel depth colors."

Bugs on bugzilla:
https://bugs.kde.org/show_bug.cgi?id=335200
https://bugs.kde.org/show_bug.cgi?id=333760
https://bugs.kde.org/show_bug.cgi?id=330329
https://bugs.kde.org/show_bug.cgi?id=369343

Implies:

  • Exposing tagging of palettes to the UI.
  • Better save and edit dialogues.(New palette UI is kinda... confusing)
  • Develop format for high bit depths and scene reffered colours. (Mail Create mailing list?)

Palettes (from discussion during halloween)

There’s 3 areas affected here:

  1. The native file format and internal data model.
  2. Import/Export of non-native formats.
  3. UI

The native file format and internal datamodel.

To some extent, the native file-format is an expression of the data-model, but in Krita we try to avoid data-dumps, so not quite that extreme.

  • We chose for a zip file, with a colors.xml, and the referenced icc-profiles. [done]
    • Outside of LCMS ‘built-in’ profiles all profiles should be stored and retrieved from the zip.
    • The XML stores the name, version, columns and comment and then has a child element ‘ColorSetEntry’ which is then used to store the data from the KoColorSetEntry. These are:
      • ID - string number unique to this color. - This is very common in palettes out in the wild, especially ones with spot colors where the ID is a unique code like 'OY348' and the name is some flowery nonsense like 'naive peachy pink'
      • Name - String with human readable name
      • Spot - bool for marking a color as spot, probably useful later, if not useful for python plugins.
      • bitDepth - a string that can be parsed to a KoID
      • Color - An XML which can be interpreted by KoColor::fromXML();
  • This model should be extended: [done]
    • We want to be able to have groups. These colorgroups CANNOT be nested. The idea is that users want to group certain colors to find them more easily. Such things exist in other formats as well as ‘pages’ or ‘groups’.
      • Within the datamodel, we will have one QVector<KoColorSetEntry> colors for non-grouped colors, ALWAYS. And a QMap with an group-identifier of sorts and the QVector<KoColorSetEntry> colors for that identifier. [done]
        • Make sure we can have a palette where the colors are ONLY grouped, or a palette file where none of the colors are grouped. [done]
      • Within the format this becomes a hierarchy. [done]
        • Figure out a method to error with group children in a group.
  • We want to allow saving a KPL file to a kra by default. This should have a special spot in the UI.

Import/Export of non-native formats.

  • Ideally, KoColorSet is refactored to have the import export seperated into plugins.
  • We also want to allow python import/export plugins.
  • Exporting to a file would be done via a seperate command. Krita should not be saving out to a non-native or non-gpl file while not prompted.
  • We will not be handling localised names internally for now. We have no method to do anything with a list of names and language locales.
  • When dealing with multiple definitions, we will go for the most precise one first and then go off the list.

Usability of the Palette Docker:

  • We need a better new palette dialog. The current one is implemented a bit lazily :p[done]
  • Add/Remove colors.[done]
  • Rename Colors[done]
  • Drag and Drop reorganisation of Colors.[done]
    • There’s also a QDnD action we can use here, maybe? (Though this would only be for QColors)
  • Groups - As pointed out above we want groups for...
    • Linking a group-color view to a layer. (Ex: Layer Character1 should show the group ‘character_one_colors’ when selected)
      • We need an ‘all colors’ view to default to. - Users will otherwise get confused.
    • Layer Split - Using the groups instead of individual colors.
    • Generate palette from image, using each layer to fill a separate group.
  • Select ‘closest’ color to the current foreground color when this is changed to give an idea which is the closest color to current. [done]
  • Palette docker should default to the KRA internal palette. This palette is auto-generated with a Black and White patch. Templates will be able to save palettes internally.
  • Swatch size, multiple options:
    • Fill out according to column size(current) [lets default to this]
    • Size defaulted according to Resolution.
    • Large Swatch Size + Column-based outlines has the issue of swatches disappearing from view.
  • Allow python plug-ins for manipulating the palette.
  • We need a dialog for editing the palette meta-data (Name, Comment, Columns)
  • Probably allow the same for a swatch. [done]

Other:

  • Append other palette to current - necessary for quickly having a existing palette added to the file-palette.
  • Search and autogenerate palette IDs vs. Palette Names.
  • Functions for appending and inserting color.
  • Internal image palette - Option to automatically fill it with color history.
  • Filter and Search options for the IDs and Names. -> These also need things in the KoColorSet.
  • ICC ColorLists will also need to be handled internally.

Todo:

  • @woltherav will be fiddling with KoColorSet and adding append/insert and similar functions.
    • She will keep overseeing @lsegovia 's parsing efforts.
  • @rempt will write python bindings for KoColor and KoColorSet so these can be manipulated, generated and read.
  • @rempt will write an initial palette docker based on the notes above. - It might be best to have something working before we do UI design to remove the more abstract elements from the conversation.
  • @rempt noted that the KisColorSetChooser.cpp (that is the little dropdown to choose the colorpalette) needs a serious make-over. Ask @scottpetrovic if he wants to look at this.
woltherav updated the task description. (Show Details)
woltherav raised the priority of this task from to Wishlist.
woltherav added a project: Krita.
woltherav moved this task to Kickstarter Stretchgoals on the Krita board.
woltherav added a subscriber: woltherav.

Initial Problem

Krit can't save HDR/Scene reffered values/values above 1.0 into any of it's availeble palette formats.
There's no file format which can carry these values.
To determine what a hypothetical new fileformat that supports these colours would need, we first need to look at the use cases of such a format, and then at the technology we have to work with.

Use cases
(Problems to solve):

What are palette files used for?

  1. Convenience. Being able to select 'primary red'quickly is a time-saver.
  2. Truth. Like the spot-colours situation, artists tend to tie a certain sense of authority to the values the computer marks as red and blue, while these are internally just numbers without much meaning. It's not unlikely that if a non-clipped format is developed, we need to bear in mind that people will attempt to use it in the illusion that it holds some kind of truth. This is a little problematic as we go to the technology side.

Technology

Krita has two colour management systems implemented:
lcms2, which handles ICC profile based conversion, and OCIO which handles interpretation and output based on LUTs.

In ICC profiles, the 'truth' aspect is somewhat easy to solve. ICC is build around the idea that a given colour has a true position in the XYZ coordinate system, and furthermore, we can embed ICC profiles into files and expect any ICC based CM to take that profile and use it to determine the colours.
You could, if you hate embedded ICC profiles, have all colours in XYZ coordinates, but that does make it difficult to edit for the artist if they open the file in a text-editor.

In OCIO the situation is not as easy: OCIO can interpret the input 'primary red' to whatever colour space you tell it is, and output it accordingly.
This means that 'primary red' could be the primary red of sRGB, ACES, Adobe RGB, you name it, with the same amount of ease.
It has been suggested to embed the name of the input colour space into the file to tell OCIO which input colour space to take, but I am wondering what we should have OCIO do if it does not have access to the LUT related to the input colour space that is named in the file.

Questions

  1. Is it wise to make an overal colour managed palette format, where we try to store the context of the colours, or are we better off just storing numbers only?
  2. So, what do we do when the palette file uses ACES, has a string with 'ACES' but OCIO doesn't have an ACES lut in it's availeble input colour spaces?
  3. How do we communicate to the user how colour space affects the 'truthiness' of the palette colours?
dkazakov raised the priority of this task from Wishlist to Normal.Oct 1 2015, 6:49 AM

I don't know anything about color spaces, but in practice I don't find much use for color palettes aside from strict palette management like in pixel art. I think it would be helpful to be able to see all the individual colors used, as sell as be able to edit them on the fly, eg recoloring sprites quickly. Seeing as krita is more of painting program I don't expect my opinion to weigh in too heavily, but I just wanted to add my two cents.

I don't know anything about color spaces, but in practice I don't find much use for color palettes aside from strict palette management like in pixel art. I think it would be helpful to be able to see all the individual colors used, as sell as be able to edit them on the fly, eg recoloring sprites quickly. Seeing as krita is more of painting program I don't expect my opinion to weigh in too heavily, but I just wanted to add my two cents.

This is indexed color management, and it's different from the palettes we are talking about, which are more swatchbooks.

Indexed color management unfortunately is something that would require a full rewrite of Krita to work. It's best you use Krita in conjunction with GIMP for this, as GIMP does have this functionality, and is quite sophisticated at it.

Some other thoughts:

We've had questions whether...

  • We could have a palette generated from the color image's history.
  • We could have palette files embedded into the *.kra file. Which would then be autoloaded somehow?
  • We can allow the reset of the columns set in a palette file(This is now only set on creation)

Then there's generation/mix functions, but those might be getting out of scope.

rempt added a subscriber: rempt.Dec 30 2015, 2:10 PM

There's also the swatchbooker format, which supports 16 bits color, cmyk and more, and which might be a good candidate to replace the ancient gpl format.

bcooksley changed the visibility from "All Users" to "Public (No Login Required)".Jan 16 2016, 8:53 AM
rempt claimed this task.Jan 22 2016, 9:16 AM
rempt moved this task from Wolthera's tasks to Bouds tasks on the Krita board.

We could have a palette generated from the color image's history.
We could have palette files embedded into the *.kra file. Which would then be autoloaded somehow?

That sounds great.
-Auto rearrangement of the swatches based on hue / sat / value would be great. But also the ability to grab one or more swatches and move them manually would be invaluable.

At the moment the size of the swatches is controlled by the width of the docker. Narrowing the docker causes the swatches to shrink down in size and vice versa. And they follow the classic square grid alignment.

What if we could have more freedom in placing the swatches? Like, a bunch of red hues at one place, then a couple of greens at another with empty space in between.

Taking this idea a bit further, what if we could also control the size of each swatch individually?

All of that still following the grid system for the placement of each swatch.

dkazakov moved this task from Backlog to Kickstarter Stretchgoals on the Krita: Next Features board.

How do people tend to create palettes? That said, I suggest also implementing the palette interface of Inkscape: Inkscape has a palette at the bottom of the window, and a small arrow at the end to choose the palette, as well as change the following options:

  • Size
  • Width
  • Border
  • Wrap

This interface is handy because a large row of swatches can be displayed this way without taking too much space, and the whole thing can be easily customized. It's so easy to access that I rarely use other color selectors in Inkscape. We could have something similar, except with a search bar for tags. I would also suggest adding 'history' to the options so a new swatch is added every time you pick a new color.

As for managing the palettes, that can be a separate pop-up. A number of palette generators exist online, perhaps Krita could implement one?

rempt updated the task description. (Show Details)Sep 26 2016, 12:39 PM

Ok, looking at @lsegovia 's patch, I realised that in Krita we always load palette colors as if they were spot colors(with a defined color space), and programs like Scribus and Swatchbooker don't necessarily do that. Which means that we need to figure out some method of having a palette color be created on the fly based on the given color space of the image if said palette color isn't a spot color.

This might explain why we have this old bug: https://bugs.kde.org/show_bug.cgi?id=339345
We load the CMYK using our default CMYK profile, while I suspect photoshop just loads it as a CMYK color in the given space of a CMYK image, and otherwise does load it in some other generic cmyk profile it has. I am not sure what to do about the latter(beyond reverse-engineering, which I don't particularly feel like). Maybe aco has some default cmyk space it uses? Or maybe we should actually load these as qcolor when not in a cmyk space and use whatever dark arts qcolor uses?

Thankfully, it seems that most spot colors in the wild are either LAB or RGB(and thus implicitly sRGB).

Ok, bugzilla rundown:

  1. https://bugs.kde.org/show_bug.cgi?id=369343 - Multiple Usability issues of Color Palette Docker

Tyson:

The current UI of Krita's Color Palette Docker is very confusing:

  1. It's called "Palette". It should be called "Color Palette" to provide context in the huge Docker menu, and to discriminate from Popup Palette.
  2. It doesn't work when no palette is loaded. New user will never figure out why there are buttons there, they can click on it, but nothing happens! Therefore, it should always have an empty palette loaded BY DEFAULT, until the user changes it.
  3. There is no indicator showing "No Color Palette Loaded" (disabled) state and no instruction in place to tell user what to do to enable it. When No palette is loaded, we should put instruction text into the empty docker, so the user may teach themselves.
  4. In "Choose Palette" dialogue, we must "Input a new name and Save" to create a new palette -- this is unintuitive. Again new user will never figure that out. We have 2 empty space for buttons there, why not add a "New Palette" button and a "Rename" Button side the existing ones? What's more, why not use right-click interaction there? That's the perfect situation for a contextual right-click, I think.
  5. Unintuitive icons. "Add Foreground color" should be an "Swap-FG/BG" icon; "Add Color" should be a Color Picker or +; Delete should be an "X" instead of a Stop sign. In fact, we are mixing DELETE with STOP SIGN everywhere in Krita, which very confusing.
  6. We also need to add "Resize" buttons to the docker. Again new user will never figure out to use Ctrl + Wheel.

    At the end of the day, I think we depend too much on the user Googling for instruction. In most cases, people just think that function of Krita is broken and give it up -- just like I did -- today I finally figured it out for the first time, because I needed it so bad and refused to believe it is still broken for so many years. Yet it took me quite sometime to finally understand Krita's Manual page on that topic. The interaction design is just too obscure, like Enigma! XD

    Reproducible: Always
  1. https://bugs.kde.org/show_bug.cgi?id=367694 - Support loading palettes saved as a CSS file

This refers to loading css palette files made by gimp... Also exporting them?

  1. https://bugs.kde.org/show_bug.cgi?id=338009 - Wish: Quick add to custom palette

It could be great to be able to add a colour to a custom palette, anytime. if I see it I can bookmark it. Maybe using a keyboard shortcut? For now it is painful to create a custom palette not just at the beginning of a project but while working on it, anytime. Let me know what you think.

  1. https://bugs.kde.org/show_bug.cgi?id=330329 - Palette docker: ability to rearrange colors

Michel:
Right now, it's not possible to change the sorting of colors in a palette directly from the UI. I wish it was possible to drag and drop colors freely around the palette grid.
Maybe even select multiple colors and then drag+drop them while keeping the same alignment.

I'm not sure how to solve the problem of dropping colors in place of other colors, would they get pushed, displaced, switched, overwritten?
Ghevan: *comments that back then palette retaining/making wasn't even possible*
Michel: *comments that making/retaining palettes is indeed implicit*
Guruguru:
The ability to rearrange colors is a request I see from Japanese users form time to time. (another request is the ability to add "name" property for each pallet color - such as "skin 1" etc)
They hope to arrange colors specifically for their use, not by color properties - so sorting option would be a short-term compromise, but not a solution.

  1. https://bugs.kde.org/show_bug.cgi?id=330893 - Make the Krita color palette docker start with a default palette

After Krita is installed, the color palette docker will start out empty.
I think it would be a good idea to make it have a default palette, even a simple one, to increase the discoverability and first time usability of this docker. Since by default it shows an empty gray space, it's not immediately clear that it indeed is a color [swatch] palette docker, when enabled.

This isn't done yet?

  1. https://bugs.kde.org/show_bug.cgi?id=335200 - Suggested color palette docker UI improvements: searching, visual improvements, organization

I'm using the color palette docker quite extensively, and list here a few UI tweaks that I think could improve its UI. These suggestions do not include the creation of new palettes, since I assume that's a more difficult issue that should be in its own wishlist report.

  • When a palette is selected, make swatches a bit larger by default. You can already make them larger/smaller with ctrl+scroll, but I think the default could be a good deal bigger, to prevent misclicks.
  • When browsing the list of palettes, make swatches a bit larger by default. They are even smaller, and ctrl+scroll does not work here.
  • Make the list of palettes searchable -- when having many palettes, it'd be nice to be able to filter palettes by name like you're able to with brushes. [DONE]
  • Make the list of palettes organizable: being able to add subfolders to organize things would be a nice feature to have. When working on different paintings or even projects, you sometimes need different sets of palettes to work with. This may not really be necessary if palettes are searchable (because you can prefix and then filter by that prefix) but I think it'd still be cool to have. Right now I just keep switching palettes in and out of the folder manually. {DONE, palettes have tags]
  • If possible, make krita automatically monitor the palette folder(s), so that when a palette is changed, it is automatically updated in the UI, and new palettes automatically appear in the list. Right now I have to (as far as I know) restart krita every time I add new palettes. An alternative could be to simply add a "reload" button to the list of palettes.
  1. https://bugs.kde.org/show_bug.cgi?id=339345 - Palette Colors in Krita look different than they actually are

This is a complicated issue where for some inane reason the photoshop devs decided that it's a-okay to have cmyk swatches without a colorspace. Krita always treats colors as if they have a colorspace, so getting around this will be tricky.

woltherav added a subscriber: scottpetrovic.EditedOct 31 2016, 6:29 PM

Palettes (from discussion during halloween)

There’s 3 areas affected here:

  1. The native file format and internal data model.
  2. Import/Export of non-native formats.
  3. UI

The native file format and internal datamodel.

To some extent, the native file-format is an expression of the data-model, but in Krita we try to avoid data-dumps, so not quite that extreme.

  • We chose for a zip file, with a colors.xml, and the referenced icc-profiles.
    • Outside of LCMS ‘built-in’ profiles all profiles should be stored and retrieved from the zip.
    • The XML stores the name, version, columns and comment and then has a child element ‘ColorSetEntry’ which is then used to store the data from the KoColorSetEntry. These are:
      • ID - string number unique to this color. - This is very common in palettes out in the wild, especially ones with spot colors where the ID is a unique code like 'OY348' and the name is some flowery nonsense like 'naive peachy pink'
      • Name - String with human readable name
      • Spot - bool for marking a color as spot, probably useful later, if not useful for python plugins.
      • bitDepth - a string that can be parsed to a KoID
      • Color - An XML which can be interpreted by KoColor::fromXML();
  • This model should be extended:
    • We want to be able to have groups. These colorgroups CANNOT be nested. The idea is that users want to group certain colors to find them more easily. Such things exist in other formats as well as ‘pages’ or ‘groups’.
      • Within the datamodel, we will have one QVector<KoColorSetEntry> colors for non-grouped colors, ALWAYS. And a QMap with an group-identifier of sorts and the QVector<KoColorSetEntry> colors for that identifier. [done]
        • Make sure we can have a palette where the colors are ONLY grouped, or a palette file where none of the colors are grouped. [done]
      • Within the format this becomes a hierarchy. [done]
        • Figure out a method to error with group children in a group.
  • We want to allow saving a KPL file to a kra by default. This should have a special spot in the UI.

Import/Export of non-native formats.

  • Ideally, KoColorSet is refactored to have the import export seperated into plugins.
  • We also want to allow python import/export plugins.
  • Exporting to a file would be done via a seperate command. Krita should not be saving out to a non-native or non-gpl file while not prompted.
  • We will not be handling localised names internally for now. We have no method to do anything with a list of names and language locales.
  • When dealing with multiple definitions, we will go for the most precise one first and then go off the list.

Usability of the Palette Docker:

  • We need a better new palette dialog. The current one is implemented a bit lazily :p
  • Add/Remove colors.
  • Rename Colors
  • Drag and Drop reorganisation of Colors.
    • There’s also a QDnD action we can use here, maybe? (Though this would only be for QColors)
  • Groups - As pointed out above we want groups for...
    • Linking a group-color view to a layer. (Ex: Layer Character1 should show the group ‘character_one_colors’ when selected)
      • We need an ‘all colors’ view to default to. - Users will otherwise get confused.
    • Layer Split - Using the groups instead of individual colors.
    • Generate palette from image, using each layer to fill a separate group.
  • Select ‘closest’ color to the current foreground color when this is changed to give an idea which is the closest color to current.
  • Palette docker should default to the KRA internal palette. This palette is auto-generated with a Black and White patch. Templates will be able to save palettes internally.
  • Swatch size, multiple options:
    • Fill out according to column size(current)
    • Size defaulted according to Resolution.
    • Large Swatch Size + Column-based outlines has the issue of swatches disappearing from view.
  • Allow python plug-ins for manipulating the palette.
  • We need a dialog for editing the palette meta-data (Name, Comment, Columns)
  • Probably allow the same for a swatch.

Other:

  • Append other palette to current - necessary for quickly having a existing palette added to the file-palette.
  • Search and autogenerate palette IDs vs. Palette Names.
  • Functions for appending and inserting color.
  • Internal image palette - Option to automatically fill it with color history.
  • Filter and Search options for the IDs and Names. -> These also need things in the KoColorSet.
  • ICC ColorLists will also need to be handled internally.

Todo:

  • @woltherav will be fiddling with KoColorSet and adding append/insert and similar functions.
    • She will keep overseeing @lsegovia 's parsing efforts.
  • @rempt will write python bindings for KoColor and KoColorSet so these can be manipulated, generated and read.
  • @rempt will write an initial palette docker based on the notes above. - It might be best to have something working before we do UI design to remove the more abstract elements from the conversation.
  • @rempt noted that the KisColorSetChooser.cpp (that is the little dropdown to choose the colorpalette) needs a serious make-over. Ask @scottpetrovic if he wants to look at this.
woltherav updated the task description. (Show Details)Nov 12 2016, 2:04 PM
Deevad added a subscriber: Deevad.Jan 2 2017, 5:19 PM

This is a pretty large list of requirements for this improved color palette. Has the scope of this work been decided after all this brainstorming (this is just a stretchgoal right?). If I do a UI for this, should I include everything listed. Obviously if I do the overall design will be more complex.

KisColorSetChooser.cpp definitely needs a redesign, as I've implemented group support, but there's no way for the user to select only a group within a palette. This is necessary for the layer-split dialogue for example.

@rempt was going to do a basic version of the palette docker first before we go and design that further, as there's certain questions that can only be answered by coding(swatch size and outlining is one of these).

I have been chugging away at making a new palette docker while staying at boud and irina's. When I was initially doing this in python, we realised it might be better to extend KisPaletteView(this widget is used in the palette docker and the lazybrush tool options amongst others).

I managed to add the following functionality past week:

  1. You will see groups/group names in the view. The index from id/ and id from index functions have been updated to deal with the new groupnames taking up space.
  2. You can rename groups.
  3. You can edit swatches.
  4. New swatches added with the palette docker will ask for a name and id.
  5. You can drag and drop swatches.
  6. You can drag and drop groups.
  7. The palettemodel now has a convenience function for providing entries at indices as well. I am wondering if I should move the add entry/add group/remove entry/remove group functionality to the model as well to make it more in line with how model/view programming should work.
  8. There's a basic ManagedColor python API now that allows people to deal with KoColor.
  9. There's a basic Palette python API as well, but this still needs to be extended with all the new shiny stuff.

Anyway, I just merged this. I will continue on working on extra stuff in my own branch, of course.

@woltherav I would like to request an additional feature which will also aid in vector object colors too. I request a shortcut for selecting background color by shift clicking on the swatches.
Currently when changing the foreground color through palette or the toolbar it changes the color of vector objects fill color, changing the background color from toolbar changes the stroke color. Adding this shortcut (Shift click to select background color) to palette docker will make it more intuitive and inline with what inkscape does, will be helpful for user due to familiarity. I have also added a bug report for this here-> bug 381789

woltherav updated the task description. (Show Details)Jul 16 2017, 12:34 PM

Added rightclick selection of background color based on the fact that is what we use elsewhere to select the background color.

Okay, I am running out of things to do to the paletteview... My next step would be to make the widget accessible to python, but I am not sure how to do about that? Like, I could wrap it in sip, but maybe it's better to make a LibKis class, and have the palettemodel and view abstracted away in that?

woltherav closed this task as Resolved.Mar 26 2018, 1:01 PM

lets resolve this for now.