Storyboard feature
Open, Needs TriagePublic

Description

A Storyboard feature would offer an alternative workflow, focused more on the story and the flow of story. This feature would make Krita more standalone animation program.

Why is it needed?
Storyboarding is an important pre-processing step for animation. It helps get a feel for the story and create a better flowing animation. So it basically helps in planning the animation or scene beforehand.

Storyboard requirements

General

  • Storyboard is stored as a single .kra file
  • Every stroyboard item is a non-empty frame on the image's timeline
  • Duration of a storyboard item is a number of hold frames after the storyboarditem's frame
  • Storyboard items are static images, they don't support internal animation

Storyboard docker

  • Shows thumbnails for every non-empty frame of the image
  • Shows selected metadata for each frame, e.g. dialogs, actions, comments and etc
  • The fields in metadata should be configured in per-document basis
  • The frames can be reordered with D&D operation.
  • The duration of the frame can be configured easily (modifies the amount of hold frames on the timeline)
  • The user can paint on the thumbnail in the docker. When painting on the thumbnail, the user paints directly on the image's frame via mechanism of KisScratchPad.
  • The frames of the image may be "pinned" to the storyboard, or "unpinned" from it

Import/Export features

  • The user should be able to export the storyboard into PDF/SVG
  • The layout of exported document should be configured somehow (use SVG for that?) (optional) The docker may have a special text view from where the user may D&D text into metadata fields.

Future ideas

  • Shift drag changes duration. (@Bollebib )
  • Locking the storyboard to avoid adding changed frames from timeline to storyboard. (@vanyossi)
  • Export every storyboard item as individal kra file. (@vanyossi)
  • Add transition markers between frames. (@vanyossi )
  • Different sized frames. (@vanyossi
  • Grouping frames into one storyboard item. ( @vanyossi )

Link to a draft_proposal here

Some links:
https://en.wikipedia.org/wiki/Storyboard

https://www.tvpaint.com/v2/content/article/feature/storyboard.php?lang=en

confifu created this task.Mar 14 2020, 5:07 AM
confifu updated the task description. (Show Details)Mar 14 2020, 5:27 AM
confifu updated the task description. (Show Details)Mar 14 2020, 7:07 AM

this is interesting but is a huge task tho. This would need lotsof mockups first...

confifu updated the task description. (Show Details)Mar 15 2020, 12:16 PM
confifu updated the task description. (Show Details)Mar 15 2020, 12:18 PM
confifu updated the task description. (Show Details)Mar 15 2020, 2:02 PM
confifu updated the task description. (Show Details)Mar 15 2020, 2:26 PM
confifu updated the task description. (Show Details)Mar 16 2020, 12:55 PM

@Bollebib Should we make the storyboard docker independent of timeline docker. That way you can add any image on the canvas or even .kra files. For "playing the storyboard" we can have a duration associated with each image in storyboard and play it from storyboard itself.

If I was doing this, I would probably make the storyboard docker a separate docker than the timeline and animation docker. Under the hood it would still use the existing animation API, but you wouldn't need those other dockers to do storyboarding. The reasoning for me is that storyboarding is a separate part in the workflow, and I think the decisions that people want to make are different. The few times I have storyboarded, this was the workflow I have used.

  1. I start with a script with only text. I like to start with planning out the frames from the script.
    1. I add descriptions and dialog to each frame like "The girl lunges forward while giving a sideways glance at her pet snake".
    2. I can figure out about how many frames I will need at this point
  2. going back through each frame and drawing the picture
  3. determine how long each frame will last in seconds (or frames) to determine pacing
  4. reorder or insert frames in the middle if certain areas need more description or dialog

I really like the idea of different views. At the beginning, seeing all the text is nice. When you are in the middle of drawing, though, you are more concerned with only the image you are currently drawing, so seeing all the other text is kind of distracting.

From a technical standpoint, we might have to extend the animation API a bit. Say if I make some storyboards and save a KRA. I would expect loading the KRA to preserve all my storyboard data. We might have to extend the animation system to support extra meta-data property like comments. We would also have to figure out how the KRA file would know there is a storyboard data. If I have 3 layers in my file for example, how is Krita going to know which one to load in the storyboard docker?

@scottpetrovic I never/rarely use text , so it should also work as a purely visual tool.

@confifu as a first glance the mockup looks okay.

number 8 seems a bit weird though. Having a visibility button would mean the comment section will dissappear when you click it no? wouldnt it be easier to have a global state for all comment types? Or do you actually need to control every single image state seperatly? that seems very granular....

number 8 seems a bit weird though. Having a visibility button would mean the comment section will dissappear when you click it no?

4,5 and 6 , i.e. the comment combobox will remain visible even when no text fields are visible. It can be used to toggle visibility of text field or create new text field.

wouldnt it be easier to have a global state for all comment types? Or do you actually need to control every single image state seperatly? that seems very granular....

We have the view option to hide all text fields. Not all scenes will have same components. Say some action scene will have no dialogue. Whereas a dialogue scene might have no action. So a universal button would not be good in my opinion.

Should we get rid of the hide option for individual text fields? Would there be any use case for individual hide buttons?

@scottpetrovic We can add a QList of storyboard structs to KisDocument. Changing kra_saver and kra_loader might be needed for getting that data to and from kra file and KisDocument. I think we wouldn't need the animation API. Would the animation API have any advantage over saving it explicitly as a member of KisDocument ?

@Bollebib @timotheegiet Is saving a .kra file for each storyboard image and loading them from those saved files okay, or we should save those images inside a single .kra file as storyboard images.

a single .Kra seems more manageable

confifu updated the task description. (Show Details)Mar 23 2020, 4:30 PM
confifu added a subscriber: rempt.EditedMar 23 2020, 4:33 PM

Options suggested for saving and loading data by @rempt on IRC on 22 March 2020

Option 1: Every board is a .kra file and can have everything a .kra file can have, including animation. A metafile describes the storyboard. The docker would load the xml file and thumbnails, but only the active board is loaded in krita
Option 2: the .kra file contains the board as separate images. You’d save the .kra file with multiple images inside. The docker would load the kra file and find the boards. All boards are loaded into memory.
Option 3: the .kra file contains layer groups, and every top-level group is a board. The docker would list the top level layer groups, and when a board is selected use the isolate layer feature on that group to show the board.
Option 4: Reusing the technique the compositions docker uses. That would make it possible for different boards in a storyboard to share elements, like a background layer. Basically, every board then consists of one or more layers from a single Krita image.

Conversation with @Bollebib on IRC on 16 March 2020

realistically tho most of the time i just work on the timeline and make my storyboard
what i had to do then is just create an exportable file in tvpaint so i had to export it
i think if krita can just handle adding and removing thumbnails for storyboard easily or if you can replace thumbnails easily that should be fine
so i'm not even sure we need autoupdating thumbnails
if there is an ID system you could always have a manuel REFRESH button at most
this is probably more manageable and easier to do

confifu added a comment.EditedMar 23 2020, 4:49 PM

Conversation with tiar on IRC on 16 March 2020

We ruled out option 1 and option 2 as they were autoupdating the thumbnail in storyboard

option 3): storyboard images are new images. They can be imported/copied from the Timeline, from specific layer, from the preview, from a file. You can paint on them ( we'll need to think how to technically solve this). They don't reference anything (even if they were copied from somewhere) so they don't update on their own.

option 4): storyboard images are imported from outside like File layers and can be only changed in that other file. ( I guess it could be together with option 3. for a animator's choice).

listening to you all I start to think that option 1) and 2) are not necessary because you want to put images from the timeline to the storboard, but you don't want it to update on it's own, which means copies, not references

I tried to sum up what I suggested on IRC today:

Storyboard requirements

General

  1. Storyboard is stored as a single .kra file
  2. Every stroyboard item is a non-empty frame on the image's timeline
  3. Duration of a storyboard item is a number of hold frames after the storyboarditem's frame
  4. Storyboard items are static images, they don't support internal animation

Storyboard docker

  1. Shows thumbnails for every non-empty frame of the image
  2. Shows selected metadata for each frame, e.g. dialogs, actions, comments and etc
  3. The fields in metadata should be configured in per-document basis
  4. The frames can be reordered with D&D operation.
  5. The duration of the frame can be configured easily (modifies the amount of hold frames on the timeline)
  6. The user can paint on the thumbnail in the docker. When painting on the thumbnail, the user paints directly on the image's frame via mechanism of KisScratchPad.
  7. (optional) The frames of the image may be "pinned" to the storyboard, or "unpinned" from it

Import/Export features

  1. The user should be able to export the storyboard into PDF/SVG
  2. The layout of exported document should be configured somehow (use SVG for that?)
  3. (optional) The docker may have a special text view from where the user may D&D text into metadata fields.

This is perfect @dkazakov. Thanks for putting these functional requirements together.

@confifu, @dkazakov, @Bollebib @timotheegiet, @Deevad - I also came up with an idea for the storyboard docker for the wireframes.

I took what confifu did and tried to simplify the user interactions a bit. I think because of how much metadata we want to show with the different text and duration, it is going to be cleaner to make that editable areas in a separate dialog. There are probably some additional shortcuts we can do for things like duration to change that faster, but hopefully the comments I wrote on the wireframes make sense with the general direction

https://phabricator.kde.org/M171

@scottpetrovic We might also need to add and remove metadata fields.

The fields in metadata should be configured in per-document basis

Hi, @confifu!

The layout of exported document should be configured somehow (use SVG for that?)

In this requirement I meant something like "Exporting to PDF" section of your proposal states. Though I guess we still have two options here. I guess we should ask @Deevad for an opinion:

  1. Option 1: make a dialog with preconfigured layout options like you suggest in the proposal. This approach is nice because it it has very low user-experience entry level.

  1. Option 2: let the user create an SVG file (e.g. in Inkscape) with empty boxes. These boxes may have some predefined IDd, like "frame1", "frame2"... and "title1", "title2". Krita could read this SVG and fill it with actual data. Later on, this SVG could be exported either to PDF or raw SVG.

I have a feeling that "Option 1" would be better for 90% of the users. At the same time, "Option 2" is a bit more configurable. I'd like to head @Deevad's opinion on that :)

I have two variations of the storyboard docker with UI directions. The first example is closer to the direction on how something like Storyboarder does it. The second one is a direction we could go based off how Storyboard Pro does it.

  1. https://phabricator.kde.org/M171/610/
    1. This direction has the docker as more of a side docker for quickly moving around keyframes. You would still use the main canvas to do all your drawing/painting. It takes up less space than the second design, but is more cumbersome when updating a lot of text
  2. https://phabricator.kde.org/M171/611/
    1. This direction kind of replaces the canvas. It would take up a lot more space and you would paint directly on the frame.

@dkazakov, @scottpetrovic, @Bollebib @timotheegiet, @Deevad Added a new mockup here

  • Added a comments combobox to add and delete metadata fields on a per-document basis.
  • Added a lock button to toggle edibility of image from storyboard docker.
  • Added export widget to manage layout of exported document. Both option 1( preconfigured layout) and option 2(specify using SVG file) would be offered. If user opts for option 2 they can chose SVG file.
  • The duration field is seconds + frame. This would be similar to minute / second format.
Bollebib added a comment.EditedMar 25 2020, 3:36 PM

-Export as PNG for different pages should also be possible, not only PDF's.
-Rightclicking a thumbnail maybe should provide the option to open the image (or a range of images) as a new krita file so you can start expanding the scene or animating. Set the size of the project and import the thumbnails into it. (wish)

@scottpetrovic looks pretty good, the seconds counter is a nice touch. Maybe shift dragging that area could make it longer or shorter? That is an efficient and fast way to do it

Having multiple frames selected and shift dragging would also set all of those frames to the chosen number.

confifu updated the task description. (Show Details)Mar 27 2020, 3:03 PM
confifu updated the task description. (Show Details)Mar 28 2020, 10:53 AM
confifu added a subscriber: vanyossi.
confifu updated the task description. (Show Details)Mar 28 2020, 10:56 AM
confifu updated the task description. (Show Details)Mar 28 2020, 11:02 AM
confifu updated the task description. (Show Details)Mar 28 2020, 11:06 AM
confifu updated the task description. (Show Details)Mar 28 2020, 11:26 AM