Timeline UI and interactions
Closed, ResolvedPublic

Description

UI and interactions with the new timeline can be discussed here

mardelle created this task.Jan 24 2017, 9:51 PM

Here there are some notes I took recently about odd timeline behaviours.
Just for eg: Kdenlive allows you to put an image (jpg, png, etc.) on the audio track and this is an evident no sense. If you are in strict mode you can't even drag it from the audio to a video tracks.

I believe that Kdenlive has to allow:
audio clips to be inserted in audio tracks only and images and video without audio embedded clips to be placed only in video tracks.
video with embedded audio clips to be placed in any kind of track, this both in mixed or strict mode, but when you put them in an audio track they behave just as they are audio only clips.

I believe that Kdenlive has not to allow:
video clips without audio to be placed or moved into audio tracks.
the audio part of a video splitted clip in strict mode to follow the video in video tracks when you drag the video part on the upper tracks.
to drag the video part of a clip into an audio track.

One very useful feature Kdenlive already has is the option to switch an audio/video clip between only audio, only video and audio video embedded by the clip menu.This is very useful to recover lost syncs (I remind that I proposed to indicate on the clips the number of frames when you loose the sync but this feature could be enough for fixing a desync issue) I think this is a strenght point Kdenlive has.

Thank you for these points, this is useful.
One related thing is about clip insertion. Right now there is an option to split audio when inserting a clip, bu t as mentionned by Massimo, this is not perfect since the audio is inserted in track n+1.
My suggestion would be to make audio splitting the new default option (as it is (some ?) commercial softwares). But then one question arises : in which audio track should the audio be inserted?
Let's say we have three video tracks and three audio tracks as follows:
v1
v2
v3
a1
a2
a3

The two possible options, in my opinion, are :

  • When inserting video in v1, insert audio in a1, for v2 choose a2 and for v3 choose a3
  • When inserting video in v1, insert audio in a3, for v2 choose a2 and for v3 choose a1

I've seen the second option used in Davinci for example, and it makes sense since it allows the audio to be just below the video if you insert in v3. What do you guys think about this?

About other features: Massimo mentionned

  • Advanced trimming features
  • Single track transitions
  • Multicam Editing
  • More sequences nesting

For trimming it should only be a combination of atomic operations (move and resize).
For single track transitions, I already have ideas about how to implement that.
For Multicam and nesting, could we define more precisely what we want to achieve ? It's a bit foggy for me right now… I have no idea what a multicam workflow in a professional software looks like, and I don't know either what is supposed to be achieved with nesting.

Hi Nicolas,
About track insertion:
if you use the strict mode, Kdenlive put the audio and the video track on the set target tracks. If you use the keyboard it is pretty natural to set where to put audio and video if you drag and drop you ask for something more automatic. So, now you are free to drop the video clip where you want but the audio is splitted on the track set as target. If you want to do what you wrote you have just to change the target for the audio. If it's possible to do have a different behaviour (the one you described) when you use the drag and drop approach I guess it's pretty good. But remember that, as the keyboard approach is faster, the majority of professionals use it. So create the new drag and drop behaviour but don't remove the target approach for keyboard.

About trimming:
all the 4 trimming features are very important. As you are using Davinci Resolve you can check for them and you'll find all 4: rolling, ripple, slip and slide (I used Davinci Resolve recently and I found all of them)
So, just a simple move and resize it's not enough. If you just wrote this referring to the development side don't take in consideration what I just said. Anyway, for taking more information about these features I remind you to read my document about it: https://kdenlive.org/video-editing-applications-handbook/#trim
About multicam you can read this: https://kdenlive.org/video-editing-applications-handbook/#multi
About nesting you can read this: https://kdenlive.org/video-editing-applications-handbook/#nesting
You can also check how multiple timelines nesting and multicam are working in Davinci Resolve.

NOTE: I just checked what I wrote in my document more than one year ago and I have to say that I wrote it as the reader was someone that already know what multicam is. If you need more details please let me know and I'll find some videos and I'll explain it better.
frdbr added a comment.Jan 30 2017, 8:31 PM

An example of how nested clips work in Blender.

-Select clips and press Ctrl + G
-To ungroups press Alt + G

Notice how you can create groups within groups.

frdbr added a comment.Jan 30 2017, 8:40 PM

Example of clip snapping in Blender.

Here you can select a whole clip or just one of the tips and press Shift + s and the selection will snap to the cursor. An improvment of this would be to also be able to do this actions without the use of the mouse.

frdbr added a comment.Jan 30 2017, 8:47 PM

For single track transitions, I already have ideas about how to implement that.

One important feature in case you haven't considered it is the ability to improvise this action to give us creative freedom. As in I select one clip and drag it to another to create the transition, but then I want to do some fine adjustments by moving it more to the left or the right or a situation where I didn't like the result and then I "remove the transition" by dragging one clip away from the other. In those cases it would be great to have the original size of the clips preserved. (In shotcut last I tested if you remove the trasition the clips get reduced in size.)

I hope I was clear, otherwise I will try to improve the explanation, please let me know.

HI to Farid and to all friends.

When I speak about nesting I don't want to mean grouped or ungrouped clips but the ability to have more timelines in the same project.
These timelines are collected in the project window as they are normal clips. They are generally indicated with a different icon but they behave like real clips.
You can drag them inside another timeline, trim them and so on.
If you open them in the timeline window you can edit them and when you go back in one timeline that uses one of them you find the changes you did. https://kdenlive.org/video-editing-applications-handbook/#nesting
Of course a nesting for collapse and uncollapse part of the timeline, as you can do in AVID, could be interesting. But this feature is not so common as the timelines nesting which you can find in all major programs: AVID, Premire, FCP, Lightworks, Davinci Resolve, etc.
What I think is that the feature for nesting timelines is mandatory the one for nesting clips is just a plus. To have both could be, of course GREAT! I just worte it because I would like to avoid confusion between the two different features.

The Blender snapping feature is a great tool. I relally like this idea. This is a very good proposal!

For which concern transitions i don't believe we have to create transitions when when we drag a clip on another one.
We have already three modes for working: in normal mode we can snap the clips to the edges. In overwrite we can overlap them. In Insert mode we can drag for inserting.
What Farid proposes has some issues: for eg if you decide to remove a transition in a later moment instead of just remove it by selecting and delete it you have to move the clip forward but probably the timeline in front of this clip will be no more empty and you probably need to change the lenght of it.
The way all the programs propose today come from a long story of trials. No software during the 90's has one track transitions. The first software who introduced this concept was AVID.
The standard behaviour you can find today in all application is:
you move the editline on the middle of a cut and by a keyboard shortcut you can add the default transition with the default lenght that you can set in the program properties.
If you desire another type of transition you can drag and drop it on the cut in the timeline.
The panel for fine tuning the transition as I shown here
https://kdenlive.org/video-editing-applications-handbook/#trans
allows you to decide if you want the transition at the center of the cut, at the beginning of the cut or at the end of the cut. Allows you to change the lenght and to refine other parameters as softness, borders, directions and so on.
You can also drag it on the timeline by moving the cut by the advanced trimming tools (this is very useful for creating fast automations for audio mixing)
My opinion is that if we want to create an application for professionals which is intuitive and simple to use is always better to use the best practise we can found around and that people have habit to use.
I also think that the transition on two tracks that Kdenlive has today must be preserved as blending tool because is a very powerful compositing feature.

Thanks for all the feedback. Here are some questions/remarks

  • For grouping : I am almost done implementing a grouping feature that looks like what we had in the previous version. Basically, you link clip together, so that when you move one, the others move accordingly. Groups of groups are possible.
  • For nesting: Nesting Timeline should be rather straightforward, it is an extension of what we currently call library. We just need to think about the UI to allow the user to switch between the "main" timeline and one of the nested timeline. Nesting Clips (which I assume is what is shown on Farid's video in blender) might be a bit trickier, I need to give it a thought to see what can be done.
  • For snapping: First question, what do we snap to ? Clip boundaries, cursor position ? Anything else ? And should all snap points have the same attractiveness ? (for example you might be more attracted to the cursor, or to something else…). A shortcut to force snapping seems a good idea. Do we keep Blender's idea and snap to the cursor only, or do we make this an action "snap me to the closest snapoint"
  • For same track transitions: I agree with Massimo's point that we should follow the industry standard for basic features like that. I just have troubles to understand how it works :p Let's say you have to clips of length = 10 frames. You insert them consecutively on the timeline (hence the total is 20 frames). Now you try to add a transition in between. How can it work without changing the total duration ?

By hoping to be helpful I drop new feedbacks and answers to your questions:

Grouping. the behaoviuor Kdenlive has at now for me is ok. I remarked what I find illogical as for eg. plain video clips and pictures in the tracks. (Read all my opinions about it in my post date jan)
https://phabricator.kde.org/T5174#77402

Nesting: What I can say about the user experience is: generally the timelines or sequences (the name changes from one software to another) are stored in the project library together with video and audio clips, titles, images and all the materials you are editing. You can arrange them or move into bins as you wish. You can also open them in the clip monitor and play. When you double click on them from the project window you can have these two behaviours depending on the software: in AVID, Lightworks and Davinci Resolve you just switch the content of the timeline window, as you can do when you open a new clip in the clip monitor.
In Premiere you open a new tab in the timeline window like you do in an internet browser and you can switch from one of this timeline tab to another, close, drag, and rearrange them for having, for eg, 2 of them side by side.
https://kdenlive.org/video-editing-applications-handbook/#nesting

About the nested clip I suggested, in the past, to use this concept for creating collapsed clips. Today is already possible to create virtual clips by using the library widget, to import them in another project and to uncollapse them for editing.

Would be wonderful if both timelines/sequences and collapsed clips were able to keep, once inserted in other timelines or moved around, the peview rendering.

Snapping: about snapping I have not my ideas too clear. I liked the video Farid posted but actually I rarely needed to use a feature like that one. I guess that an editline which changes color when the dragged clips are alligned to it could be good. What I mostly miss is probably a ripple delete on the right click when I open the contextual menu in one empty space in the timeline, even if I can set an in a out point and I can extract the empty portion.

Transtion: the approach is pretty simple. If you put 2 clip for a total lenght of 20 frames and your default lenght is superior, your transition will be long just 20 frames if it'll be placed on the center of the cut.
The important thing is that the following clip has frames before the cut and the previous clip has frames after the cut. If we put on the timeline a clip which start from the first frame we'll be able just to insert a transition which start at the end of the cut.
So: we take 2 trimmed clips (which have more frames before and after the cuts) we put them on the timeline side by side. If we put a transition in the middle of the cut Kdenlive will use frames which are after the cut of the previous clip and frames that are before the head cut of the following clip. Then, if we decide that the transition is placed before the cut: the program will cross the latest frames before the cut of the first clip and the same number of frames of the following clip placed before the cut.
Or after the cut: we'll use the frames which are after the cut of the first clip crossed with the same number of frames placed after the cut of the following clip.
By the mouse we can drag the transition around for refining the position, shrink it or make it longer or editing the cut by the trimming feature by keeping the transition in position.
In the transition setting window we have the traditional view on 2 tracks and we can refine the position.

I believe that the images I inserted in my document are explicit enough:
https://kdenlive.org/video-editing-applications-handbook/#trans
This is a video about Premiere pro which could be an introduction even if very incomplete:
https://www.youtube.com/watch?v=xq2nk5p4EU0
This ne is about AVID. It's always partial (about to save transitions presets) but it could be useful too:
https://www.youtube.com/watch?v=gfbowDOFBjo

I hope all this can help you to set the right mind set.
Every other question is well accepted. I hope to help you the way you need with this.

frdbr added a comment.EditedFeb 5 2017, 4:50 PM
  • For snapping: First question, what do we snap to ? Clip boundaries, cursor position ? Anything else ? And should all snap points have the same attractiveness ? (for example you might be more attracted to the cursor, or to something else…). A shortcut to force snapping seems a good idea. Do we keep Blender's idea and snap to the cursor only, or do we make this an action "snap me to the closest snapoint"

If you can spare 15minutes have a look at this video, it has good visual examples, better than my explanations by writing.

  • For same track transitions: I agree with Massimo's point that we should follow the industry standard for basic features like that. I just have troubles to understand how it works :p Let's say you have to clips of length = 10 frames. You insert them consecutively on the timeline (hence the total is 20 frames). Now you try to add a transition in between. How can it work without changing the total duration ?

I am still trying to understand this concept since I haven't used proprietary editing tools since 2004. Here are some examples:

Final Cut:

Adobe Premiere:

frdbr added a comment.Feb 6 2017, 5:51 PM

Undestanding Grouping, Nesting and Compound clips.

The previous video example about grouping clips in Blender I realized that other editors call this concept as compound clips.

While Nesting is the ability to have various timelines as tabs.


PS
Am I posting too much stuff that is confusing rather than helping out? Please let me know. Thanks.

Thanks Farid for your support and to look for videos: I know it's a very time consuming task.
For helping about transitions I create a very short video with all the useful info. I hope it can help to save time in understanding the approach.
In the video I didn't talk about the shortcuts for inserting a default transition on the timeline in the cut where the edit line is: in Premiere Pro they are: Shift+D for video transition, Ctrl+Shift+D for audio cross dissolve.

I posted this in T5638 but putting it here since it is more appropiate:

Check out this cool functionality that Blender has making the workflow faster. Notice how with a single click and drag of the mouse you can turn on/off the icons.

Also the headers could have a functionality to easily switch their position:

I am also willing to help design some UI/UX stuff.

What do you guys think?

Hi everybody, it's been a while since I didn't bother you with UI questions =)
I'm currently working on the compositions (aka "several tracks transitions"). Here are some questions:

  • Should we check for collisions between the compositions? To put it differently, could we have a situation with two overlapping compositions ? Note that the b_track for both might be different.
  • How to deal with snapping ? I'm assuming that compositions should snap to existing snap-points (clips in/out for example), but should they CREATE snap-points (ie do we want to snap things to compositions in/out) ?
  • How to deal with groups ? Should compositions be allowed to be part of groups ? Do we require that there is at least one Clip in the group ?
  • Do we try to have a better representation of compositions for which the b_track is not the one just below the a_track ? Currently, (as in previous versions) a composition is a rectangle overlapping between the track and the next one, but if the b_track is not the next one, that doesn't really make sense, does it ?

Hoping to hear back from you on that !

As for Farid's suggestions, the track reordering would be indeed interesting, but I'm a bit less sure about the click and drag stuff to turn on/off icons. We'll see if it is easy to implement, but now it is not top priority :)
Cheers

Hi,

  • Overlapping composition is possible in MLT, but might lead to unexpected result. Until now we did not allow it, so I would be in favor of prohibiting it.
  • Composition / snapping: I would say yes to creating snap on composition. You sometimes may want to start a clip as soon as a transition ends
  • Groups: a user might want to move several compositions 10 seconds later. So in my opinion, there should be no difference / limitation between clips and compositions regarding grouping
  • Currently in the new QML timeline, compositons are not overlapping tracks, but are on their track, with height being half of the track height. I think the overlapping we had previously was not that good and would be in favor of a UI hint of the real transition track. Maybe some kind of transparent overlay on the destination track..
frdbr added a comment.Mar 27 2017, 8:22 PM

Just a thought about groups to further the discussion. I think down the line we can groups tracks as well that way we can reorder groups of tracks. A usecase of this is when lets say we create a group of music tracks and a group of audio fx tracks and then we can move them up or down the timeline for adjustments with the video tracks.

frdbr added a comment.Apr 16 2017, 3:44 AM

A suggestion for you to consider whenever you see it fit. It would be great to have an option to split the stereo channels of an audio track into two mono tracks. This is handy when sometimes we record from the external mic and the camera's internal mic at the same time and then while editing need to use, remove or apply effect to only on one of the stereo channels. I can make a mockup if needed.

NB
Currently the split audio separates the audio from the video tracks but both channels remain on the same track.

frdbr moved this task from Feature ideas to Refactoring on the Kdenlive board.Nov 12 2017, 7:01 PM
jlskuz closed this task as Resolved.Nov 12 2023, 2:12 PM
jlskuz claimed this task.