Global Kinetic Scrolling
ClosedPublic

Authored by eoinoneill on Sep 25 2018, 10:15 PM.

Details

Reviewers
rempt
emmetoneill
Group Reviewers
Krita
Summary

This patch expands on Krita's current 'Kinetic Scrolling' option originally added in the D8015 patch submitted by @poke1024. Co-Authored by Emmet.

This patch takes the concept of Kinetic Scrolling and applies it uniformly across Krita's user interface, where applicable. To do so, we've created a new KisKineticScrolling namespace that encompasses most of the behavior of kinetic scrolling in a single place, which allows for adding consistent kinetic scrolling behavior to any UI element with just a few function calls. Using the new namespace we were able to easily add kinetic scrolling to all of Krita's scrollable user interface elements including dockers, the toolbox, the brush preset selector, palettes, animation timeline, etc. Because all of these elements share the same code path, they all use the same user-configured settings. The namespace also has a couple of utility functions, including a cursor updater to unify the visual feedback.

We've also taken the time to make some small changes to the kinetic scrolling options (Configure Krita > General > Tools > Kinetic Scrolling). Now all of the kinetic scrolling options are inside a checkable QGroupBox which is used to enable or disable the feature. As such, we have removed 'disabled' from the dropdown menu, which is used to configure which input is used to scroll. We've replaced the sensitivity combobox with a KisSliderSpinBox (from 0 to 100). We've changed "Show Scrollbars" to "Hide Scrollbars" (unchecked by default).

Finally, we've added an option to use middle click for kinetic scrolling and have made that the default for multiple reasons. The foremost reason being that middle click is already used for panning in Krita's canvas view and thus creates a uniform behavior across all of krita's ui. Secondly, middle click is basically unused throughout most of krita's UI while left and right click are used for other things. The user can bind middle click to their tablet pen. The main idea is that middle click and drag becomes a kind of standard way to "pan" krita's various UI elements which improves control and add coherency to Krita's ui.

Code Note: In order to expose the KisKineticScroller namespace to various parts of the code, I'm currently doing some ugly stuff with CMakeLists in order to get access to the namespace in other modules. If there's a better way to do this, let me know because I'm still relatively new to CMake.

Test Plan

Try various kinetic scrolling settings to make sure everything works consistently. Please try testing multiple scrollable widgets with middle clicks + drag to make sure most major UI elements feature kinetic scrolling.

Diff Detail

Repository
R37 Krita
Lint
Lint Skipped
Unit
Unit Tests Skipped
eoinoneill created this revision.Sep 25 2018, 10:15 PM
Restricted Application added a reviewer: Krita. · View Herald TranscriptSep 25 2018, 10:15 PM
Restricted Application edited projects, added Krita; removed Krita: Abyss. · View Herald Transcript
eoinoneill requested review of this revision.Sep 25 2018, 10:15 PM
emmetoneill resigned from this revision.EditedSep 26 2018, 12:30 AM

(Removing myself as a reviewer since I helped a bit with this one.)

Also, after applying the patch on my setup I found that Middle-Click + Drag wasn't set by default (although it should be!), but maybe it just inherited the settings that were already in my config?

This patch takes the concept of Kinetic Scrolling and applies it uniformly across Krita's user interface, where applicable. To do so, we've created a new KisKineticScrolling namespace that encompasses most of the behavior of kinetic scrolling in a single place, which allows for adding consistent kinetic scrolling behavior to any UI element with just a few function calls. Using the new namespace we were able to easily add kinetic scrolling to all of Krita's scrollable user interface elements including dockers, the toolbox, the brush preset selector, palettes, animation timeline, etc. Because all of these elements share the same code path, they all use the same user-configured settings. The namespace also has a couple of utility functions, including a cursor updater to unify the visual feedback.

You need to make sure it doesn't interfere with normal drag-and-drop functions.

We've changed "Show Scrollbars" to "Hide Scrollbars" (unchecked by default).

I can't quite agree with this change. I think there should either be
the scrollbar (or some alternative indicators) to give an indication
of how much content is there to be scrolled, perhaps except for the
toolbox because it doesn't seem natural there.

Moreover, if you start enabling this by default, there would probably
be a bunch of bug reports of people complaining the scrollbars are
gone because they never check the release notes or the options.

Finally, we've added an option to use middle click for kinetic scrolling and have made that the default for multiple reasons. The foremost reason being that middle click is already used for panning in Krita's canvas view and thus creates a uniform behavior across all of krita's ui. Secondly, middle click is basically unused throughout most of krita's UI while left and right click are used for other things. The user can bind middle click to their tablet pen. The main idea is that middle click and drag becomes a kind of standard way to "pan" krita's various UI elements which improves control and add coherency to Krita's ui.

Note that not all tablet pens have two configurable buttons,
especially those that use Windows Ink. Also, on the canvas one can
hold the spacebar to act as a replacement for the middle button. On
touchscreen devices, the user can also use two/three finger gestures
to pan the canvas.

I think the typical user would also expect single-finger touch drag to
scroll But from the experience with D8015 I don't think you can
configure multiple triggers for QScroller, can you?

Btw, for some dumb reasons, Windows Ink pen actions on Windows 10 ver.
1709 or later also sends touch events. This should also be taken into
account.

emmetoneill added a comment.EditedSep 26 2018, 5:07 AM

Hi Alvin,

You need to make sure it doesn't interfere with normal drag-and-drop functions.

We planned for this as much as possible, which is part of the reason behind using middle-click + drag as the default (the other major part being consistency with the canvas). However, left-click + drag does work in the vast majority of situations (though there are bound to be some bugs, which is part of what needs to be tested). Having said that, I think this has been designed in such a way that it would be easy to adapt to potential conflicts; if there is a conflict between a certain docker's functionality and the user's preferred scrolling input, it would be easy to disable kinetic scrolling for that particular combination or (uglier) to force a different button for that one docker.

So far in my own testing I haven't found any major show-stoppers. Let us know if you find any major conflicts and we can work on mitigating them.

I can't quite agree with this change. I think there should either be
the scrollbar (or some alternative indicators) to give an indication
of how much content is there to be scrolled, perhaps except for the
toolbox because it doesn't seem natural there.

Moreover, if you start enabling this by default, there would probably
be a bunch of bug reports of people complaining the scrollbars are
gone because they never check the release notes or the options.

This is a bit of a misunderstanding, I think.

Scroll bars are still enabled by default, as they probably should be.
It's merely that the checkbox says "Hide Scrollbars" now, and that users have to opt into hiding them.
It's purely a semantic change, the users opts into "hiding" the scrollbars instead of opting out of "showing" them.

Note that not all tablet pens have two configurable buttons, especially those that use Windows Ink.

Yeah, but can that really be helped...?

I don't have the means to test a wide variety of tablets here, but I'd imagine that if the user has a tablet with a somewhat limited number of buttons then, naturally, their ability to interact is going to be limited one way or another.
If anything that's more reason to have left-click + drag as well as middle-click + drag and touch + drag options;
the middle-click option is better for preventing clashes with other UI functions,
but if users can't spare the extra button for binding middle-click, they can use left-click + drag instead (which may eventually be slightly limited as we tune this feature to prevent functionality clashes).

In other words, users with limited inputs are going to have to make some choices, but I consider that pretty par-for-the-course.

I think the typical user would also expect single-finger touch drag to
scroll But from the experience with D8015 I don't think you can
configure multiple triggers for QScroller, can you?

That's right, to the best of my knowledge QScroller is still limited to a single input trigger per widget; left-click, middle-click, right-click, or touch.

Left-click is doable, though UI clashes are bound to show up and will need to be addressed. (It's included as an option, but probably shouldn't be default.)
Right-click is a non-starter, as it already has a well-understood global function for opening context menus. (It's excluded from the options)
Touch will need further testing from the people who have the hardware to do so. (But it was already in there due to D8015, so we kept it around)

And that really leaves middle-click as the most sensible default option, which together with canvas panning,
feels pretty natural in my opinion on both mouse and tablet pen and basically doesn't get in the way with anything.

Btw, for some dumb reasons, Windows Ink pen actions on Windows 10 ver.
1709 or later also sends touch events. This should also be taken into account.

Hmm... It's hard for me to know what effect that might have.
If you have the time, please try to test Windows Ink against the various input types (left-click/middle-click/touch).

rempt added a comment.Sep 26 2018, 8:30 AM

I'm not totally sure about this, and I agree with Windragon that we don't want to hide the scrollbars by default. But in any case, I've got two remarks about the code:

libs/ui/CMakeLists.txt
19

This shouldn't be necessary. The ui lib already links to the widgets library, so it should be able to use KisKineticScroller directly.

libs/widgets/KisKineticScroller.h
30

If you export the definitions, other libraries can use the functions directly.

This is a bit of a misunderstanding, I think.

Scroll bars are still enabled by default, as they probably should be.
It's merely that the checkbox says "Hide Scrollbars" now, and that users have to opt into hiding them.
It's purely a semantic change, the users opts into "hiding" the scrollbars instead of opting out of "showing" them.

Ah sorry, I misread the message.

Note that not all tablet pens have two configurable buttons, especially those that use Windows Ink.

Yeah, but can that really be helped...?

I don't have the means to test a wide variety of tablets here, but I'd imagine that if the user has a tablet with a somewhat limited number of buttons then, naturally, their ability to interact is going to be limited one way or another.
If anything that's more reason to have left-click + drag as well as middle-click + drag and touch + drag options;
the middle-click option is better for preventing clashes with other UI functions,
but if users can't spare the extra button for binding middle-click, they can use left-click + drag instead (which may eventually be slightly limited as we tune this feature to prevent functionality clashes).

In other words, users with limited inputs are going to have to make some choices, but I consider that pretty par-for-the-course.

Yes, the options for Windows Ink tablet users are limited. But it
looks like that Windows Ink tablets are getting more popular (thanks
to 2-in-1 laptops), so it seems to be a good idea to find a better way
to accommodate those users.

I think the typical user would also expect single-finger touch drag to
scroll But from the experience with D8015 I don't think you can
configure multiple triggers for QScroller, can you?

That's right, to the best of my knowledge QScroller is still limited to a single input trigger; left-click, middle-click, right-click, or touch.

Left-click is doable, though UI clashes are bound to show up and will need to be addressed. (It's included as an option, but probably shouldn't be default.)
Right-click is a non-starter, as it already has a well-understood global function for opening context menus. (It's excluded from the options)
Touch will need further testing from the people who have the hardware to do so. (But it was already in there due to D8015, so we kept it around)

And that really leaves middle-click as the most sensible default option, which together with canvas panning,
feels pretty natural in my opinion on both mouse and tablet pen and basically doesn't get in the way with anything.

Actually, only right-click press itself is for opening context menus
and some context-sensitive actions. I don't think right-click drag is
used for anything, or is it? Using right-click drag for scrolling is
not so implausible.

Due to how the API works, pen button on Windows Ink always maps to
right click (and if there is a second button it's usually mapped to
emulate an eraser tip). It's not (and doesn't make sense to be)
user-configurable. At this moment it might even make sense to consider
mapping right-click dragging to act like the middle button on the
canvas.

I don't disagree with making middle-click drag scrolling. IIRC Blender
does this too. But when considering tablet pens, we shouldn't be fixed
on mapping pen buttons to mouse buttons in a 1:1 manner.

The optimal default is to have multiple *sensible* ways for different
devices to trigger scrolling. If this is not possible, no amount of
options will help. One thing to consider though, is that it's
technically possible to distinguish between tablet, touch and mouse
events so it's possible to have different behaviour for different
devices. The limitation here is imposed by QScroller and the
underlying QFlickGestureRecognizer which is very unfortunate. If
only they are more flexible. Would it be worth it to fork it or roll
our own kinetic scrolling implementation? Perhaps not really. But then
a group of users would not be able to use this function in the optimal
way.

Btw, for some dumb reasons, Windows Ink pen actions on Windows 10 ver.
1709 or later also sends touch events. This should also be taken into account.

Hmm... It's hard for me to know what effect that might have.
If you have the time, please try to test Windows Ink against the various input types (left-click/middle-click/touch).

Unfortunately, I can't spare enough time to do this, but if @rempt can
make me a build with this patch I might give it a go, though it
wouldn't be any in-depth testing.

rempt added a comment.Sep 26 2018, 8:47 AM

I can try to make a build, but I'm a bit preoccupied with the 4.1.2 release today.

Alvin, is your issue that the middle mouse button is used by default? Because there are other scroll methods available already in the existing code(finger scroll and mouseclick scroll). I've been using the latter on my system.

I like this patch, I had wanted to do something like it. I do think the 'disabled' state should probably come back. Who knows what kind of weirdness people's input devices might have. Once we're sure that people have tested the feature and there's no major bugs, we can think of removing 'disabled' from the list.

Alvin, is your issue that the middle mouse button is used by default? Because there are other scroll methods available already in the existing code(finger scroll and mouseclick scroll). I've been using the latter on my system.

My concern is that the default should be to enable the gesture for
multiple input types (pen/touch/mouse) and with different "mouse
button" triggers for each. But I know it's too much to ask for with
the current QScroller implementation.

Since there can only be one trigger with this implementation, having
the middle mouse button the default seems to be an acceptable choice
given the reasons. I'm just slightly annoyed there isn't a better
option.

emmetoneill added a comment.EditedSep 26 2018, 10:46 PM

I'm not totally sure about this

Hi Boud. What about it is giving you pause?

I agree with Windragon that we don't want to hide the scrollbars by default.

Yeah, Eoin and I agree with that too, scrollbars should be shown by default.

The only proposed changes to the default settings are Kinetic Scrolling enabled by default and middle-click + drag for kinetic scrolling by default.
(And even then, personally I don't mind if KS is enabled by default, but I think it would be nice.)

We'll go ahead and make those code fixes when we get the chance.

I can try to make a build, but I'm a bit preoccupied with the 4.1.2 release today.

We're in no rush to get this patch integrated and Eoin and I are probably going to be a bit busy with over the next few days anyway,
so take your time! I know there's a lot going on right now from releases, to the fundraiser, to bugfixing, etc...


Ah sorry, I misread the message.

No worries, Alvin. I think we worded the change poorly.

Yes, the options for Windows Ink tablet users are limited. But it
looks like that Windows Ink tablets are getting more popular (thanks
to 2-in-1 laptops), so it seems to be a good idea to find a better way
to accommodate those users.

Sounds reasonable to me. We'll see what we can do for Windows Ink users, of course.
It's really going to take some testing to highlight the specific issues,
and neither Eoin or I can do that as we only have access to Wacom tablets.

Actually, only right-click press itself is for opening context menus
and some context-sensitive actions. I don't think right-click drag is
used for anything, or is it? Using right-click drag for scrolling is
not so implausible.

Ok. I was thinking that context menus often pop up on mouse down,
which would interfere with the scrolling. And since right-click release events
often trigger menu selections (does that happen on Windows too?
I can't remember off the top of my head) it might be prone to accidents.

Of course, all that stuff could be worked around and there's no harm in testing it!

Due to how the API works, pen button on Windows Ink always maps to
right click (and if there is a second button it's usually mapped to
emulate an eraser tip). It's not (and doesn't make sense to be)
user-configurable. At this moment it might even make sense to consider
mapping right-click dragging to act like the middle button on the
canvas.

Ok ok, I see..

The optimal default is to have multiple *sensible* ways for different
devices to trigger scrolling. If this is not possible, no amount of
options will help.

Yeah. Eoin was similarly annoyed by the 'either X or Y' limitation that QScroller imposes.

One thing to consider though, is that it's
technically possible to distinguish between tablet, touch and mouse
events so it's possible to have different behaviour for different
devices. The limitation here is imposed by QScroller and the
underlying QFlickGestureRecognizer which is very unfortunate. If
only they are more flexible. Would it be worth it to fork it or roll
our own kinetic scrolling implementation? Perhaps not really. But then
a group of users would not be able to use this function in the optimal way.

I'm not opposed to that, but I don't think we should do it preemptively.
Testing within the current limitation will give a good idea of where the potential problems are
and what changes might be necessary.


I like this patch, I had wanted to do something like it. I do think the 'disabled' state should probably come back. Who knows what kind of weirdness people's input devices might have. Once we're sure that people have tested the feature and there's no major bugs, we can think of removing 'disabled' from the list.

Glad to hear it, Wolthera.

Kinetic Scrolling can still be totally disabled by unchecking the "Kinematic Scrolling (needs restart)" QGroupBox.
I think this is a bit better than having it inside the same drop-down menu as the input choice
because it grays out the other kinetic scrolling-related options when disabled.

Actually, only right-click press itself is for opening context menus
and some context-sensitive actions. I don't think right-click drag is
used for anything, or is it? Using right-click drag for scrolling is
not so implausible.

Ok. I was thinking that context menus often pop up on mouse down,
which would interfere with the scrolling. And since right-click release events
often trigger menu selections (does that happen on Windows too?
I can't remember off the top of my head) it might be prone to accidents.

Of course, all that stuff could be worked around and there's no harm in testing it!

Ah wait, I totally forgot that context menus on Linux does show on
mouse down. Native Windows applications on the other hand shows
context menus on mouse up. (Can someone remind me what Qt does on
Windows?)

emmetoneill added a comment.EditedOct 2 2018, 7:16 PM

If D8015 didn't take care of it already, this would also close this wishlist/bug: https://bugs.kde.org/show_bug.cgi?id=324005

eoinoneill updated this revision to Diff 42981.Oct 6 2018, 7:54 PM

So I've made the necessary revisions to the cmake file situation. Now the functions should be exported for linking.

I experimented with the right click kinetic drag functionality. While it works for some widgets (timeline_frames_view, widgets without any right click behavior), widgets such as the node view and brush preset pallet seem to have stubborn right click behaviors that don't work well with the kinetic scrolling. The usual behavior of a right menu with kinetic scrolling seems to be that if you hold down right click after a certain period of time (roughly 2 seconds) it will pop open the context menu. You can also tap right click to get the context menu. However, the nodes widget and some others sees to have some custom right click behavior that creates the context menu instantly, bypassing kinetic scroll behavior. Because of this, I'm going to leave right click select off for now.

Windragon, if you wish to test out right click select, the configuration option is commented out in kis_dlg_preferences.cc.

eoinoneill updated this revision to Diff 42987.Oct 6 2018, 8:54 PM

Had to update this again due to a major naming refactor in backend I forgot to account for.

rempt added a comment.Oct 7 2018, 8:31 AM

I see this:

QObject::connect: Cannot connect (null)::stateChanged(QScroller::State) to KoToolBoxScrollArea::slotScrollerStateChange(QScroller::State)

eoinoneill updated this revision to Diff 43013.Oct 7 2018, 10:27 AM

So I've corrected some of the issues regarding:

  • Properly formatting if statements
  • ToolboxScrollArea missing signal, as @rempt posted above.
  • Added all parameters of Kinetic Scrolling to configuration file for potential user overrides (not all are exposed in options panel).
  • Added kinetic scrolling to brush preset history docker.

List of unaffected widgets:

  • Filter selection dialog
  • Settings dialog in general (esp. keyboard shortcuts)
  • Brush pattern configuration widget (preview label has scrollbars and doesn't kinetic-scroll)
eoinoneill updated this revision to Diff 43139.Oct 8 2018, 4:38 PM

Changed KisKineticScrolling module from widgets to widgetutils in order to give other widgetutils access to KineticScrolling.

Added Kinetic Scrolling to the following remaining widgets:
Filter Widgets
Pattern Dock -> Preview Area
Canvas Input Config Page
Shortcut Editor Config Page
New Palette View
Animation Curve Dock
Channels Dock
Composition Docker
SVG Symbol Collection Dock

rempt accepted this revision.Oct 10 2018, 3:26 PM
This revision is now accepted and ready to land.Oct 10 2018, 3:26 PM
eoinoneill closed this revision.Oct 30 2018, 1:35 AM