Touchpad gesture support for MacBooks
ClosedPublic

Authored by poke1024 on Oct 3 2017, 9:35 AM.

Details

Reviewers
rempt
alvinhochun
Group Reviewers
Krita
Summary

This adds full native touch support (pinch, rotate, zoom, smart zoom) for MacBook touchpads and similar devices.

Needs QNativeGestureEvents (don't know if non-OSX platforms support this); also needs to use the updated shortcut settings file from this PR in order work.

Kept here for reference and maybe a first review, I've applied for an KDE dev account, and might resubmit this once I have it.

Diff Detail

Repository
R37 Krita
Lint
Lint Skipped
Unit
Unit Tests Skipped
poke1024 created this revision.Oct 3 2017, 9:35 AM
rempt added a subscriber: rempt.Oct 3 2017, 9:40 AM

Cool! I'll test it on my macbook later on. You've got your dev account now, btw, just got the mail from the sysadmins!

You mean touchpad gesture support, not touch (as in touchscreen) support, right? Or are the two mixed together on OS X?

I'll check whether it breaks current touchscreen support on Windows. I don't think Qt supports the precision touchpad on Windows but I will try it too...

Was two finger pan and zoom already working with the existing touch gesture code?

alvinhochun added a comment.EditedOct 3 2017, 12:44 PM

Got this while compiling on Windows, not sure if related to this patch or not...

[ 43%] Linking CXX shared library ..\..\bin\libkritaui.dll
CMakeFiles\kritaui.dir/objects.a(kis_input_manager.cpp.obj): In function `KisInputManager::profileChanged()':
F:/dev/krita/new/src/libs/ui/input/kis_input_manager.cpp:666: undefined reference to `KisInputManager::Private::addNativeGestureShortcut(KisAbstractInputAction*, int, KisShortcutConfiguration::GestureAction)'
F:/dev/krita/new/src/libs/ui/input/kis_input_manager.cpp:666:(.text+0xf28): relocation truncated to fit: R_X86_64_PC32 against undefined symbol `KisInputManager::Private::addNativeGestureShortcut(KisAbstractInputAction*, int, KisShortcutConfiguration::GestureAction)'
collect2.exe: error: ld returned 1 exit status
mingw32-make[2]: *** [libs\ui\CMakeFiles\kritaui.dir\build.make:10112: bin/libkritaui.dll] Error 1
mingw32-make[1]: *** [CMakeFiles\Makefile2:16246: libs/ui/CMakeFiles/kritaui.dir/all] Error 2
mingw32-make: *** [Makefile:140: all] Error 2
alvinhochun requested changes to this revision.Oct 3 2017, 1:17 PM
alvinhochun added a subscriber: dkazakov.

Looks like this patch is missing the definition of KisInputManager::Private::addNativeGestureShortcut.

Also, my opinion is that this patch can be split into several smaller commits. I believe small commits can help bisecting bugs (if there are any to be found) and it should make review a bit easier. Now, I'm not sure if it is a good idea to really put up multiple smaller reviews, but what you can do is to make a branch, make some small commits, update this review with a diff of your branch against the fork point (or the latest commit on master that is merged into your branch), and make your branch available to be pulled somewhere (it can be a fork on github or even pushed to the main repo with branch name starting with your first name plus a forward slash). But I better ask Boud (@rempt) and/or Dmitry (@dkazakov ) about this...

This revision now requires changes to proceed.Oct 3 2017, 1:17 PM

@alvinhochun Yes, pan and zoom are implemented, but on OS X, pan actually maps to zoom as it is reported as a mouse wheel event by Qt, it's very confusing, and zoom works but not, touch wise, as reliable as the native gestures.

Yes, only touchpad, no touchscreen support, though I would hope some day this API might be used to connect to the touch features of Wacom Intuos and Cintiqs as well.

poke1024 updated this revision to Diff 20307.Oct 3 2017, 4:42 PM

Adds missing KisInputManager::Private::addNativeGestureShortcut (sorry for that)

Confirmed that this patch doesn't break existing touch gestures on Windows, and that the "precision touchpad" isn't supported here.

It's funny because I started reading about gesture support in Qt two days ago and was thinking of the possibility of re-implementing the touchscreen gestures with QPinchGesture, and then you submitted this patch adding yet another way of handling gestures... except it only works on OS X...

@alvinhochun I should have mentioned that it's not really enabled for other platforms as I didn't want to break anything. You can try to commend out the two #ifdef Q_OS_OSX and see what happens; it then will override Krita's implementation. But if QT does not send any QNativeGestureEvents on specific platform, this basically breaks touch support for the platform.

@alvinhochun I should have mentioned that it's not really enabled for other platforms as I didn't want to break anything. You can try to commend out the two #ifdef Q_OS_OSX and see what happens; it then will override Krita's implementation. But if QT does not send any QNativeGestureEvents on specific platform, this basically breaks touch support for the platform.

Does this mean this patch might break proper touchscreen support on OS X?

@alvinhochun I believe not. First, the code only overrides multi finger gestures, not single finger touch inputs. And this is done by processing the QNativeGestureEvent that Qt gives us. The classic Krita touch implementation of those touch events that are detectable using QNativeEventGestures under OS X is only deactivated because there would be a conflict otherwise: for pinch and rotate, Qt sends a QNativeEventGestures as well as a QTouchEvent, which means Krita would process the same gesture twice with different code paths, which is obviously not desirable. A similar thing happens with pan events (they come as QWheelEvent as well as QTouchEvent on OS X). So, the only reason some code paths are overriden is to not handle the same gesture twice. We might of course do something like: if a QNativeGestureEvent arrives, then ignore all following or near QTouchEvents; but that would assume that the QNativeGestureEvent always comes first and the QTouchEvents for the same physical finger movement comes second, and that is not documented in Qt. So it's a bit difficult how to really deal with this cross platform wise, if one does not know which events the platform will give us. On OS X it's simple. But on other systems this might have to be an option like "Enable native gesture events".

rempt added a comment.Oct 5 2017, 7:23 AM

I wonder if using QNativeGestureEvent for pan, zoom and rotate makes sense for other platforms as well.

But on other systems this might have to be an option like "Enable native gesture events".

Couldn't your implementation be disabled by default and be enabled the first time a QNativeGestureEvent is received?
This way it would be enabled on all systems that support these events.

I kind of think it might be better to just handle all touch gestures by Krita itself, this way we can have more fine-grained control on how touch interaction works.
(I have some ideas but I need to first write them down.)

I am compiling a version to test if QNativeGestureEvent works on Windows, but I don't think Qt supports it.

rempt added a comment.Oct 5 2017, 7:36 AM

Hm, I have some trouble actually using gestures other than zoom on the touchpad of my macbook pro (2015) -- I'm not sure how to activate panning and rotating...

Tested, doesn't work on Windows.

It looks like QTouchEvent::device() returns QTouchDevice and QTouchDevice::type() can be used to differentiate touchscreen and touchpad events (though really only seem to be useful on OS X atm). Can you perhaps use it to allow touchscreen gestures on OS X?

@alvinhochun I'm not sure there are many ways to use OS X with a touch enabled screen. I have a touch enabled Wacom Cintiq but their touch drivers are unfortunately just a hack mapping to certain keyboard shortcuts and they seem completely separate from Apple native touch events you get from using a MacBook trackpad, so no QNativeGestureEvents here.

Oh no, clearly I have no idea how bad touchscreen support on OS X is. I'm just worried there will be a regression, but it sounds like there won't be any then.

alvinhochun retitled this revision from Touch support for MacBooks to Touchpad gesture support for MacBooks.Oct 8 2017, 4:09 PM
rempt accepted this revision as: rempt.Oct 9 2017, 9:37 AM

I think I'm not dexterous enough to get smart zoom to work, but the rest works fine on OSX, and I don't see a regression on Linux.

I haven't seen any visible regressions on Linux

alvinhochun accepted this revision.Oct 9 2017, 12:14 PM

No regression on Windows.

Do I need to accept this because I made a change request?

This revision is now accepted and ready to land.Oct 9 2017, 12:14 PM
rempt added a comment.Oct 9 2017, 12:17 PM

One acceptance is enough I think, formally.

pushed.
https://commits.kde.org/krita/cbb6ed2daa053715aeba943eba78bde31c0d6e39

We should document somewhere that Mac users need to delete/update their local kritadefault.profile (usually under Application Support/krita/input) in order for this to work as the new modified kritadefault.profile needs to be fetched.

poke1024 closed this revision.Oct 22 2017, 11:57 AM