Here are the notes from the design discussion we had today about how to fix "two kioslaves accessing the same MTP device at the same time cannot work":

shared library classes (initially kio-extras/mtp/shared, can be moved to KIO later, once stable)

MTPDevice { int id, QString name... }
MTPFolder, MTPFile (or whatever)
DBus XML (with e.g. a listDevices method)

kio-extras/mtp/kiod_module

(DBUS API) listDevices() implemented using libmtp
When making API changes, run qdbuscpp2xml by hand and copy the result into the above shared library.

kio-extras/mtp/kioslave

listDir() { blocking DBUS call to listDevices() ==> converted to UDSEntry }

amarok or any other application wanting to do MTP

Blocking calls from the main thread would block the GUI, so instead, rather something like:
- ListDevicesJob { async DBus call to listDevices() }
- OR sync calls from a thread
- OR sync calls from helper process
(this is why KIO wouldn't provide any direct C++ API for this, the API is the DBus XML)

iirc we implemented a similar approach for bluetooth, we had similar limitations for OBEX, so perhaps however implements this can take inspiration from that code.

Apparently, I'm working on exactly this approach for the last week. :) I moved all the MTP handling to a KDED module. The KIO slaves then connects to this KDED module via dbus. It will still take some time to be finished but I get pretty promising results so far.
I will be at the Akademy today (16.08.18) at around 5pm, so if you are around we can discuss more on this topic. Sadly am not able to take my laptop with me to show you how it works so far...

In T9390#155256, @akrutzler wrote:

Apparently, I'm working on exactly this approach for the last week. :) I moved all the MTP handling to a KDED module. The KIO slaves then connects to this KDED module via dbus. It will still take some time to be finished but I get pretty promising results so far.
I will be at the Akademy today (16.08.18) at around 5pm, so if you are around we can discuss more on this topic. Sadly am not able to take my laptop with me to show you how it works so far...

That's awesome. Yes I'm at the venue and I'll be happy to discuss this with you!

Great, see you later!

From an historical point of view: the last idea from the then kio-mtp author and maintainer was to implement a dbus daemon directly in libmtp. You can find the work in the dbus-daemon branch of libmtp:

https://sourceforge.net/p/libmtp/code/ci/dbus-daemon/tree/

and on the wiki page and the mailing list discussion around June 2013:

https://sourceforge.net/p/libmtp/wiki/MTP%20Daemon/

https://sourceforge.net/p/libmtp/mailman/libmtp-discuss/?viewmonth=201306

Maybe it could be useful to look at that too so that can be useful for all libmtp users.

Over the past few weeks, I've been working on a better / more reliable way to access MTP devices with the KIO framework. I'm just about to show you what I have done so far. If you have any ideas that improves my work, don't hesitate and let me know :)

Structure
This project consists of 3 different parts.
The daemon. A KIOD module that provides a D-Bus interface for MTP device management. This is where all the low-level MTP magic happens. It’s also the only part which depends on libmtp.
The library. A KDE library that provides high level daemon access for any app within the Qt/KDE world. Note: this lib is far away from being a public API as I want to hear your design considerations first.
The slave. A KIO slave that uses the library mention above to give all applications, using the KIO framework, MTP support.

D-Bus interface
Just a few things about the IPC between daemon and library.

For the D-Bus interface, I had the following goals in mind:

• use a object-oriented approach
• access files/folders by their file-path instead of their MTP item ID

From the point of view of a file manager, there are several MTP devices connected; all of them provides at least one MTP storage, in which all files/folders (objects) are located. Every object should be accessible by the file path as this feels much more natural.
With that, I came up with a basic set of objects for the daemon:

As an example, the object tree for one device with one available storage looks like this:

What works
Note: If you were unable to operate your device with any other MTP client out there, this implementation won’t help you either as it may depend on the same low-level MTP implementation (libmtp) as the others.

• Discovery of devices, storages, files and folders
• Stat files for filesize, last modification time and mime type
• Free space of storages
• get files
• put files (with temporary file in between)
• copy (a big amount of) files and folders from/to the device to/from local file system
• Create new folder
• Create new file (context menu)
• Rename files and folders (kioclient5)
• Delete files and folders
• Open files directly on the device with Kate, Okular, VLC,…
• Edit and save files with Kate
• Access devices from multiple apps (but still: just one operation possible at the same time because of the design of MTP)

Actually all of that works like a charm. The bad thing is that I am currently not able to test multiple devices at a time. Also, my One Plus 3T doesn’t support any external SD card (one storage only). I'm trying to get some more devices for testing.

Software setup

• Arch Linux with the latest KDE software (kdesrc-build) in separate prefix
• kioclient5, Dolphin, Kate, Okular, Gwenview as reference apps

Bugs fixed
https://bugs.kde.org/show_bug.cgi?id=319880
https://bugs.kde.org/show_bug.cgi?id=325924
https://bugs.kde.org/show_bug.cgi?id=336456
https://bugs.kde.org/show_bug.cgi?id=372860
https://bugs.kde.org/show_bug.cgi?id=382046
https://bugs.kde.org/show_bug.cgi?id=383314
https://bugs.kde.org/show_bug.cgi?id=396527

Limitations/Issues
Daemon

• No track/playlist/album management
• No concurrent operations for different devices The daemon is currently single threaded; if one device is doing a long operation (copying big files,…) it blocks the event loop. AFAIK, for one device it will always be like that due to MTP limitations (just one operation at the same time). But maybe that works for 2 different devices (one operation for each device at the same time)?

KIO slave

• Unable to copy/move files directly on the device (libmtp doesn’t support it)
• Unable to rename something in dolphin (button always disabled). I would appreciate any help.
• No way to notify the user that the daemon is currently busy due to pending operations.
• The put() operation currently stores a temp-file and afterwards sends this the daemon instead of sending the data chunks directly.

One real issue is the way how libmtp is doing the file and folder discovery. It’s a blocking method that gives no information about its progress. It may take a while to list all objects of folders, containing lots of files. Be patient when opening folders like camera or DCIM.
Gwenview doesn’t work properly for me. I get the thumbnails but I’m unable to open the actual image (but sometimes it does, yay).

Next steps

• Implicit / explicit closing of devices. Once a device is opened by the daemon, it remains open, preventing third-party apps from using the device.
• KMTP library: pimpl, better documentation, (other name?), ...
• Add more properties (battery level, model name,…). Currently just the properties for file management are provided.
• Multi threaded daemon? To be honest, I’m considering to stay with a single threaded solution. IMO the most obvious use case I can think of is copying some files to/from a single device. So I ask myself if it’s really worth it, making the daemon more complex for a use case that is (IMO) very rare. Maybe we should encourage power users to go for KDE Connect instead :)
• Give an ADB based slave a try (as a separate project)? MTP is quite limited by design, some users claims that ADB is doing a much better job. As far as I remember, ADB was always working out of the box for me. But I have not done too much research on how it works with KIO.
• Waiting for your device specific corner case :)

Get the source
I created a kio-extra clone repository at git://anongit.kde.org/clones/kio-extras/akrutzler/kmtpd.git. All the latest stuff will be there in the mtp_next branch. If you feel more comfortable with a Diff please let me know.

Happy testing!

I would suggest pushing this as a branch of the main kio-extras repository (is your clone just a clone of kio-extras with your changes on top?) to ensure it doesn't get "lost" (and would make merging easier). Eventually a Phabricator diff would also help for people reviewing the code.

Excuse my late reply. I have a few more devices for testing and it works pretty well. After a little refactoring of the code, I will create a Diff soon to, as you mentioned, reach more people to review it.