Currently some of KIO file operations have some optimized path when working with local files avoiding using ioslave but use direct files operations.
This makes those file operations block the calling Thread, freezing it until the file operation has finished.
This is troublesome and we need to improve this.
Identified KIO calls concerned are so far :
- KIO::del
- dir listing
It causes bugs (I suspect there are more spread across apps using KIO) :
- https://bugs.kde.org/show_bug.cgi?id=293888 - Poor performance with mounted network locations
https://bugs.kde.org/show_bug.cgi?id=390748 - Deleting a file freezes dolphin UI during the deletion(fixed by D24962)- https://bugs.kde.org/show_bug.cgi?id=342056 - Slow file copy with many small files
- https://bugs.kde.org/show_bug.cgi?id=353813 - File Browser freezes Kate when opening a file in a folder with thousands of files
- https://bugs.kde.org/show_bug.cgi?id=281270 - Inconsistent notifications during/after file operations (copying, moving, deleting, compressing, extracting) depending on amount of data/items involved
With @dfaure we discussed a way to use an io worker QThread to run those optimized path out of the main thread.
In essence the work will be about moving the io code logic to a separate worker class which will run in a separate thread.
Please It won't help the performance, but performance perception since we will then to have progressive file listing for instance.
This pseudo code presents the basic logic for the DeleteJob, details will likely change (I am not sure if using invokeMethod is necessary, connect with signal/slot might be enough) :
DeleteJob() {
QThread m_ioThread:
IOWorker worker = IOWorker() # subclass QObject
worker->moveToThread(m_ioThread)
worker->invokeMethod(worker, &IOWorker::deleteDirs, this):
m_ioThread.start();
}
IOWorker::deleteFiles () {
[...]
if (timer.elapsed() > delay) { // 200 ms to follow value used in sftp ioslave for instance
invokeMethod(job, &IOWorker::(slotProcesedFilesChanged| slotDeleteFileInProgress), numFilei, sizeRemoved);
timer.restart();
}
[...]
}
IOWorker::deleteDirs(){
[...]
// empty dir with deleteFiles
[...]
invokeMethod(job, &IOWorker::slotWorkerFinished);
}We need to relay event back and force then between the job and its worker object inside its execution thread.