kservice uses kdirwatch to be notified when one of the directories containing it's source of information changes.
But also the method TimestampChecker().checkTimestamps(allResourceDirs) is called once every 1500 miliseconds.
When copying files from any directory to another one, this check introduces a big use of io/wait.
Details
- Reviewers
dfaure - Group Reviewers
Frameworks
Before, when copying 20Gb files, the cpu spent in io/wait, as seen by ksysguard, was around 80%.
After, when copying 20Gb files, the cpu spent in io/wait, as seen by ksysguard, is around 50%
Diff Detail
- Repository
- R309 KService
- Lint
Lint Skipped - Unit
Unit Tests Skipped
This goes against the whole redesign of ksycoca that I did some time ago. It's supposed to check timestamps on dirs, to rebuild the cache on demand, much like many other caches out there. Did you run the unittests in kservice after this change? I strongly doubt they pass.
I don't understand why a copy triggered by the user (so, I assume, not related to ~/.local) would trigger an unusual effort on the part of ksycoca? How can this be related?
It's just the I/O of the copying happening at the same time as the I/O from ksycoca when using the K menu or something?
Yes, you are right, it doesn't pass the unittests with this tentative patch.
Some local changes should have triggered the heavy load, because now I'm not able to reproduce them.
But there is a problem somewhere (I'll investigate further) because today, everytime I change directory in dolphin (even in /d/....), or change tabs in konsole, or change file to edit in kate or save a file in kate, a messagebox is shown or a notification is shown, there is a check for the timestamps, and this check has to check all the directories in the QMap:
checking file timestamps for QMap(("/etc/xdg/menus", 1515667084956)("/home/jtorres/.config/menus", 1515507849227)("/home/jtorres/.local/share/applications", 1515507849219)("/home/jtorres/.local/share/kservices5", 0)("/home/jtorres/.local/share/kservicetypes5", 0)("/home/jtorres/.local/share/mime", 1512993366020)("/usr/local/share/applications", 1515667254872)("/usr/local/share/kservices5", 0)("/usr/local/share/kservicetypes5", 0)("/usr/local/share/mime", 0)("/usr/share/applications", 1515667274588)("/usr/share/kservices5", 1515667109372)("/usr/share/kservicetypes5", 1515667086640)("/usr/share/mime", 1515667195828))
And the directories watched by KDirWatch m_fileWatcher are:
kf5.kcoreaddons.kdirwatch: Entries watched:
kf5.kcoreaddons.kdirwatch: [ Entry for "/", dir, using INotify inotify_wd=6, has 0 clients , nonexistent subentries: 0x20e45f0 "/.directory" ]
kf5.kcoreaddons.kdirwatch: dependent entries:
kf5.kcoreaddons.kdirwatch: 0x20e45f0 "/.directory" NonExistent
kf5.kcoreaddons.kdirwatch: [ Entry for "/.directory", file, non-existent, using INotify inotify_wd=-1, has 2 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-3" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: by "KDirWatch-5" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/etc/xdg/taskmanagerrulesrc", file, using INotify inotify_wd=9, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-8" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.cache/ksycoca5_es_EE55KafscqW408BVSKZuHAbUvuI=", file, using INotify inotify_wd=1, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-1" ( 2 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.config", dir, using INotify inotify_wd=8, has 0 clients , nonexistent subentries: 0x43c53f0 "/home/jtorres/.config/taskmanagerrulesrc" 0x446ba20 "/home/jtorres/.config/klaunchrc" ]
kf5.kcoreaddons.kdirwatch: dependent entries:
kf5.kcoreaddons.kdirwatch: 0x43c53f0 "/home/jtorres/.config/taskmanagerrulesrc" NonExistent
kf5.kcoreaddons.kdirwatch: 0x446ba20 "/home/jtorres/.config/klaunchrc" NonExistent
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.config/klaunchrc", file, non-existent, using INotify inotify_wd=-1, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-9" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.config/ksmserverrc", file, using INotify inotify_wd=7, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-7" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.config/plasma-org.kde.plasma.desktop-appletsrc", file, using INotify inotify_wd=5, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-2" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.config/plasmarc", file, using INotify inotify_wd=2, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-2" ( 60 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/home/jtorres/.config/taskmanagerrulesrc", file, non-existent, using INotify inotify_wd=-1, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-8" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/usr/share/icons/breeze/index.theme", file, using INotify inotify_wd=4, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-2" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/usr/share/plasma/desktoptheme/default/metadata.desktop", file, using INotify inotify_wd=3, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-2" ( 1 times) ""
kf5.kcoreaddons.kdirwatch: [ Entry for "/usr/share/solid/actions", dir, using INotify inotify_wd=10, has 1 clients ]
kf5.kcoreaddons.kdirwatch: by "KDirWatch-10" ( 1 times) ""
Just to let you know that I know how to reproduce the high I/O of plasmashell and continuous calls to Kservice.
You can reproduce it just having opened the vivaldi web browser. (I never though it could be a problem).
If it goes to a webpage that still uses flash, the effect is even bigger.
! In D9840#203803, @ngraham wrote:
Do any of the active or recently-landed patches address this?
No, but I have one in https://phabricator.kde.org/D10342
Can you strace vivaldi to see what it's doing? It must be doing something wrong if it's triggering so much I/O.
This is related to https://phabricator.kde.org/D10342
I've straced vivaldi and main process is continuously doing futex and gettid, and the active child the same + write + sendto. Every tab is a process, just like chromium.