Allow balancing of Stats saving timers
ClosedPublic

Authored by trufanov on Jun 24 2020, 2:16 PM.

Details

Reviewers
stikonas
Summary

I've faced with a following problem in my KTorrent. I have 350 torrents that I would like to seed them at once. So my queue setting is big enough and a lot of torrents are active at a single moment. After KTorrent launch in a 5 minutes I'm starting to experience GUI hangs. KTorrent becomes completely blocked for a minute or two. Its process have "disk wait" state.
I've traced down this problem to the loop:

while (i != qman->end())
            {
                bt::TorrentInterface* tc = *i;
                if (tc->getStats().running)
                {
                    tc->update(); // HERE
                    updated = true;
                }
                i++;
            }

In KTorrent: https://invent.kde.org/network/ktorrent/-/blob/master/ktorrent/core.cpp#L1002

And this operations in void TorrentControl::update() in libktorrent:

// to satisfy people obsessed with their share ratio
           if (stats_save_timer.getElapsedSinceUpdate() >= 5 * 60 * 1000)
           {
               saveStats(); // HERE
               stats_save_timer.update();
           }

https://invent.kde.org/network/libktorrent/-/blob/master/src/torrent/torrentcontrol.cpp#L245

The app is hanging on stats_file->sync(); in saveStats(). It calls KConfig::sync under the hood.

If saveStats(); is removed - the problem disappears.

I think the problem is that 350 torrents try to save their Stats files one by one in a loop in a single thread bcs they all strted at once and have timer >= 5 min at once. The data they save to Stats files is always has changed state as they write time of seeding in it.

I suggest to balance this a bit. I've tried to dynamically add a minimal delay between next saveStats is allowed. The delay is a 5 min / number of active torrents. If there are more then 500 torrents then I suggest to use 5 min * (1+ number of torents / 500) period.

This patch adds the interface between TorrentControl instance and QueueManager on clients side. TorrentControl requests permission to save Stats from QueueManager and QueueManager by default grants it if more than 5 minutes is elapsed since last sync. That's equal to current behavior.

The next patch I'm going to submit for KTorrent project will override this default virtual method and will permit Stats syncing with additional restriction: N msecs should elapse since the last permission was given. That's a simplest load balance I think out and it seems to work. KTorrent not hangs in disk wait state anymore. At least not bcs of Stats file savings.

Diff Detail

Repository
R472 KTorrent Library
Lint
Lint Skipped
Unit
Unit Tests Skipped
trufanov requested review of this revision.Jun 24 2020, 2:16 PM
trufanov created this revision.
stikonas accepted this revision.Jun 24 2020, 6:40 PM

Other than whitespace issue it seems fine.

src/torrent/torrentcontrol.cpp
534

indentation

This revision is now accepted and ready to land.Jun 24 2020, 6:40 PM
trufanov closed this revision.Jun 25 2020, 11:08 AM

Hmm... I forget to fix indentation.
It's quite annoying that KTorrent and Libktorrent projects both have different styles of indentation: tabs and spaces.

The changes were commited with: https://invent.kde.org/network/libktorrent/-/commit/8fc90833f2747adcee209cef29f626dd034022ac

Hmm... I forget to fix indentation.
It's quite annoying that KTorrent and Libktorrent projects both have different styles of indentation: tabs and spaces.

The changes were commited with: https://invent.kde.org/network/libktorrent/-/commit/8fc90833f2747adcee209cef29f626dd034022ac

Maybe it's worth running astyle so both have spaces?