File ioslave : use Better setting for sendfile syscall
ClosedPublic

Authored by meven on Apr 4 2020, 12:20 PM.

Details

Summary

Changes :

  • use sendfile when copying file bigger than 2 GB
  • copy 512 kB instead of previously 32 kB for each sendfile or read/write copying iteration

This effectively increase the copying speed, since the filesystem is asked to copy more at once, it lets it reach higher speed before getting the next data to copy.

CCBUG: 342056
CCBUG: 402276

Test Plan

Copying large files or folders with medium sized files (for instance photos) is noticebly faster.

Diff Detail

Repository
R241 KIO
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
meven created this revision.Apr 4 2020, 12:20 PM
Restricted Application added a project: Frameworks. · View Herald TranscriptApr 4 2020, 12:20 PM
Restricted Application added a subscriber: kde-frameworks-devel. · View Herald Transcript
meven requested review of this revision.Apr 4 2020, 12:20 PM

any idea what buff_src.st_size < 0x7FFFFFFF was for?

reduce the overhead of processedSize dbus progress notification in each iteration since it is called less often, adding some more speed.

From what I recall in copy job the processed size in emitted by a timer and not every time a chunk got processed

meven added a comment.Apr 4 2020, 1:37 PM

any idea what buff_src.st_size < 0x7FFFFFFF was for?

None, it does not makes sense to me, perhaps, we had an implementation limitation.

reduce the overhead of processedSize dbus progress notification in each iteration since it is called less often, adding some more speed.

From what I recall in copy job the processed size in emitted by a timer and not every time a chunk got processed

You are right, it is emitted every 100ms in SlaveBase::processedSize, so this part was overzealot.

bruns added a subscriber: bruns.Apr 4 2020, 2:09 PM

@dfaure: probably this:

man sendfile, NOTES

The  original Linux sendfile() system call was not designed to handle large file offsets.  Consequently, Linux 2.4 added sendfile64(), with a wider type for the offset argument.  The glibc sendfile() wrapper function transparently deals with the kernel differences.
dfaure added a comment.Apr 4 2020, 2:38 PM

Sounds like the size check should stay then.

bruns added a comment.Apr 4 2020, 3:01 PM

Sounds like the size check should stay then.

Pessimizing sendfile for everyone, for the sake of Linux 2.2.x users?

dfaure accepted this revision.Apr 4 2020, 3:11 PM

Ah sorry, I missed the fact that one doesn't actually need to call sendfile64() explicitly.

And if sendfile fails we have a fall back, so yeah, might as well try in all cases.

This revision is now accepted and ready to land.Apr 4 2020, 3:11 PM
meven edited the summary of this revision. (Show Details)Apr 4 2020, 6:40 PM
meven edited the summary of this revision. (Show Details)Apr 4 2020, 6:51 PM
This revision was automatically updated to reflect the committed changes.