smb: fast copy

Authored by sitter on Feb 19 2020, 1:22 PM.

Description

smb: fast copy

Summary:
see https://bugs.kde.org/show_bug.cgi?id=291835#c57 for background

  • reading now happens inside a future. should be safe since we don't have any other threads doing anything while we wait.
  • the future feeds into a buffer from which the main thread will take file segments and write them to disk
  • buffer has 4 segments and synchronizes the threads via wait conditions
  • the size of a segment is determined somewhat dynamically between 64kb and 4mb. the larger a file is the more it benefits from larger read requests

under perfect conditions this yields approximately mount-level copy
performance, unfortunately those are hard to hit so on average it's usually
less (somewhere in the range of 10 to 20% depending on the actual file
size and server type).

for many tiny files performance is about where it was before. the larger
the files get the greater the gains from this diff though.

specifically here's some samples I've taken:

  • for downloads from windows10
    • 1G & 4G file
      • compared to 20.04 is ~77% faster
      • within 10% of windows
    • 8G file
      • compared to 20.04 is ~79% faster
      • within 5% of windows
  • uploads to windows10
    • all sizes
      • compared to 20.04 is ~50% faster
      • now comparable performance to windows
  • for remote-to-remote file copies from windows10 to smbd 4.11.6
    • 1000 x 5K files
      • no change, dreadfully slow, likely problem in KIO internals
    • 1G file
      • compared to 20.04 is ~45% faster
      • within 8% of windows
    • 4G file
      • compared to 20.04 is ~95% faster
      • and somehow 18% faster than windows (could be a fluke)

I've done transfers for 128M, 256M, 512M, 1G, 4G and partially 8G.
Differences not mentioned are either unchanged, negligible or in line with
documented trends.

BUG: 291835
FIXED-IN: 20.08

Test Plan:

  • fallocate -l 1G file
  • copy around
  • copy kio-extras around

Reviewers: ngraham, cfeck, Frameworks, Dolphin

Subscribers: mmustac, meven, hallas, anthonyfieroni, asturmlechner, kde-frameworks-devel, kfm-devel

Tags: Dolphin, Frameworks

Differential Revision: https://phabricator.kde.org/D27504

Details

Committed
sitterApr 3 2020, 5:03 PM
Differential Revision
D27504: smb faster copy to local
Parents
R320:7e679b1366fc: Use more UDSEntry::reserve() calls to avoid reallocs on multiple inserts
Branches
Unknown
Tags
Unknown