Don't include any directory sizes in DirectorySizeJob
AbandonedPublic

Authored by davidedmundson on Nov 21 2018, 12:19 PM.

Details

Reviewers
dfaure
apol
Summary

Running stat on a directory gives a size of on ext3 gives a size of

  1. Running stat on a directory on NTFS gives a size of 0.

If you copy some files to another FS the total size is different and it
falsely appears that some data is lost.

We're not trying to count the disk usage, otherwise we'd be looking at
block usage not adding up individual files. Therefore this patch removes
including the directory size as it's a meaningless value.

Test Plan

More precise unit test passes on ext4

Dolphin properties shows the same value for copied files on an ext3 and
ntfs file system

Diff Detail

Repository
R241 KIO
Branch
master
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 5213
Build 5231: arc lint + arc unit
Restricted Application added a project: Frameworks. · View Herald TranscriptNov 21 2018, 12:19 PM
Restricted Application added a subscriber: kde-frameworks-devel. · View Herald Transcript
davidedmundson requested review of this revision.Nov 21 2018, 12:19 PM
davidedmundson edited the summary of this revision. (Show Details)
davidedmundson edited the test plan for this revision. (Show Details)

Squash

apol accepted this revision.Nov 21 2018, 11:37 PM
apol added a subscriber: apol.

LGTM

This revision is now accepted and ready to land.Nov 21 2018, 11:37 PM

Hmm hmm. The goal of this was to behave like "du" as much as possible. As you say, the real bug isn't that we count directories, but that we count files "as is" instead of blocks. But I'm not sure how easily we could get block information from here.

(sure it's convenient for the unittest, but less meaningful for the users....)

adridg added a subscriber: adridg.Nov 22 2018, 3:49 PM

With UFS, stat(1) tells me

[adridg@beastie .../kio/build]$ stat .
2080587972 222556 drwxr-xr-x 9 adridg adridg 4294967295 17 "Nov 22 14:40:28 2018" "Nov 22 14:42:22 2018" "Nov 22 14:42:22 2018" "Nov 22 14:39:51 2018" 131072 17 0x800 .

So that's a size of 17 -- on UFS that looks like the count of the files in the directory (I added a file and then size is 18).

Without this patch: jobtest has 71 tests, all pass.
With this patch: jobtest has 71 tests, all pass.

(Note that I ran the test against Frameworks 5.51, since I don't have 5.52 installed yet)

sasch added a subscriber: sasch.Dec 31 2018, 2:54 PM
sasch added a comment.May 3 2019, 11:30 PM

What's blocking here to merge this in Master?

dfaure added a comment.May 6 2019, 2:50 PM

I don't object to the patch, but maybe the properties dialog (and other users of this job) should indicate that the number being shown is "sum of file sizes, not actual disk space usage, which is larger" (blocks, inodes, directory sizes...)

bruns added a subscriber: bruns.May 7 2019, 4:24 PM

I don't object to the patch, but maybe the properties dialog (and other users of this job) should indicate that the number being shown is "sum of file sizes, not actual disk space usage, which is larger" (blocks, inodes, directory sizes...)

or smaller - compression, deduplication ...

davidedmundson abandoned this revision.May 7 2019, 9:18 PM

I put it up as an RFC, as I spent too long debugging why I had a difference.
I don't like that we're in this uncanny valley of neither disk space nor sum of file size, but maybe I should do it properly and show two values (or one in a tooltip or something).

dfaure added a comment.May 7 2019, 9:20 PM

Well, this patch can still go in then, it makes one of the two values correct :-)

sasch added a comment.Jun 27 2019, 6:28 PM

@davidedmundson any reference or version when you plan to make this "live"?