In D15939#337418, @ngraham wrote:

Works great, thanks for all these Baloo patches! They are very much appreciated. Code looks good. As with the others, please commit for Frameworks 5.52.

While you're cranking away at these, might I suggest a few others that are sorely needed:

• No apparent effect after removing a folder from the index; need to restart or kill all Baloo processes: https://bugs.kde.org/show_bug.cgi?id=373430 & https://bugs.kde.org/show_bug.cgi?id=353559

I can look at these, shouldn't be too hard.

• Baloo doesn't index or search symlinked folders: https://bugs.kde.org/show_bug.cgi?id=362226

I guess this link also should be added to previous two :)

As for symlinked folders - I once looked into, a year ago or so. As far as I remember, those can be really tricky (but I don't really remember all the details):

• There might be duplicated entries in index pointing to same file, which doesn't look good - if we index the same folder twice (symlink and original; but I guess we can always rely on resolved path, which should be unique - apart from hard links?)
• There might be recursive symlinks / loops, which should be handled (i.e. dir1/subdir -> dir2 and dir2/subdir -> dir1/; but it looks like QDirIterator can handle symlink loops?)
• There might be a symlink to a folder, which has some parent excluded from indexing
• ...and maybe more

I'll see if there is something not easily manageable.

In D15939#337534, @poboiko wrote:

As for symlinked folders - I once looked into, a year ago or so. As far as I remember, those can be really tricky (but I don't really remember all the details):

• There might be duplicated entries in index pointing to same file, which doesn't look good - if we index the same folder twice (symlink and original; but I guess we can always rely on resolved path, which should be unique - apart from hard links?)
• There might be recursive symlinks / loops, which should be handled (i.e. dir1/subdir -> dir2 and dir2/subdir -> dir1/; but it looks like QDirIterator can handle symlink loops?)
• There might be a symlink to a folder, which has some parent excluded from indexing
• ...and maybe more
  
  I'll see if there is something not easily manageable.

Adding to all the points you already mentioned, I am also against following symlinks:

• if a symlink points inside a hierarchy already included, there is nothing gained
• if a symlink points outside, it can be included explicitly using includeFolders

Prior to any implementation attempt I would like to see a specific use case for following symlinks, which is not already covered.

I've found the relevant bug: https://bugs.kde.org/show_bug.cgi?id=333678

In D15939#337757, @poboiko wrote:

I've found the relevant bug: https://bugs.kde.org/show_bug.cgi?id=333678

Hah, looks like you pushed a commit for it last year!

I'm not wedded to symlink support, but if we're not going to do it, we should close the bug with some good reasons.

Hah, looks like you pushed a commit for it last year!

I'm not wedded to symlink support, but if we're not going to do it, we should close the bug with some good reasons.

Well, I remembered that I did something on that, but I totally forgot what exactly. That's how I've found this bug - lurking through commit log :)

Adding to all the points you already mentioned, I am also against following symlinks:

• if a symlink points inside a hierarchy already included, there is nothing gained
• if a symlink points outside, it can be included explicitly using includeFolders
  
  Prior to any implementation attempt I would like to see a specific use case for following symlinks, which is not already covered.

So, it seems like it is not just about whether we should follow symlinks when indexing or not. There are apparently also some troubles with "Search From Here" in Dolphin...
I agree - I think we should start with looking what are the possible use cases for symlinks, and which of them work (or not). If the situation can be improved cheaply (i.e. without changing DB schema :), then it would be nice

Does this run at startup? If so, this will erase the entries of files on a removable volume not already mounted.

Does this run at startup? If so, this will erase the entries of files on a removable volume not already mounted.

Whoa, thanks for the notice. Not cool, forgot about it.
Probably need to add some checks in IndexCleaner, about the device.

But how does it work now? I mean, do files from not mounted devices pop up as valid search results?

...And that's why we committed this for 5.52. :-) Now we have a month to fix it--and we should make sure we do!

https://phabricator.kde.org/D11529 was already up for review, implemented the index cleaner and checked for removeable volumes before removing index entries. Exporting the storagedevices object was required: https://phabricator.kde.org/D15047. The alternative, implementing a path lookup function is here: https://phabricator.kde.org/D15843.

In D15939#341094, @smithjd wrote:

https://phabricator.kde.org/D11529 was already up for review, implemented the index cleaner and checked for removeable volumes before removing index entries. Exporting the storagedevices object was required: https://phabricator.kde.org/D15047. The alternative, implementing a path lookup function is here: https://phabricator.kde.org/D15843.

All these are far from submittable, so clean your stuff up first before making any further requests.

In D15939#341094, @smithjd wrote:

https://phabricator.kde.org/D11529 was already up for review, implemented the index cleaner and checked for removeable volumes before removing index entries. Exporting the storagedevices object was required: https://phabricator.kde.org/D15047. The alternative, implementing a path lookup function is here: https://phabricator.kde.org/D15843.

Sorry, I was not aware of those patches (didn't do my homework). I'll look at those!

UPD: but will those help? I mean, if device is not present at the moment, Solid won't pop it up, and index entries will be forgotten.
It seems like it will help only if device is present, but not yet mounted.

In D15939#341728, @poboiko wrote:

In D15939#341094, @smithjd wrote:

https://phabricator.kde.org/D11529 was already up for review, implemented the index cleaner and checked for removeable volumes before removing index entries. Exporting the storagedevices object was required: https://phabricator.kde.org/D15047. The alternative, implementing a path lookup function is here: https://phabricator.kde.org/D15843.

Sorry, I was not aware of those patches (didn't do my homework). I'll look at those!

UPD: but will those help? I mean, if device is not present at the moment, Solid won't pop it up, and index entries will be forgotten.
It seems like it will help only if device is present, but not yet mounted.

Yes. The volume is ignored even if the storage device is unplugged from boot.

Exercise:

1. Take a USB memory stick
2. plug it into one USB port
3. run "stat <mountpoint>"
4. unmount, plug into different port
5. run "stat <mountpoint>" again

You will get different device ids for each port. If you plug in a different stick, the device id will be there, but contents will not match and all old entries are removed.