Implement client-side threading cache

Authored by dvratil on May 30 2016, 8:13 PM.

Description

Implement client-side threading cache

The cache is a simple QHash<ChildId,ParentId> that we build when
folder is opened for the first time and persist it in a cache
file for each folder. Aggregation state (grouping and threading)
is also stored in the cache file so we can check whether the cache
matches the current aggregation configuration before we load it.
If the aggregation has changed we simply discard the cache file
and perform full un-cached threading.

There is a second QHash<ItemId, Item*> in ThreadingCache which
we fully populate in Pass1. Pass1 may already yield some perfect
matches thanks to the Child-Parent cache, but only if the Parent
Item* has already been inserted into the second QHash - this does
not happen much as we retrieve Items in reverse order so children
will arrive before their parents. If we can't do a perfect match
but we have the Parent ID available in cache we just move the Item
to Pass2 and go on to next one. Otherwise we let Pass1 do full
evaluation which will insert the Item to cache if perfect match
is found or queue it for Pass2.

In Pass2 we have the second QHash fully populated so we perfect-
match all Items using the ChildId->ParentId cache. Items that are
known to not have a parent (i.e. thread leaders) are scheduled for
Pass4, Items that are not available in cache are sent for full
evaluation through Pass2 (and Pass3 if needed) and inserted into
the cache.

Finally Pass4 performs grouping. There is no caching for that
right now because the grouping is dynamic and there are no real
stable identifiers for group headers. We could possibly cache all
the fixed groups (i.e. sender, receiver or subject) and maybe even
fixed-date groups (e.g. "January 2014", "March 2015") and only
let Pass4 calculate dynamic groups ("May", "Two weeks ago",
"Yesterday", ...) but the gain here would be minimal as we are
usually dealing with very few groups. The real bottleneck of
Pass4 is beginInsertRows()/endInsertRows() as threads are shifted
around - something to look into next.

On my system this speeds up opening a folder with 50000 emails
by ~30%.

Differential Revision: https://phabricator.kde.org/D1636

Details

Committed
dvratilMay 30 2016, 8:20 PM
Differential Revision
D1636: Implement client-side threading cache
Parents
R94:c2dc0811dc2e: Astyle kdelibs
Branches
Unknown
Tags
Unknown