We want to be able to fulltext search for mails.
Subject only would be a good start but more is welcome.
Description
Description
Status | Assigned | Task | ||
---|---|---|---|---|
Resolved | cmollekopf | T1746 Fulltext search | ||
Resolved | cmollekopf | T8038 Conversation view search | ||
Resolved | cmollekopf | T8087 Deal with failing transactions | ||
Open | None | T8095 Server side searching |
Comment Actions
Options are:
- xapian
- sqllite
- baloo?
There is also whoosh which is a fulltext index on top of lmdb, but written in python (but could provide inspiration)
Comment Actions
Xapian is likely an easy candidate with as it supports single-writer, multi-reader semantics. We could just dump the full text blobs in there, and then do any additional filtering (flags, ....) later on using the indexes we have in lmdb or the actual data available.
Comment Actions
Search will work along those lines:
- A search query is issued, i.e. a simple fulltext query matching "anything" (meaning we'll have to define which properties that entails exactly).
- The search is executed against the local search store, resulting in some immediate results. This is of course only possible for already downloaded content (that is also being indexed), so worst case this will yield no results.
- Simultaneously the search is also sent to the resource in the form of a search command. This will trigger a search in the backend.
- The results from the backend are transported back to the query and a the download of any match is triggered (so we have subject etc available).
- As soon as the result is downloaded with sufficient data it can become part of the visible query result.
Some considerations:
- One key problem will be to get consistent results online and offline. It's okay to get less results offline vs. online, but given all data the results should be consistent.
- While we can do as-you-type searching in the local store, this will probably not work for the backend search.
- Started searches that become irrelevant (because the query was changed etc.), will need to be aborted to avoid getting stuck on outdated searches (we will only be able to execute a limited amount of searches in parallel.
Comment Actions
We'll likely start off with fulltext query only when connected to the server. Locally you can still filter by subject, sender etc.