Indexing encrypted mails
Open, Needs TriagePublic
Actions

Description

Currently we can't index encrypted mails, so we can't search for text inside the encrypted mails. This is a pitty, cause with wie cant to push end-to-end encrypted mails, we need to give the user a possibility to search for encrypted mails like the unencrypted ones.
Two issues we need to solve first:

while indexing mails, we need to decrypt them -> this will trigger the Keyentry dialog for the user a kind of random
how to store the indexes for encrypted mails securly. My thought is, that we should store the indexes also encrypted with the same key they are encrypted for. So we do not leak information into another domain. Is Plasma Vaults a solution (@ivan)?
for a propper indexing, we need to depend on mimetreeparser. Because maimetreeparser has already the logic how we need to parse the mail to get the "interesting content" and especially with encrypted mails, we should also understand the structure while indexing. And if we solve "T742 Add Memory Hole support", than we should also index the updated headers and not the dumb ones, that are accessible from outside. But therefor we would need to split mimetreeparser out of messagelib, otherwise we create a circular dependency.

Related Objects
Search...

		Status	Assigned	Task
		Open	dvratil	T7014 Make Indexing Great Again
		Open	None	T8447 Indexing encrypted mails

knauss created this task.Apr 6 2018, 7:00 AM

Related to T7014

I have a question to that because I encountered this issue years ago when I used Thunderbird with Enigmail. I assume the indexing we are talking about here takes place in the local mail client, not in a web client, right? I wonder why incoming encrypted mails are also stored encrypted? Why not always decrypt those mails permanently? This would prevent losing old mail because of lost private key and there would be no problem with indexing. In this Enigmail bug report (https://sourceforge.net/p/enigmail/bugs/1/) from 2003 this request was made. The main reason for not implementing it is a technical one ("this feature is blocked by necessary Thunderbird changes") not fundamental ones. I thought the goal of end-to-end encryption is protect the data when it is underway and not also on the end-points themselves. Are there any reasons why KDE PIM does not decrypt incoming encrypted mail permanently? (or is there already and option to do that? Then sorry for the fuzz)

@gregormi This is only partially related. We already have an option how to automatically decrypt all incoming emails in KMail, although it's not very intuitive to set up. You can create a local filter (Settings -> Configure filters) for all incoming emails and choose the "Decrypt" action. The reason it's not done by default is due to privacy concerns. For example you may want to keep your personal emails always encrypted on your employer's computer so they can't read your private emails.

knauss added a parent task: T7014: Make Indexing Great Again.Apr 6 2018, 8:54 AM

@gregormi also think about the privacy impact. If you use an unsecured laptop, the mails can be accessed without any password etc. Also when you use IMAP the unencrypted mails would saved back to your mailserver and would be accessible there. It might be usescases for that. But I think the propper way is to make sure, that we can handle the encrypted mails all the way down without removing the encryption.

Vault does not support public-private key encryption - it is out of the project's scope. And it would be quite strange for the user to see items like "KMail Index for ABCDEFGH01234... key" in the list of existing vaults. :)

Parts of Vault could probably be reused and wrapped in an API to allow creating/opening/closing an encrypted directory. The public-private key encryption could be an addon - generate a random password for the vault, and keep it in a gpg-encrypted file. When it needs to be opened, the user would be presented with the same dialogue used for mail decryption.

As for the indexing and other stuff. The encrypted message can be indexed when the user opens it - that way, there would be no surprising password dialogs popping up.

And, when searching, the results could have a "Search also in encrypted mail" button which, when clicked, would ask the user to decrypt the indices for encrypted mail.

@knauss
"Also when you use IMAP the unencrypted mails would saved"

Not if the decrypted data is only saved as a internal message meta-data in kmail/akonadi, while the original encrypted message is preserved to be synced back to imap.

@ivan: tanx for the input

In T8447#136984, @ivan wrote:

Vault does not support public-private key encryption - it is out of the project's scope. And it would be quite strange for the user to see items like "KMail Index for ABCDEFGH01234... key" in the list of existing vaults. :)

Parts of Vault could probably be reused and wrapped in an API to allow creating/opening/closing an encrypted directory. The public-private key encryption could be an addon - generate a random password for the vault, and keep it in a gpg-encrypted file. When it needs to be opened, the user would be presented with the same dialogue used for mail decryption.

what technology do you use for vault? because we are talking about multible GB big files, that need to be accessed. Is it luks like technoloy (block-level encryption)?

As for the indexing and other stuff. The encrypted message can be indexed when the user opens it - that way, there would be no surprising password dialogs popping up.

mmh but what to do if your are starting using kmail, than you have already mails, than you would have to open all of them.

And, when searching, the results could have a "Search also in encrypted mail" button which, when clicked, would ask the user to decrypt the indices for encrypted mail.

good idea.

@knauss
"Also when you use IMAP the unencrypted mails would saved"

Not if the decrypted data is only saved as a internal message meta-data in kmail/akonadi, while the original encrypted message is preserved to be synced back to IMAP.

let's cut this discussion, as it is not the topic here. Yes it is possible, but we would need to make super sure, that we do not accidentally sync back the data to IMAP. This is a high potential of leaking protected data.

In T8447#136980, @dvratil wrote:

@gregormi This is only partially related. We already have an option how to automatically decrypt all incoming emails in KMail, although it's not very intuitive to set up. You can create a local filter (Settings -> Configure filters) for all incoming emails and choose the "Decrypt" action. The reason it's not done by default is due to privacy concerns. For example you may want to keep your personal emails always encrypted on your employer's computer so they can't read your private emails.

Thanks for the hints. I understand the special privacy concerns on employer's computers. I look from the perspective of home users which want to communicate privately. As I tried to use mail encryption in the past and the first thing that happened to me was a lost private key and therefore lost archived mail. I know of other people which had the same problem and therefore stopped using end-to-end mail encryption altogether. Depending on which group (professionals or home users) are the main user base, one might consider changing the default behaviour: let the professional have a switch to enable stronger security measures if needed. Yes, this is only partially related to this task -> Shall I open a separate task or wish bug report for further discussion?

In T8447#136983, @knauss wrote:

@gregormi also think about the privacy impact. If you use an unsecured laptop, the mails can be accessed without any password etc.

This does also apply to mails which were already received without encryption, right? Then logically all mails should be stored encrypted. For me, securing a local machine is a related but separate topic.

Also when you use IMAP the unencrypted mails would saved back to your mailserver and would be accessible there. It might be usescases for that. But I think the propper way is to make sure, that we can handle the encrypted mails all the way down without removing the encryption.

Thanks for giving the IMAP case. This makes sense. Though I still think that having encrypted mail in the private mail archive establishes an additional barrier to use end-to-end encryption at all (and/or increases the risk of losing mail for inexperienced users).

neofytosk removed a subscriber: neofytosk.Apr 6 2018, 9:32 AM

@knauss

Vault uses FUSE, no block-level for the time being. I was toying with Tomb and preallocated fixed-size containers in the filsystem which uses Luks in the back, but I'm not sure it can be made to work properly.