Improve mail rendering performance
Closed, ResolvedPublic

Description

Loading mails is not exactly fast.
I guess one part is the WebView not being all that fast, so we have a slight delay before we can see the mail.
A quick check shows that the OTP is done generating HTML typically in ~5ms tops for regular mail (fancy graphical mails maybe ~10ms) , but as soon as we have a signature it takes ~500ms.
An encrypted mail takes even up to ~800ms.

Perhaps we can improve this to some degree, but otherwise we would probably have to improve it by processing the mails ahead of time. The primary usecase for fast switching is if you go through your list of mails I think, so we could i.e. always prepare a couple WebViews for the next mails and thus make switching feel much faster than we can realistically achieve otherwise.

It was thought that encryption with RSA keys was the bottleneck, but:

~|⇒ openssl speed rsa4096
Doing 4096 bit private rsa's for 10s: 1167 4096 bit private RSA's in 10.00s
Doing 4096 bit public rsa's for 10s: 75982 4096 bit public RSA's in 10.00s
OpenSSL 1.0.2g-fips  1 Mar 2016
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx)
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -Wall -O2
-g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
                  sign    verify    sign/s verify/s
rsa 4096 bits 0.008569s 0.000132s    116.7   7598.2
openssl speed rsa4096  20.04s user 0.01s system 99% cpu 20.055 total

which means we should be able to decrypt an email in a couple of milliseconds.

So the minimum bar for OTP to deliver a decrypted/signature verified version of the mail (excluding attachments, and excluding any network roundtrip to fetch keys), is 50ms on an average laptop (I see no reason why we shouldn't be able to reach 10ms or better).

If we have to fetch keys, they should be cached the next time, so everything greater 50ms should be a one off.

cmollekopf moved this task from Backlog to Milestore: MailClient on the Kube board.Nov 1 2016, 8:58 PM
mbohlender moved this task from Milestore: MailClient to 0.2 on the Kube board.Nov 8 2016, 1:28 PM
mbohlender edited projects, added Kube (0.2); removed Kube.
cmollekopf triaged this task as Normal priority.Feb 10 2017, 2:20 PM
cmollekopf lowered the priority of this task from Normal to Low.
cmollekopf moved this task from 0.2 to Backlog on the Kube board.Feb 21 2017, 12:40 PM
cmollekopf edited projects, added Kube; removed Kube (0.2).
cmollekopf lowered the priority of this task from Low to Wishlist.Mar 1 2017, 10:05 AM

The main bottleneck seems to be signature verification and decryption.

And the main problem with that seems to be "keyserver-options auto-key-retrieve" in gpg.conf. Without it things are much more responsive it seems.

There does not seem to be any way to override auto-key-retrieve programatically => documentation issue.

cmollekopf closed this task as Resolved.Mar 19 2021, 10:45 PM
cmollekopf claimed this task.