Run simple functionality tests with actual webmail services in CI
Open, NormalPublic

Description

We should be running those tests ideally with packaged versions.

I have a enhancement to "sinksh selftest" in mind that allows testing of:

  • IMAP create mail
  • IMAP sync
  • Send mail and receive in IMAP inbox
  • create and sync via CalDAV/CardDAV

The tests don't have to be extensive, just the very basics.

We should then run these tests with a couple of supported services directly in the CI before publishing (this will require test accounts for that purpose).

Another benefit will also be that we then have a standard testbench for people to tests compatibility with whatever their service of choice happens to be,
and that should make is easier for us to figure out whether we have a compatibility issue that we have to fix on our end, or whether the service just happens to be incompatible somehow.

cmollekopf triaged this task as Normal priority.