Ensure we always reset the external reference override

Summary:
So far there were apparently cases where this got stuck on enabled even
when switching between emails.

BUG: 394554

Reviewers: cgiboudeaux, knauss, mlaurent

Reviewed By: knauss

Subscribers: kde-pim

Tags: KDE PIM

Differential Revision: https://phabricator.kde.org/D13096