Only delete the contact if the file is actually deleted
KDirWatch will notify a deleted and created signal when a file is modified.