A directory can be added multiple times to the config by just adding a
trailing slash to it, as the config removes any trailing slashes before
storing it, and the contains(path) will not match.

Normalize the path to the same format as used by the config and use it
for both adding and removing it.

While using it for removal is not necessary, it allows to use both
variants and matches the "add" behavior, where both variants have the
same effect.