git: Ensure in right srcdir if a fallback to git-clone happens.

This apparently worked accidentally in most cases but it doesn't
interact well with options like 'remove-after-install' from previous
module builds.