build: Try to mitigate a race in the build order
This is known to require to delete the build directory and is
https://gitlab.kitware.com/cmake/cmake/issues/17357