Don't keep the default tile data blocked from swapping

We don't access the raw data of it usually. And is cases we do
we should block it separately.

This patch is needed to be able to migrate the leftover of tiles
to the new pool when a document is closed.