The TotalRandomColorSource spends a whole lot of time in color management code to properly do the whole color profile dance (i guess). For that purpose, it'd probably be better if the colorspace itself does the random color generation itself, rather than the TotalRandomColorSource learning about different channel formats so it can generate the right bit depth on its own.