Only select the closest swatch, not actually select the color.

Selecting the closest swatch forces colors in the palette to be used instead,
which might be interesting in the future, but now is mostly annoying.

This needs testing, no idea why this was used instead, might have
an underlying reason.