Optimize per channel filter.
Don't create the transformation for every patch. The speed difference is very noticable if lightness is changed.