'case-sensitive' seems to be the correct spelling
Other applications (like Firefox) call it "Match Case".
I think "Match case" or "Case-sensitive match" are okay, but "match case-sensitive" doesn't sound good. I don't have a proper grammatical justification for that though.
"Match case" is acceptable. It refers to "typecase", the box in which movable type was stored. The convention was that capital letters were stored in the uppermost of two boxes, hence "upper case" and "lower case".
"Match case" or "Case-sensitive match" sound natural, but if you want the word "match" to be first use "Match case-sensitively".
The hyphen is a stylistic choice, apparently British English uses hyphens and American English does not. I'm from Canada, and both methods are used.
If it's grammatical justification you need: "case-sensitive" functions as an adjective, which requires a noun. The ambiguity of the missing -ly suffix or the missing noun is what makes it sound weird.