Destroying the ContrastInterface on the server side before the client has
a chance to cleanup results in a protocol error:
wl_display@1: error 0: invalid object 7

Which would terminate the client. If we would not destroy the resource,
but only delete the ContrastInterface it could result in
heap-use-after-free.

So just don't do anything, the client needs to cleanup which will result
in the ContrastInterface being deleted.