[Panel Configuration] Make sure "More Settings" button gets unchecked whenever the window hides
Not just when we manually close it