Currently, if Yakuake loses focus when the user clicks away from the main window the window stays open even when the "Keep open" option is switched off. This happens only under Wayland, X11 behaves correctly.
Yakuake seems to be built a lot around X11-specific behavior. This is the only way I could think of that works correctly with Wayland.