Even when a user scours the system settings + desktop config looking how to disable this button, the user will have no indication this feature is called the "Desktop Toolbox". The button itself will show the current activity when not placed in a corner (eg: "Default" activity). When you click the menu, it does not have a "Desktop Toolbox" label either.
So I suggest that we show the icon next to the checkbox. Here's what it looks like with all 4 combinations of Breeze Light/Dark Color Theme + Desktop Theme.