[Kickoff] Remove old arrow icon code

This wasn't visible in Breeze but might break older themes that still ship with
this icon. The way it was implemented, just added ontop of the delegate, wasn't
particularly good anyway.

BUG: 357411
FIXED-IN: 5.6.0