Fix key refreshing in some cases

Let's say you edit the properties of a group member. Refreshing will delete
the group and show the group members in the list of keys, i.e. you will get
the group members twice. Pressing F5 and everything is fine again. So why not
do it right the first time? ;)