We've been having the configure button on notifications for a long time but so far it would only ever invoke KNotification configuration UI with no way for an application to provide its own, more versatile or extensive, UI.
When an action named "settings" is present in the notification the "configure" button will instead invoke this action.