Wasn't as easy I thought it would be ;) The "Configure Kdenlive" dialog is borrowed from the Kconfig dependency so it doesn't have an ui file in the source code of Kdenlive.
Since this is a framework/dependency related "bug", I figured it would have appeared in other KDE products as well and after some hunting - I found this revision on Plasma.

I basically implemented what they came to conclusion to, in /dialogs/kdenlivesettingsdialog.cpp
And it works out successfully. We can adjust the size if we want by adjusting the parameters, but I find the dialog now perfectly sized.