You could use 802.11k for asking the client what is in his range and send some 802.11v fast bss transition to the client to switch to 5Ghz. Then 802.11r would work too. 
There are different milestones you have to do:
-
Network-Manager Patch has to be accepted for supporting 802.11r (if u use network-manager...)
- Fast BSS Transition be reachable via ubus
When some daemon could be implemented independent of the hostapd. Personally, I think this would be the nicest solution.
I experimented with deauthing clients, trying to supress probe responses, deny authentication tries. There are always corner cases. And often you need some "hearing map" approach. And this only works if you have some static scenario or you can use some antenna to scan for clients. In addition, you have mac-randomisierung when the clients scans and other challenges you have to cope with...
There are techniques sending a channel switch announcement
to the client without switching the channel. For example the 2.4 GHz and 5 GHz run with the same BSSID and the 2.4 GHz would send some channel switch announcement to the client that it now switches to the 5 GHz frequency. However, the interface won't switch the frequency but the client will switch. to the new interface which has the same BSSID. This is some kind of seamless handover.
In addition, 802.11q introduced important information about the channel utilization and the number of connected clients. With that information clients could make good decisions.
Why load balancing matters?
If you have several APs in your network cooperation becomes important. Clients often do not know how the backhauling looks like. What happens if the AP is connected via powerline link, or is some part of a mesh network.
In addition, SDN is some feature trend that can be optimized when APs have the ability to steer clients to different APs in the network.
I wrote more in a blogpost. 