Disabling learning and turning the switch into a hub did indeed fix the issue. I added additional debugging output as I would like to figure out why entries from the switches FDB can't be removed, so when an entry is added or removed that is now logged in dmesg.
# bridge fdb add 11:22:33:44:55:66 dev wan self vlan 1
RTNETLINK answers: Invalid argument
Doing
# bridge fdb add 11:22:33:44:55:66 dev wan self
doesn't log anything, but I believe it should be added to the switch's FDB as I specified the self option? At this point I have seen my debug outputs during boot, but never afterwards, not even when roaming and the assisted learning should be doing it's thing....
FYI here is the patch to enable macs seen on non-physical interfaces to be removed from the fdb.
This is on top of Robimako's patch
Appears to work ok with my limited testing .