I have solved it!
Here is the solution for others:
set the wireguard interface with a higher metric then the real wan interfaces in "network > interfaces" (example main wan = 10, other wan = 20, wireguard = 30)
setup the 3 interfaces (main wan, other wan, wireguard) in "network > multiwan manager > interface" and the 3 members in "network > multiwan manager > member". The trick here is to set the "Flush conntrack table" option to both "connected" and "disconnected" for the two real wan interfaces.
setup a real policy for the 2 real interfaces (failover or loadbalanced. i only tested failover) and a dummy policy for the wireguard interface in "network > multiwan manager > policy"
this is the second trick: create a first rule that routes all traffic with the destination address of your wireguard peer (vpn endpoint) to the failover policy, then create a second rule that routes all traffic (destination address 0.0.0.0/0) to the dummy wireguard policy
finally, set a keepalive interval for your peer in "network > interfaces > wireguard interface > peer" (for example 10 seconds)
Now, when my router boots, the wireguard tunnel is enstablished through the main wan. As soon as the main wan goes down, within a minute (you can tune the mwan3 parameters to shorten this interval) the tunnel is re-enastablished through the backup wan. When the main wan goes back, within a minute the tunnel is re-enstablished through the main wan.