Weird bandwidth issue with double NAT and Windows clients

I have the following issue and literally tried everything already and cant figure out, what's the root cause of this, how it is explained, and then also how to maybe fix it.

Setup:

Cable Router 192.168.0.1 <-> OpenWRT router, 192.168.0.2, 10.0.0.199 <-> Lan 10.0.0.x

So I am using double NAT, masquerading the WAN port of the OpenWRT router.

The issue now is, that downloads randomly slow down on all Windows clients after a few seconds. Starting at max speed of 500mbit and then slow down rapidly to around 60-120mbit. Sometimes it randomly just works for 20 minutes or so and then is broken again. Also then it can happen the download randomly is just stuck at 5MB/s or something over and over again.

You can see the issue happening here:

Windows client on lan side:

Starting the same download on the OpenWRT router with no issue:

Also with another Linux client, a NAS, on LAN side too, no issue:

It seems to be a weird issue exclusively to Windows and with double NAT in this setup.

The total weird issue though is, that some servers dont trigger this issue, even on the Windows clients, for example:

I have totally no idea, how this could be explained.

I tried everything so far, including resetting the tcp stack of Windows with no change.

There is no SQM / QoS set up on the OpenWRT router.

If anyone have an idea, how this can be explained and fixed, I would be very welcomed about that!