TP-Link Archer C2600 ping spikes after upgrade

After upgrading to the latest version (19.07.3) I started noticing rubber banding in games and discord calls breaking up.

When tested using my desktop I got ping spikes as high as 1000ms.
This also happened when pinging from open wrt itself, even though the cpu usage is low.
I am running open wrt as a switch + access point without any dhcp server or firewall, I have a separate server computer working as a router.
Even when I ping the server from the open wrt router I get these ping spikes.

How can I debug this issue, was any change introduced recently?

Was it fine before you upgraded, and if so, which version were you running that was fine?

I didn't have any problems before. Found an old .bin with version 19.07.2, it might be that version.

Here is a graph over the ping spikes.

It seems like the problem was on my next hop (the debian router).
Will share the solution if I can find it.