I can't help you with nftables as this stuff is way beyond my skill level but I would suggest that you also take a look at @ldir's approach where you would be able prioritize your gaming/console traffic via DSCP's for in+egress.
I think that his approach is quite clever + straighforward and you should be able to get solid results if you prioritize your gaming traffic via DSCP's.
See: SQM cake: traffic prioritisation
There is also a diffserv4 script, so you don't need to patch cake in order to give his approach a try: https://github.com/ldir-EDB0/sqm-scripts/commit/e701873cf6393d056360dbc4f174db602ca02f09
I'm not 100% sure what the requirements for this approach are but I think you only need kmod-sched-ctinfo + kmod-sched-connmark (on top of sqm) to get it working.