Yes,
Tried with packet steering and irqbalance, same result, but, both of them are not needed in order to reach gigabit in 21.02.1 stock
The device is wrt3200acm dual-core at 1.8 Ghz
Could you please let me know any test where i can see the % idle in output?
run
top -d 1
in a terminal, then run a speed test, look in the output for:
Mem: 83172K used, 397268K free, 640K shrd, 12036K buff, 15036K cached
CPU: 0% usr 4% sys 0% nic 95% idle 0% io 0% irq 0% sirq
here it shows 95% idle
1 Like
The lowest result was 42% idle, with the download of 761Mb and upload 803
Ok, likely this means you have 2 cores and one is saturating, RPS may not be working.
@dave14305 Now that I have the dnsmasq + nftset deal worked out on my owrt box, I can confirm that IPv6 DSCP is getting set on inbound packets with my setup. I'm using the map dscpct as defined here: NFtables and QoS in 2021 - #214 by dave14305
This is a packet from an active YouTube stream where I have classified googlevideo.com as CS3:
r1---sn-p5qlsndr.googlevideo.com 2607:f8b0:4004:37::6
FWIW, I have switched to using just CS1, CS0, CS3, and CS4 since these align nicely well with CAKE's implementation.
Could you run this command to get a feel for the marks being saved for IPv6 connections?
grep ^ipv6 /proc/net/nf_conntrack | grep -oE "mark=[0-9]+ " | sort | uniq -c
Thanks!
Sure thing!
root@OpenWrt:~# grep ^ipv6 /proc/net/nf_conntrack | grep -oE "mark=[0-9]+ " | sort | uniq -c
89 mark=0
1 mark=1660944384
20 mark=33554432
1 mark=50331648
And here's IPv4 just for comparison:
root@OpenWrt:~# grep ^ipv4 /proc/net/nf_conntrack | grep -oE "mark=[0-9]+ " | sort | uniq -c
110 mark=0
1 mark=2197815296
27 mark=33554432
15 mark=50331648
3 mark=587202560
For anyone not decimal to hex savvy:
Summary
root@OpenWrt:~# grep ^ipv6 /proc/net/nf_conntrack | grep -oE "mark=[0-9]+ " | cut -d'=' -f2 | while read line; do printf 'mark=0x%x\n' $line; done | sort | uniq -c
35 mark=0x0
21 mark=0x2000000
1 mark=0x63000000
1 Like
So a lot of the IPv6 connections aren't getting processed at all by the ct rules (mark=0), others are getting saved with CS0 (mark=33554432).
I'm following you. I'll look through my ruleset again in a bit and see if I can see anything obvious as to the mark=0.
FYI, I turned on my Apple TV (to Netflix) and saw a significant uptick in marks:
IPv6 Marks:
64 mark=0x0
34 mark=0x2000000
9 mark=0x63000000
IPv4 Marks:
112 mark=0x0
24 mark=0x2000000
2 mark=0x23000000
13 mark=0x3000000
1 mark=0x83000000
Isn't mark=0 indicative of BE/CS0? I wonder if I've confused this test by having this included in my ruleset:
table netdev tagingress {
...
### EDIT BELOW: set device for wanin and for lanin
chain dscpwanin {
type filter hook ingress device eth0 priority 0; ## This is my WAN iface
ip dscp set cs0
ip6 dscp set cs0
jump porttags
}
If it indicated BE/CS0, it would take that zero value, shift it by 26 bits and OR it with the fixed bit (logic from ctinfo_layercake.qos).
# echo $(( 0 << 26 | 0x2000000 ))
33554432
These definitely seems suspicious, the fact that the dscp on ipv6 crosses a byte boundary and that these commits say that stuff can "hopefully get fixed in an incremental patch" makes me think maybe it dropped off someone's radar and never got fixed?
It might be worth it to ask on the nftables mailing list.
I tried several times to subscribe today but vger never replied to me. Now it occurs to me that maybe I can send without subscribing first.
I'm wondering if the map I declared needs to have the ip6 dscp values shifted left by 6 bits to account for the double-byte key size alluded to in the commits. Can't test it now as I've switched to the old iptables build for the weekend.
At what point in the ruleset would you shift left 6 bits? If it's something easily conveyed, I'd be happy to add it to my ruleset and pass back the results.
I just noticed something... shouldn't chain qos_sqm_remap also contain a ct mark set for IPv6 like this?
chain qos_sqm_remap {
# Add rules to modify non-zero DSCP incoming from LAN
# Convert the current DSCP value to an equivalent conntrack mark using the map
# Then save it in the high bits of the mark for restoration with act_ctinfo
ct mark set ip dscp map @dscpct counter
ct mark set ip6 dscp map @dscpct counter
ct mark set ct mark lshift 26 or 0x3000000 counter
}
It'd be in the hard-coded values in the dscpct6 map, but I don't think that even makes sense since the type is already specified.
Just for the hell of it, let me know if these are accepted (I've only shifted the CS values).
map dscpct6 {
typeof ip6 dscp : ct mark
elements = {
cs0 : 0x00,
0x200 : 0x08,
0x400: 0x10,
0x600 : 0x18,
0x800 : 0x20,
0xa00 : 0x28,
0xc00 : 0x30,
0xe00 : 0x38,
be : 0x00,
af11 : 0x0a,
af12 : 0x0c,
af13 : 0x0e,
af21 : 0x12,
af22 : 0x14,
af23 : 0x16,
af31 : 0x1a,
af32 : 0x1c,
af33 : 0x1e,
af41 : 0x22,
af42 : 0x24,
af43 : 0x26,
ef : 0x2e
}
}
Then add a rule for ip6 immediately after the ct mark ip dscp rule.
ct mark set ip dscp map @dscpct
ct mark set ip6 dscp map @dscpct6
ct mark set ct mark lshift 26 or 0x2000000
Save a backup of your file since I don't think this is actually going to work. But I'm more comfortable wrecking your router than my own. 
Not so good, but I'm not dead in the water 
root@OpenWrt:~# nft -c -f /etc/nftables.conf && service nftables restart
/etc/nftables.conf:318:13-17: Error: Value 512 exceeds valid range 0-63
0x200 : 0x08,
^^^^^
1 Like
Yeah, it was a wild idea that sounded better in my head.
One workaround might be to collapse dscp values down to a small set and then have 3-4 explicit rules to set the CT mark explicitly. Until the map issue is resolved.
1 Like
As I was tinkering this evening, I noticed I stopped seeing CS3 marks on my YouTube traffic. I methodically backed out a couple changes I made and found that chain qos_sqm_remap is setting packets back to CS0.
I realized that after I added ct mark set ip6 dscp map @dscpct counter to qos_sqm_remap, that when I started losing my CS3 mark on my test video traffic.
Right now, I have the qos_sqm_remap rules commented out
Summary
chain qos_sqm_remap {
# Add rules to modify non-zero DSCP incoming from LAN
# Convert the current DSCP value to an equivalent conntrack mark using the map
# Then save it in the high bits of the mark for restoration with act_ctinfo
# ct mark set ip dscp map @dscpct counter
# ct mark set ip6 dscp map @dscpct counter
# ct mark set ct mark lshift 26 or 0x3000000 counter
}
and here is the spread of marks I am now seeing:
IPv6 Marks:
41 mark=0x0
20 mark=0x2000000
1 mark=0x3000000
3 mark=0x63000000
IPv4 Marks:
107 mark=0x0
42 mark=0x2000000
1 mark=0x23000000
2 mark=0x63000000
2 mark=0x6b000000
2 mark=0x83000000
Still not great (or even right?) but at least I'm seeing marks on packets while Wiresharking and seeing CAKE doing its thing.