Any idea what’s going on here @dave14305?
No, but it appears to be one of the pitfalls of only relying on ct state instead of the conditional bit.
To be more clear, this is only affecting traffic on the forwarded port (55550). The initial connection between peers is marked CS1 but subsequent traffic flow is marked CS0.
My use of ‘skuid’ at the client doesn’t appear to be an issue either. I observe the same thing when setting it up by MAC from within cake-qos-simple.
I think you’d only need second line then, right?
And it’s fine it’s just I believe you lose the potential benefit of the conditional set if that is indeed actually faster - and actually I wonder whether checking and then conditionally setting based on the outcome of the check really is faster than just blindly setting?
Maybe someone with more knowledge than I can answer that.
Either of the commands works. I don't use both. Sorry I wasn’t clear. I’m currently using the first one.
oifname wan ct state new,untracked,established goto classify-and-store-dscp
After observing what was happening I added ‘established’ and it worked. Then I looked at a previous version you released that didn’t include ‘ct state’ and noted that both worked.
Ah, got it! Thanks. What is the significance of ‘established’ I wonder? Should we perhaps make it the default if it still works for our normal use cases, but also fixes special cases like yours?
With a port forward, the ct state is new on the first inbound packet from the wan. But on the reply packet from the lan, the ct state will move to established. But the second packet is the first one seen by your dscp chains.
Hmm. Isn’t that second packet associated with LAN response the one that should set DSCP for that connection?
I’m not sure it’s a special use case. I haven’t tested it on Windows but I think this could affect anything that requires an open port that you’d want to adjust DSCP value. Torrenting comes to mind which is a perfect use case for cake-qos-simple.
BTW – I’ve come to really like ‘skuid’ on Linux clients. It allows you to adjust sensitive ports based on user/application. Once you get the user file permissions set, it’s pretty close to a Windows setup. I create a “bulk” “video” and “voice” user and add the appropriate rules to the client firewall. Now I can call any application with any priority.
I found it easier than network-namespaces and the forwarding rules required to poke at a running application… it just didn’t fit well in my head.
Sure, but if you add established state to the rule, you end up reprocessing every egress packet. Whether that’s desirable or not is subjective.
I’m just thinking out loud here but would ‘related’ work and would that be more efficient if it did? I’m happy to test if you think it’s worth it.
ct state established ct direction reply would be more granular than all established. Related doesn’t really apply here (I don’t think).
1 Like
I hadn’t taken a look at performance before reporting this issue. Adding ‘established’ is definitely not worth it. Both cores of my Belkin RT3200 are pinned to the max with my 80 Mbps connection.
I tested with ‘related’ and it didn’t work. I also tested with ‘ct state established ct direction reply’ but performance was still horrid.
The most efficient and effective option is to not use ct at all.
oifname wan goto classify-and-store-dscp
The performance hit appears to be 5-10% more (hard to tell exactly) than ‘ct state new,untracked’. The performance hit is worth it to me and I don’t think this limitation existed in previous iterations of cake-qos-simple. The last time I updated was in April.
Ah cool. So there we have it, the conditional set gives a very slight performance increase, and doesn’t work in at least one use case. And more advanced conditional sets that cover that use case give worse performance. That seems logical, mindful that the benefit of dispensing with setting every packet is offset by the cost associated with the conditional check.
And I’m left wondering what to leave as the default. Perhaps ‘ct state new,untracked’ as is set now, but just add an entry in the README explaining that this is a performance tweak that won’t work in certain cases like port forwarding and that testing is needed and possible replacement with:
‘ oifname wan goto classify-and-store-dscp`.
Any thoughts?
I consider cake-qos-simple in its current form an advanced user tool so I don’t know that there is a wrong answer.
As a user, I prefer ‘just works’ as the default and ‘performance tweaks’ as options. I think this is what most users would expect. It seems to be the most common approach used in the software industry.
1 Like
Maybe there is another way here. How about:
- ct state new, untracked → run classifier
- ct state new, untracked, established → run classifier
- else → skip classifier
Can that somehow be implemented in nftables in an efficient way that beats just:
oifname wan goto classify-and-store-dscp
I wonder?
I had originally implemented the evaluation of a conditional bit - see the code replaced with this commit:
That is:
chain classify-and-store-dscp {
meta nfproto ipv4 ct mark set (@nh,8,8 & 252) >> 2
meta nfproto ipv6 ct mark set (@nh,0,16 & 4032) >> 6
ct mark set ct mark or 128
}
oifname wan ct mark & 128 == 0 goto classify-and-store-dscp
and:
tc filter add dev "${ul_if}" parent ffff: protocol all matchall action ctinfo dscp 63 128
tc filter add dev "${ul_if}" parent 1: prio 1 protocol all matchall action ctinfo dscp 63 128 continue
Would that have also suffered from the same issue? I'm thinking yes. But this implementation might give one of us more inspiration.
Any thoughts or ideas @dave14305?
I think it would have worked fine. Regardless of ct state, it would have classified the first (reply) packet bound for the wan interface, and it would be done.
I don’t know how the ingress filter to restore the DSCP behaves on the first inbound packet from WAN when there is no DSCP yet stored in conntrack, or no conntrack entry at all at that ingress point.
It just dawned on me that Comcast recently changed there default from CS1 to CS0. It’s likely that I wouldn’t have noticed any issue back in April since I was seeing what I expected with my rule… CS1.
Can you follow @dave14305's explanation above? I'm admittedly working for a legal hearing and so not giving this my full attention. I think for the benefit of myself and other readers it would be super helpful if you could outline what is happening with your port forwarding and classification in terms of packets and their direction and why the present mechanism fails but the older mechanism would not have failed. Perhaps we should simply go back to the old mechanism. It'd be good to know how the performance of that compares. Could you test?
I retested using the same version I used prior to updating to your most recent commit (I restored from a backup).
I can confirm that @dave14305 is correct. The old version did work with my port forwarding rule and the Comcast change made no difference.
WRT performance they're similiar. With your most recent version including oifname wan goto classify-and-store-dscp being slightly better. Possibly due to using 'postrouting' in your more recent versions instead of 'output' and 'forward'?
I’m not on his or your level. If you still need detailed info would you or @dave14305 please provide me with a tcpdump command or instructions on how to give you exactly what your looking for? I’d gladly report back.