Need help regarding SQM

So for most mixed multi-flow use cases triple and dual will be quite similar in fairness and triple does not need to factor the direction into the equation. The dualmodes however need to take direction into account, hence dual-srchost and dual-dsthost. The advantage of the dual configuration is that it can give you strict per-internal-IP address fairness, while triple only approximates that. So in corner cases it is not straight forward to predict how triple is going to behave. But in the end this is a policy decision you, as admin of your network/router, will have to make.

1 Like

It’s switch0 setup. Internet<---->THIS MAIN ROUTER (OpenWrt) <----> Wifi-router(Default TP-link firwmare)
VLAN: Port status 1/ eth1 tagged/ LAN1-LAN4 untagged
Connection type: PPPoE (So I guess there is no need to specify overhead. Cake handles it automatically?)

Correct. So ack-filter only for egress.

I agree default RTT 100ms is the best choice overall & I shouldn't change it. But the thing is that even without load due to default 100ms RTT enemies are able to peek & shoot me before I could even react. I don't have empirical statistics to prove but I did a lot of tests while playing lan game & found putting less than 30ms RTT makes it a fair match. I think it has to do with tight target so SQM has to send packets without waiting for 5ms more?

Got it :smiley:

Alright so for this setup Squash DSCP on inbound packets (ingress) should be SQUASH.

But here we're avoiding DSCP tags on ingress flow (ingress shaping is disabled on both SQM instances) hence this option won't affect result at all right?

So go with default options for both ingress/egress?

sch_cake: Add DiffServ handling document suggests to tag gaming packets to CS4. I tried that but the end result was horrible even in unloaded network. CS6/CS7 works better for me.

No not at all, you always need to specify the overhead, otherwise you will only get what the kernel might automatically add (0 bytes for pppoe). What kind of link technology are you using?

So depending on where you live in relation to the game servers it might make sense to change these, but given your numbers I would leave this at 100ms.

That is not how this parameter works, you basically just tell cake how long it should give each isolated flow to respond to a drop or ECN mark before dropping/marking more, if this is below the real RTT the flow will never reach full speed even if it would be the only active flow, setting it to high is more benign for well responding flows and for non-responsive flows the fq (flow queueing) isolation will make sure, that the increased latency coming from non responsive flows is only experienced by those flows themselves.

SQM will not really wait on purpose, these 5ms is just what cake will acccept as standing queue under saturating load (in cake you can not set this directly, so if you really need a shorter target you also need to reduce interval, which you will pay for with reduced throughput for TCP flows with longer RTT than interval, again a policy decision).

Policy decision, you need to make, but unless testing what shenanigans my ISP does to DSCP marks, I normally squash them.

It will make all the special DSCP assignments in your firewall.user moot for ingress...

Sure.

I would assume this comes from your firewall.user. It appears that you are trying to combine a standard sqm-scripts installation with the bespoke qos-scheme from https://forum.openwrt.org/t/ultimate-sqm-settings-layer-cake-dscp-marks/ that is a bit tricky.

I guess you need to decide whether you want to go all in with the ultimate scheme or start (at least initially) with a more run-of the mill "white-bread" sqm-scripts install ;). I would personally start with the white-bread approach, if only to be able to appreciate all the goodness the ultimate-scheme buys you (as its complexity cost is obvious). But again, your network, your decisions.

ISP have setup their switch at our house. Till switch fiber connection followed by cat5e Ethernet connection to my router. I'll need help with this one too to calculate overhead then.

So basically it'll only matter under load understood :smiley:

I'm honestly confused on this one. I made this easy diagram to represent packets flow through SQM instances & firewall. Please correct me.

Firewall rules only apply right after SQM correct? The whole reason I went with this approach is that once incoming packets passes through either one of the SQM instances (ingress shaping selectively disabled) firewall rules (DSCP tagging) applies to all packets then goes to next SQM instance, here they are prioritize based on tags (thanks to layer.cake). Whereas, normally with basic single SQM instance for both ingress & egress prioritization of packets never happens inside the queue since iptable rules apply after SQM & not before it. Correct?

Your diagram is more or less correct, but ... and this is key... br-lan itself doesn't do any queueing, only the individual ethernet and wifi interfaces queue. So putting a SQM instance on it doesn't help.

in order to avoid this problem, I invented the veth combination. One of the veths is in the bridge, and the other is a kind of "injector" that can queue. So you modify the inbound route to send packets to veth0 where they queue, and then are sent to veth1 which receives them in the bridge and br-lan bridges them to the ethernet or wifi.

@moeller0, I am not really up for creating a package to instantiate my hfsc tuned QoS with firewall rules etc and a LUCI config... but if you know someone who has the skills and time to do that, I'd be happy to provide consulting assistance. Any ideas of someone we should tag?

2 Likes

But in Ultimate SQM settings: Layer_cake + DSCP marks OP says at the bottom

For routers that have Switch0 interface, you don't need to use veth method!

Mine is switch0 interface

it's not the name switch0 that is the issue, he's talking about DSA architecture for switch, which is different from your device.

2 Likes

Oh thanks for letting me know :smiley: I'll build a new image asap

1 Like

What is the name of your ISP and the name of the plan? That often helps to figure the true overhead out from the ISPs documentation, but realistically for fiber links I really really do not know what really needs to be accounted, and I have no reliable way to measure the effective per-packet overhead, so this is going to be a bit of guess work, which we should start by looking into the ISPs manuals.

Thanks for the diagram. With the original config, you did not do ingress shaping at SQM1 at all, now with ingress enabled you do, but the firewall rules that make sure all ingress packets contain the correct dscp labels for cake to treat them correctly will only get applied after SQM1's ingress cake instance did its work, which is too late. Sure SQM2 might do a bit of what you want for ingress, but really it will only be able to work on what SQM1's ingress let into your network. This is why the ultimate scheme uses the clever dual veth ingress routing to make sure iptables runs before the ingress cake instance gets to work.

So I agree with your sentiment in

Except I do not believe that br-lan is the right place to put any shaper on at all, bridge devices are special and a shaper there would also throttle your wifi to LAN traffic if it would work properly at all.

Could you please post the output of tc -s qdisc after some use with your current SQM config, so we should be able to see whether the br-lan cake instance catches any data at all.

1 Like

As much as I would like to see that myself, I am constantly hobbled by -ENOTIME/-EOUTOFTIME so can not realistically volunteer myself, and I have nobody at hand.

That said, on my todo list is still @ldir's clever approach to copy egress DSCPs to ingress packets before cake gat hold of them. Which also suffers from the two errors above...

1 Like

ISP is SSCN & the upper tier ISP is Hathway Fiber Broadband. Plan is 10mbps/month unlimited. I'm really sorry I couldn't find any manual on their websites. So if you're short of free time then I can just ask an engineer over there.

Hey anything I can contribute in this forum :smiley:

I restarted SQM & ran internet speed test. Result:

root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev ifb0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 31468005 bytes 97040 pkt (dropped 0, overlimits 0 requeues 4)
 backlog 0b 0p requeues 4
  maxpacket 1400 drop_overlimit 0 new_flow_count 2 ecn_mark 0
  new_flows_len 0 old_flows_len 1
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 221317917 bytes 1114647 pkt (dropped 0, overlimits 0 requeues 16)
 backlog 0b 0p requeues 16
  maxpacket 1502 drop_overlimit 0 new_flow_count 13 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 1192752088 bytes 1390499 pkt (dropped 0, overlimits 0 requeues 6)
 backlog 0b 0p requeues 6
  maxpacket 1392 drop_overlimit 0 new_flow_count 6 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8035: dev br-lan root refcnt 2 bandwidth 9800Kbit diffserv8 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100.0ms raw overhead 0
 Sent 18711308 bytes 27287 pkt (dropped 1703, overlimits 29353 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 117056b of 4Mb
 capacity estimate: 9800Kbit
 min/max network layer size:           42 /    1494
 min/max overhead-adjusted size:       42 /    1494
 average network hdr offset:           14

                  Tin 0        Tin 1        Tin 2        Tin 3        Tin 4        Tin 5        Tin 6        Tin 7
  thresh       9800Kbit     8575Kbit     7503Kbit     6565Kbit     5744Kbit     5026Kbit     4398Kbit     3848Kbit
  target          5.0ms        5.0ms        5.0ms        5.0ms        5.0ms        5.0ms        5.0ms        5.0ms
  interval      100.0ms      100.0ms      100.0ms      100.0ms      100.0ms      100.0ms      100.0ms      100.0ms
  pk_delay        132us          0us          4us        749us       10.3ms          0us         53us        987us
  av_delay         17us          0us          0us        148us        1.6ms          0us          3us         53us
  sp_delay         13us          0us          0us         10us         13us          0us          3us         11us
  backlog            0b           0b           0b           0b           0b           0b           0b           0b
  pkts            17261            0            1         2866          195            0           46         8621
  bytes        17303494            0         1392      1727283       167933            0        10514      1959772
  way_inds            0            0            0            0            0            0            0            0
  way_miss            8            0            1           44            7            0           29            3
  way_cols            0            0            0            0            0            0            0            0
  drops            1629            0            0           73            1            0            0            0
  marks               0            0            0            0            0            0            0            0
  ack_drop            0            0            0            0            0            0            0            0
  sp_flows            1            0            0            1            0            0            2            1
  bk_flows            0            0            0            0            0            0            0            0
  un_flows            0            0            0            0            0            0            0            0
  max_len          1444            0         1392         1494         1392            0          499          520
  quantum           300          300          300          300          300          300          300          300

qdisc noqueue 0: dev eth1.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8033: dev pppoe-wan root refcnt 2 bandwidth 4900Kbit diffserv8 dual-srchost nat nowash ack-filter split-gso rtt 100.0ms raw overhead 0
 Sent 11019471 bytes 27078 pkt (dropped 844, overlimits 16324 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 93248b of 4Mb
 capacity estimate: 4900Kbit
 min/max network layer size:           36 /    1480
 min/max overhead-adjusted size:       36 /    1480
 average network hdr offset:            0

                  Tin 0        Tin 1        Tin 2        Tin 3        Tin 4        Tin 5        Tin 6        Tin 7
  thresh       4900Kbit     4287Kbit     3751Kbit     3282Kbit     2872Kbit     2513Kbit     2199Kbit     1924Kbit
  target          5.0ms        5.0ms        5.0ms        5.5ms        6.3ms        7.2ms        8.2ms        9.4ms
  interval      100.0ms      100.0ms      100.0ms      100.5ms      101.3ms      102.2ms      103.2ms      104.4ms
  pk_delay       18.3ms          0us        2.0ms        7.7ms          0us          0us        389us         16us
  av_delay        9.6ms          0us         66us        4.2ms          0us          0us          8us         14us
  sp_delay         24us          0us         35us         11us          0us          0us          8us         11us
  backlog            0b           0b           0b           0b           0b           0b           0b           0b
  pkts            16458            0           18         3006            0            0           30         8410
  bytes         9810038            0        20231      1520342            0            0         2311       909348
  way_inds            0            0            0            0            0            0            0            0
  way_miss            8            0            6           48            0            0           30            1
  way_cols            0            0            0            0            0            0            0            0
  drops             775            0            0           65            0            0            0            0
  marks               0            0            0            0            0            0            0            0
  ack_drop            0            0            0            4            0            0            0            0
  sp_flows            4            0            0            1            0            0            5            0
  bk_flows            0            0            0            1            0            0            0            1
  un_flows            0            0            0            0            0            0            0            0
  max_len          1480            0         1378         1480            0            0          106          165
  quantum           300          300          300          300          300          300          300          300

This sounds vaguely familiar :slight_smile:

1 Like

Okay let's try veth method :smiley:
In interfaces I see 1.br-lan (LAN) 2. pppoe-wan (WAN)
Download link speed: 10mbps (98% is 9.8mbps)
Upload link speed: 5mbps (98% is 4.9mbps)
So there will be three SQM instances as follow:

I'm pretty sure I did something wrong here or didn't put accurate data

in firewall.user veth is assigned

####veth start's here####

## set up a pair of veth devices to handle inbound and outbound traffic
ip link show | grep veth0 || ip link add type veth

## get new veth interfaces up
ip link set veth0 up
ip link set veth1 up

## trun on promisc mode,sometimes it's needed to make bridge work
ip link set veth1 promisc on

## add veth1 to bridge
brctl addif br-lan veth1

## just to make sure there's nothing inside those 2 tables
ip rule del priority 100
ip route flush table 100

## add routing for veth0 this will handle all traffic
ip route add default dev veth0 table 100
ip rule add iif $WANIF table 100 priority 100

###veth end ###

But there's no traffic when I type ifconfig

veth0     Link encap:Ethernet  HWaddr A2:42:E0:37:02:7F
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

veth1     Link encap:Ethernet  HWaddr 92:40:0E:B8:7C:8F
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

did you restart the firewall?

A quick google session yielded inconclusive results. It seems Hathway mainly uses DOCSIS, but due to the majority stake that jio bought of Hathway, they might be reselling Jio's true glass fiber. So if you can ask an engineer that would be the best way (assuming the engineer is allowed to answer).

Okay so the br-lan cake sees some traffic, but still this is the wrong place to put sqm unless you also want to throttle wifi to LAN data transfers.

That's an astute research. Never knew about that & sure I'll ask over there. For now I set it to 44 which works well :smiley:

I tried veth method but there's no traffic going through both veth0 & veth1 Why? (ofc I restarted my router).

Even restarted the router but it's same..

Update: I found it ambiguous as I've explained it below.

I don't understand the problem here my firewall.user is

#!/bin/bash -x
IPT="iptables"
WANIF="pppoe-wan" #wan interface

#tc qdisc add dev wlan0 root mq #setup multi queue for wifi device(optional)

####veth start's here####

## set up a pair of veth devices to handle inbound and outbound traffic
ip link show | grep veth0 || ip link add type veth

## get new veth interfaces up
ip link set veth0 up
ip link set veth1 up

## trun on promisc mode,sometimes it's needed to make bridge work
ip link set veth1 promisc on

## add veth1 to bridge
brctl addif br-lan veth1

## just to make sure there's nothing inside those 2 tables
ip rule del priority 100
ip route flush table 100

## add routing for veth0 this will handle all traffic
ip route add default dev veth0 table 100
ip rule add iif $WANIF table 100 priority 100

###veth end ###

##ipset for streming sites, etc; they are bening filled by dnsmasq
ipset create streaming hash:ip
ipset create usrcdn hash:ip
ipset create bulk hash:ip
ipset create latsens hash:ip

$IPT -t mangle -N dscp_mark > /dev/null 2>&1
$IPT -t mangle -F dscp_mark
## check if POSTROUTING already exits then jumps to our tables if not, add them

$IPT -t mangle -L POSTROUTING -n | grep dscp_mark || $IPT -t mangle -A POSTROUTING -j dscp_mark

iptmark(){
    $IPT -t mangle -A dscp_mark "$@"
}

## start by washing the dscp to CS0

iptmark -j DSCP --set-dscp 0

#A robust 2 rules to detect realtime traffic

# mark connections that go over 115 packets per second, not prioritized
iptmark -p udp -m hashlimit --hashlimit-name udp_high_prio --hashlimit-above 115/sec --hashlimit-burst 50 --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55 -m comment --comment "connmark for udp"

# unmarked UDP streams with small packets get CS6
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 22,25,53,67,68,123,143,161,162,514,5353,80,443,8080,60001 -m connbytes --connbytes 0:940 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6 -m comment --comment "small udp connection gets CS6"

#large udp streams like video call get AF41
iptmark -p udp -m connmark ! --mark 0x55 -m multiport ! --ports 22,25,53,67,68,123,143,161,162,514,5353,80,443,8080,60001 -m connbytes --connbytes 940:1500 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class AF41 -m comment --comment "large udp connection gets AF41"

########
##Browsing
########
## medium priority for browsing
iptmark -p tcp -m multiport --ports 80,443,8080 -j DSCP --set-dscp-class CS3 -m comment --comment "Browsing at CS3"

##################
#TCP SYN,ACK flows
##################
#Make sure ACK,SYN packets get priority (to avoid upload speed limiting our download speed)
iptmark -p tcp --tcp-flags ALL ACK -m length --length :128 -j DSCP --set-dscp-class CS3
iptmark -p tcp --tcp-flags ALL SYN -m length --length :666 -j DSCP --set-dscp-class CS3

#Small packet is probably interactive or flow control
iptmark -m dscp ! --dscp  24 -m dscp ! --dscp  18 -m dscp ! --dscp  34 -m dscp ! --dscp  40 -m dscp ! --dscp  48 -m length --length 0:500 -j DSCP --set-dscp-class CS3

#Small packet connections: multi purpose (don't harm since not maxed out)
iptmark -m dscp ! --dscp  24 -m dscp ! --dscp  18 -m dscp ! --dscp  34 -m dscp ! --dscp  40 -m dscp ! --dscp  48 -m connbytes --connbytes 0:250 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS3


########################################
# Streaming Media (videos/audios)
########################################
#Known video streams sites like netflix
iptmark -m set --match-set streaming src,dst -j DSCP --set-dscp-class AF41 -m comment --comment "video audio stream ipset"

# some iptv provider's use this port
iptmark -p tcp -m multiport --ports 1935,9982 -j DSCP --set-dscp-class AF41 -m comment --comment "some iptv streaming service"

#known usrcdn like google or akamai

iptmark -m set --match-set usrcdn src,dst -j DSCP --set-dscp-class AF21 -m comment --comment "usrcdn ipset"

#########################################
# Background Traffic (Bulk/file transfer)
#########################################
#bulk traffic ipset, like windows udates and steam updates/downloads
iptmark -p tcp -m set --match-set bulk src,dst -j DSCP --set-dscp-class CS1 -m comment --comment "bulk traffic ipset"
iptmark -p udp -m set --match-set bulk src,dst -j DSCP --set-dscp-class CS1 -m comment --comment "bulk traffic ipset"
iptmark -p tcp -m connbytes --connbytes 350000: --connbytes-dir both --connbytes-mode bytes -m dscp --dscp-class CS0 -j DSCP --set-dscp-class CS1 -m comment --comment "Downgrade CS0 to CS1 for bulk tcp traffic"
iptmark -p tcp -m connbytes --connbytes 350000: --connbytes-dir both --connbytes-mode bytes -m dscp --dscp-class CS3 -j DSCP --set-dscp-class CS1 -m comment --comment "Downgrade CS3 to CS1 for bulk tcp traffic"
iptmark -p udp -m multiport --port 60001 -j DSCP --set-dscp-class CS1 -m comment --comment "bulk torrent port UDP"

########################################
# Latency Sensitive (gaming/voip)
########################################
##ICMP, to prioritize pings
iptmark -p icmp -j DSCP --set-dscp-class CS5 -m comment --comment "ICMP-pings"

#DNS traffic both udp and tcp
iptmark -p udp -m multiport --port 53,5353,8888 -j DSCP --set-dscp-class CS5 -m comment --comment "DNS udp"
iptmark -p tcp -m multiport --port 53,5353,8888 -j DSCP --set-dscp-class CS5 -m comment --comment "DNS tcp"

#NTP
iptmark -p udp -m multiport --port 123 -j DSCP --set-dscp-class CS6 -m comment --comment "NTP udp"

#High priority ipset
iptmark ! -p tcp -m set --match-set latsens src,dst -j DSCP --set-dscp-class CS6 -m comment --comment "latency sensitive ipset" ## set dscp tag for Latency Sensitive (latsens) ipset,udp

iptmark -p tcp -m set --match-set latsens src,dst -j DSCP --set-dscp-class CS5 -m comment --comment "latency sensitive ipset" ## set dscp tag for Latency Sensitive (latsens) ipset

#########################################
# Rainbow Six Siege
#########################################
iptmark -p udp -m multiport --port 3074,6015,10000:10099 -j DSCP --set-dscp-class EF -m comment --comment "Rainbow Six Siege UDP" 
#########################################
# COD Mobile
#########################################
#iptmark -p udp -m multiport --port 88,500,3074,3544,3478:3479,4380,4500,27000-27036 -j DSCP --set-dscp-class CS6 -m comment --comment "COD" 
iptmark -p udp -m multiport --port 7500:7700 -j DSCP --set-dscp-class CS7 -m comment --comment "COD Ports"

When I restart my router it shows both Veth0 & Veth1 but doesn't show any traffic through them at all. BUT when I manually restart firewall again it starts showing me traffic (Not as much as pppoe-wan interface)

veth0     Link encap:Ethernet  HWaddr D2:2D:84:36:88:2A
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1541 errors:0 dropped:152 overruns:0 frame:0
          TX packets:123 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:217344 (212.2 KiB)  TX bytes:17019 (16.6 KiB)

veth1     Link encap:Ethernet  HWaddr A6:78:C2:29:E1:82
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:123 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1541 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:17019 (16.6 KiB)  TX bytes:217344 (212.2 KiB)

After few seconds devices connected to the router don't get internet access at alll. But if I ping through router to google/cloudflare address it works.
/etc/config/network file:

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth1.1'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'

config interface 'wan'
	option ifname 'eth0'
	option proto 'pppoe'
	option username 'username'
	option password 'password'
	option keepalive '2 5'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '1 2 3 4 0t'

/etc/config/firewall:

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config include 'miniupnpd'
	option type 'script'
	option path '/usr/share/miniupnpd/firewall.include'
	option family 'any'
	option reload '1'

Finally in firewall.user

# mark connections that go over 115 packets per second, not prioritized
iptmark -p udp -m hashlimit --hashlimit-name udp_high_prio --hashlimit-above 115/sec --hashlimit-burst 50 --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55 -m comment --comment "connmark for udp"

Is this syntax correct? Because iptable shows error "Couldn't load match `hashlimit':File not found". Someone help?

You probably need to install the hashlimit kernel module package, something like kmod-ipt-hashlimit or the like.

You may need to add veth0 to the LAN firewall zone, possibly Veth1 as well