Ultimate SQM settings: Layer_cake + DSCP marks


#81

try this now without iptables!

## set up a 2 pair of veth devices to handle inbound and outbound traffic
ip link show | grep veth0 || ip link add type veth

## get new veth interfaces up
ip link set veth0 up
ip link set veth1 up

## trun on promisc mode,sometimes it's needed to make bridge work
ip link set veth1 promisc on

## add veth1,veth3 to bridge
brctl addif br-lan veth1

## just to make sure there's nothing inside those 2 tables
ip rule del priority 100
ip route flush table 100

## add routing for veth0 this will handle all slow traffic
ip route add default dev veth0 table 100
ip rule add iif eth1 table 100 priority 100

also go to your lan interface and add veth1 to the bridge.
add this to your /etc/config/network

config interface 'veth1'
	option proto 'none'
	option ifname 'veth0'
	option auto '1'

run a download then look at network--->interface-->veth0

image


#82

Yep, looks like that. Did what you said.
Rebooted, reran script too.

root@OpenWrt:~# ifconfig
br-lan    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fdec:4e63:cbbd::1/60 Scope:Global
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10744 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8950 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:607567 (593.3 KiB)  TX bytes:34116007 (32.5 MiB)

eth0      Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10775 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8988 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:804800 (785.9 KiB)  TX bytes:34156330 (32.5 MiB)
          Interrupt:37

eth0.1    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10737 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8957 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:606919 (592.6 KiB)  TX bytes:34116753 (32.5 MiB)

eth1      Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:23765 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9961 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:34822869 (33.2 MiB)  TX bytes:673240 (657.4 KiB)
          Interrupt:36

eth1.2    Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet addr:192.168.8.2  Bcast:192.168.8.255  Mask:255.255.255.0
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:7834 errors:0 dropped:0 overruns:0 frame:0
          TX packets:9954 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:33757859 (32.1 MiB)  TX bytes:632598 (617.7 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:36 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3362 (3.2 KiB)  TX bytes:3362 (3.2 KiB)

veth0     Link encap:Ethernet  HWaddr 7E:CE:E6:C9:3A:F7
          inet6 addr: fe80::7cce:e6ff:fec9:3af7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:394 (394.0 B)  TX bytes:746 (746.0 B)

veth1     Link encap:Ethernet  HWaddr B6:FB:E0:39:6A:A4
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:7 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:746 (746.0 B)  TX bytes:394 (394.0 B)

root@OpenWrt:~#

No data going through except initial stuff.
brb, gonna go dinner real quick.


#83

now try eth1.2 instead of eth1
install ip-full
i think it's a problem with busybox ip.
then lets see.

Enjoy :slight_smile:


#84

Ty!

You mean change interface back in script?


#85

YES!
also install ip-full.


#86

Ok! Think i shoukd have ip-full as i installed all the stuff in top of script, unless i gotta remove lite version first then reinstall ip-full.


#87

just install ip-full without remove something.
cause ip is busybox applet.and ip-full is iproute2 package.


#88

Done, but said was already up-to-date.

internet Browsing activity goes back through Veth0/1 now, but back to intermittent DNS resolving or something that looks vaguely like it :stuck_out_tongue:

root@OpenWrt:~# ifconfig
br-lan    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fdec:4e63:cbbd::1/60 Scope:Global
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1246 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1416 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:161094 (157.3 KiB)  TX bytes:210748 (205.8 KiB)

eth0      Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2166 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2273 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:271316 (264.9 KiB)  TX bytes:1770269 (1.6 MiB)
          Interrupt:37

eth0.1    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2138 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2244 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:230407 (225.0 KiB)  TX bytes:1757628 (1.6 MiB)

eth1      Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1602 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1185 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:1605924 (1.5 MiB)  TX bytes:187353 (182.9 KiB)
          Interrupt:36

eth1.2    Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet addr:192.168.8.2  Bcast:192.168.8.255  Mask:255.255.255.0
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:948 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1179 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1550928 (1.4 MiB)  TX bytes:181897 (177.6 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1648 (1.6 KiB)  TX bytes:1648 (1.6 KiB)

veth0     Link encap:Ethernet  HWaddr A6:FD:B4:32:24:F9
          inet6 addr: fe80::a4fd:b4ff:fe32:24f9/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:905 errors:0 dropped:0 overruns:0 frame:0
          TX packets:828 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:83113 (81.1 KiB)  TX bytes:1546880 (1.4 MiB)

veth1     Link encap:Ethernet  HWaddr 0E:0D:54:F0:45:0F
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:828 errors:0 dropped:0 overruns:0 frame:0
          TX packets:905 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1546880 (1.4 MiB)  TX bytes:83113 (81.1 KiB)

Back to square one xD
Im curious to see what my wifi actually does from my iphone, if its also intermittent webpage opening or no. (yeah same deal)


#89

Hmmm
try running those commands now on terminal:

ifconfig eth0.1 192.168.1.3
ifconfig veth1 192.168.1.4

it's my dinner time; i will back after dinner


#90

No dice.

Enjoy!


#91

I think you need to diagnose the intermittence. You say it only happens when routing through the veth? I wonder if the issue is that packets destined for the router from the WAN shouldn't go through the veth (or get dropped in the firewall) so that the DNS resolver can only handle stuff it has already cached not any new DNS?

Vaguely seems like packets destined for the router shouldn't go through the routing table at all though.

I'd suggest get a laptop on your LAN, reproduce the intermittent issue, and then do a packet capture of it and see if you can see what's up.


#92

Thanks!
run ip in terminal then paste output here.


#93

i think it's ip command problem, the problem is when you install ip-full it will not replace the busybox ip applet.
so lets check if it's ip command problem!


#94

@dlakelan
what is the difference between those 3 commands, and which behavior we will see ?
iptables -t mangle -A PREROUTING -p udp -m multiport --dport 3478:3481,5000:5500 -j DSCP --set-dscp 48
iptables -t mangle -A POSTROUTING -p udp -m multiport --dport 3478:3481,5000:5500 -j DSCP --set-dscp 48
iptables -t mangle -A OUTPUT -p udp -m multiport --dport 3478:3481,5000:5500 -j DSCP --set-dscp 48

when i was reading on sites some is using on prerouting,postrouting and output.


#95
root@OpenWrt:~# ip
Usage: ip [ OPTIONS ] OBJECT { COMMAND | help }
       ip [ -force ] -batch filename
where  OBJECT := { link | address | addrlabel | route | rule | neigh | ntable |
                   tunnel | tuntap | maddress | mroute | mrule | monitor | xfrm |
                   netns | l2tp | fou | macsec | tcp_metrics | token | netconf | ila |
                   vrf | sr }
       OPTIONS := { -V[ersion] | -s[tatistics] | -d[etails] | -r[esolve] |
                    -h[uman-readable] | -iec |
                    -f[amily] { inet | inet6 | ipx | dnet | mpls | bridge | link } |
                    -4 | -6 | -I | -D | -B | -0 |
                    -l[oops] { maximum-addr-flush-attempts } | -br[ief] |
                    -o[neline] | -t[imestamp] | -ts[hort] | -b[atch] [filename] |
                    -rc[vbuf] [size] | -n[etns] name | -a[ll] | -c[olor]}

#96

it's ok, it's using ip-full ip command.
try to reboot your router then run script again!


#97

Neh, nothing. Just again intermittent dns resolving on client end.

I'm at a loss at this point. PErhaps I should be content with just upstream QoS and leave downstream alone. Would've been nice to get both Ingress and Egress shaping but well... Something is just wrong somewhere.

Perhaps its a deviation in the openwrt builds we use?

You also seem to have packages like ```
opkg install kmod-tcp-bbr -->to enable BBR tcp congestion control,it's really improve network response.


But my system after opkg update cant even find that package anywhere.

#98

i think a tcpdump and wireshark will help to identify the intermittent dns problem.

that's normal, because you are on stable branch and kmod-tcp-bbr is only on snapshot.
i'm using snapshot, maybe upgrading to snapshot will help fixing your problem!

EDIT: as last solution before doing anything else try to install those following packages and see if it's fix your problem.

opkg update
opkg remove dnsmasq ; opkg install dnsmasq-full kmod-netem kmod-ipt-ipopt kmod-nf-nathelper-extra kmod-sched kmod-sched-cake ip-full ipset iptables-mod-conntrack-extra iptables-mod-extra iptables-mod-ipopt iptables-mod-nat-extra


#99

PREROUTING is executed after receiving and before routing. So if you want to route based on DSCP for example you need to do it here.

POSTROUTING is executed after receiving and routing.

OUTPUT is only executed for output of packets generated from the router


#100

so if i want to tag ingress packets should i tag it on postrouting or prerouting.
also if i want to tag egress packets, should i use output?
in my case i see most of times in wireshark only ingress packets is tagged for example:

source                          destination                    dscp-tag
192.168.1.130                       8.8.8.8                       CS0
8.8.8.8                           192.168.1.130                   CS6