Ultimate SQM settings: Layer_cake + DSCP marks

Your suggestion that it's an intermittent DNS resolving issue is interesting but so far not necessarily definite. The fact that it occurs with veth only suggests perhaps an issue with routing or iptables getting screwed up in the slightly strange routing that we use in the veth setup. Some more debugging with logs or packet captures is what's needed here. Can you reproduce the issue reliably and under what circumstances? What are the symptoms? So far, it's too vague even what your problem is.

True, it could just be new routes screwing up intermittently and not necessarily DNS.

Wel yeah, i can reproduce consistently once the veth is up and running. Symptom is one time opening website works, few seconds later i cant open the next new website and simply get 'Page not found'.

One thing I know is that my continuous ping to a random external wan IP keeps working in windows." without interruption. Before and after initiating script / veth

go to firewall then and in general settings make input,output,forward to accept,also do it for lan and wan.
lets see if it will fix the problem.
also open the /etc/sysctl.conf and paste the following

net.ipv6.conf.default.forwarding=1
net.ipv6.conf.all.forwarding=1
#enable connbytes in iptables
net.netfilter.nf_conntrack_acct=1

net.ipv4.conf.default.arp_ignore=1
net.ipv4.conf.all.arp_ignore=1
net.ipv4.ip_forward=1
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_responses=1
net.ipv4.igmp_max_memberships=100
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_keepalive_time=120
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_timestamps=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_dsack=1

# disable bridge firewalling by default
net.bridge.bridge-nf-call-arptables=0
net.bridge.bridge-nf-call-ip6tables=0
net.bridge.bridge-nf-call-iptables=0
net.bridge.bridge-nf-filter-vlan-tagged=0
net.bridge.bridge-nf-filter-pppoe-tagged=0

then run:

sysctl -p
sysctl -w

i think it should work fine!

When you get page not found, can you from a command line run some kind of DNS lookup of the website DNS name and see what happens (not sure what OS your computer is running but either "host" or "nslookup" are candidates for command line DNS lookup commands).

i think it's not a dns problem, but it's a firewall problem.
if it's a dns problem browser should say couldn't resolve host!

static int cake_config_diffserv4(struct Qdisc *sch)
{
/*  Further pruned list of traffic classes for four-class system:
 *
 *          Latency Sensitive  (CS7, CS6, EF, VA, CS5, CS4)
 *          Streaming Media    (AF4x, AF3x, CS3, AF2x, TOS4, CS2, TOS1)
 *          Best Effort        (CS0, AF1x, TOS2, and those not specified)
 *          Background Traffic (CS1)
 *
 *              Total 4 traffic classes.

as you said i will switch to CS3, then lets see!
but i think i should lower the connbyte number to a lower,cause my download speed is 1 mbps!
BTW: diffserv8 is not needed most of times?!

At 1Mbps let's say you want your connections to become bulk after they're receiving for 10 seconds continuously, so connbytes should be 10000000/8 = 1250000 bytes or more so the quantity you put is in the right range. If you want to make it more quickly become bulk, you could do say 5 seconds = 625000 bytes

1 Like

i think it's better to make it 625000, cause most of webpage's are between 30kb to 300kb

So Im now on latest snapshot, redid all the above on first post, still same situation ha! Looks like no DNS issue after all. If i try to reping to an IP address and no DNS name it sometimes work sometimes not. So its just initiating the socket which fails half the time.

2018-11-27 21:05:29.319: From 192.168.1.1: port unreachable; bytes=56 seq=0001 TTL=64 ID=dccd time=0.277ms
2018-11-27 21:05:30.335: From 192.168.1.1: port unreachable; bytes=56 seq=0002 TTL=64 ID=dd07 time=0.402ms
2018-11-27 21:05:31.335: From 192.168.1.1: port unreachable; bytes=56 seq=0003 TTL=64 ID=dd5b time=0.320ms
2018-11-27 21:05:32.335: From 192.168.1.1: port unreachable; bytes=56 seq=0004 TTL=64 ID=ddb5 time=0.397ms
2018-11-27 21:05:33.336: From 192.168.1.1: port unreachable; bytes=56 seq=0005 TTL=64 ID=ddee time=0.388ms
2018-11-27 21:05:34.351: From 84.241.225.42: bytes=28 seq=0006 TTL=57 ID=6f4a time=22.932ms
2018-11-27 21:05:35.351: From 84.241.225.42: bytes=28 seq=0007 TTL=57 ID=70d6 time=24.988ms
2018-11-27 21:05:36.351: From 84.241.225.42: bytes=28 seq=0008 TTL=57 ID=72de time=22.741ms
2018-11-27 21:05:37.352: From 84.241.225.42: bytes=28 seq=0009 TTL=57 ID=7481 time=22.763ms
2018-11-27 21:05:38.352: From 84.241.225.42: bytes=28 seq=000a TTL=57 ID=76a2 time=24.922ms

Takes like 5 seconds before it works for instance.

1 Like

did you tried the firewall fix

Now we're getting somewhere. This seems like maybe a problem reaching your gateway or with ARP, or with return packets from your gateway getting squashed until a firewall related connection is established or something. The fact that it eventually works suggests that until "something" happens the packets don't know where to go, then eventually that thing happens (an ARP reply, a conntrack entry, something etc) then it works.

You could try creating a new OpenWRT "interface" place it in the same firewall zone as your LAN and putting the veth0 as the physical interface.

One sec guys. I might have progress here.

Was reading up here:

And at the bottom of page this guy is talking about MAC adress on the bridge interface.

So i went into luci and to bridge and overrode the MAC address to 00:00:00:00:00:00

Since then, so far so good. No issues.

root@OpenWrt:~# ifconfig
br-lan    Link encap:Ethernet  HWaddr 56:B2:C1:03:2B:D3
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:36562 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8086 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2514747 (2.3 MiB)  TX bytes:3437620 (3.2 MiB)

eth0      Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:37033 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28052 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:3117405 (2.9 MiB)  TX bytes:83604549 (79.7 MiB)
          Interrupt:37

eth0.1    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:34673 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25497 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2214639 (2.1 MiB)  TX bytes:82103451 (78.2 MiB)

eth1      Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:62912 errors:0 dropped:0 overruns:0 frame:0
          TX packets:33482 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:86162589 (82.1 MiB)  TX bytes:2799778 (2.6 MiB)
          Interrupt:36

eth1.2    Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet addr:192.168.8.2  Bcast:192.168.8.255  Mask:255.255.255.0
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:21184 errors:0 dropped:0 overruns:0 frame:0
          TX packets:33476 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:83357789 (79.4 MiB)  TX bytes:2665162 (2.5 MiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:112 errors:0 dropped:0 overruns:0 frame:0
          TX packets:112 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:9951 (9.7 KiB)  TX bytes:9951 (9.7 KiB)

veth0     Link encap:Ethernet  HWaddr B6:FF:40:2B:D7:E7
          inet6 addr: fe80::b4ff:40ff:fe2b:d7e7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:43 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18447 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4326 (4.2 KiB)  TX bytes:80741872 (77.0 MiB)

veth1     Link encap:Ethernet  HWaddr 56:B2:C1:03:2B:D3
          inet6 addr: fe80::54b2:c1ff:fe03:2bd3/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:18447 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:80741872 (77.0 MiB)  TX bytes:4326 (4.2 KiB)

wlan0     Link encap:Ethernet  HWaddr 62:38:E0:10:AE:D1
          inet6 addr: fe80::6038:e0ff:fe10:aed1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1890 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2038 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:326310 (318.6 KiB)  TX bytes:2179811 (2.0 MiB)

wlan1     Link encap:Ethernet  HWaddr 62:38:E0:10:AE:D0
          inet6 addr: fe80::6038:e0ff:fe10:aed0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:65 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:8550 (8.3 KiB)

root@OpenWrt:~#

Strangely i dont see mac address changed in ifconfig on the bridge.

Since it seems to work now so far, and my inbound/outbound data seems to go through the veth0/1 interface, I can carry on and apply SQM and adjust iptables to my preference?

In luce my bridge now looks like this tho:

**Protocol:** Static address
**Uptime:** 0h 10m 33s
**MAC:** 00:00:00:00:00:00
**RX:** 3.20 MB (43874 Pkts.)
**TX:** 3.67 MB (9382 Pkts.)
**IPv4:** 192.168.1.1/24

br-lan is using the mac of veth1 in your ifconfig output. I'm not sure what you did, and/or whether it just has to do with maybe forcing the bridge to clear out MAC caches or something, but if it works... The other option would be to generate a random valid mac address and hand that to the bridge:

https://www.hellion.org.uk/cgi-bin/randmac.pl?scope=local&type=unicast

I don't think it makes sense to give all zeros as a mac address, but perhaps that's interpreted in some specific way by the command line?

Well, I think I can confirm its working now.

Just to test, I put SQM on Veth0 and I can control WAN ingress limit, and for eth1.2 SQM I can control the WAN egress limit.

Sounds good no? :smiley:

works!

Receiving UDP packets with Experdited forwarding now from the rules i set as well. WHOOP!

Good. I think what happened was that adding the veth1 to the bridge changed the bridge mac address and then clients on your LAN weren't aware of it and had problems. If you set a proper static locally administered MAC on your bridge that should solve the problem.

Good possibility. Nobody else stumbled on this?

It seems like it might depend on randomness and the model of your hardware, whether the veth is chosen as the bridge MAC depends on if it's the smallest or largest MAC or whatever. not everyone would see this issue.

Ok, so problem partially solved.

What happens is, i run the script at startup. Works bad.
What I need to do at that point is go to LuCi -> br-lan bridge -> advanced settings -> change MAC address
(Im not even sure if its just the fact that it changes or not that makes it work)

Not sure if i have to redo the following though:
ip route add default dev veth0 table 100
ip rule add iif eth1.2 table 100 priority 100

After this, it just keeps working well.

Yeah, something's not right, where do you put the script to set up veth? Post it here please for review. Also /etc/config/network

I started of manual, but no difference to putting it in LuCi -> Startup

# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

./root/net.sh

exit 0

Script:

ip link add type veth
ip link set veth0 up
ip link set veth1 up
ip link set veth1 promisc on

iptables -t mangle -F PREROUTING

ip route add default dev veth0 table 100 ## send everything that uses table 100 to veth0
ip rule add iif eth1.2 table 100 priority 100 ## if it comes in wan interface use table 100 for routing

iptables -t mangle -A PREROUTING -p udp -m iprange --src-range 5.135.129.0-5.135.129.255 -j DSCP --set-dscp-class EF
iptables -t mangle -A PREROUTING -p udp -m iprange --src-range 198.27.85.0-198.27.85.255 -j DSCP --set-dscp-class EF

I already made the bridge in LuCi to veth1 but manual command makes no difference, so doesn't really matter.

/etc/config/network:


config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'

config interface 'lan'
	option type 'bridge'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option delegate '0'
	option metric '1'
	option ifname 'eth0.1 veth1'
	option macaddr '00:00:00:00:00:00'

config interface 'wan'
	option ifname 'eth1.2'
	option proto 'static'
	option ipaddr '192.168.8.2'
	option netmask '255.255.255.0'
	option gateway '192.168.8.1'
	option dns '1.1.1.1 8.8.8.8'
	option delegate '0'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0 1 2 3 5t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '4 6t'

Note: The 00-00-00-00-00-00 mac is what i have to change in LuCi UI, i pretty much switch it back and forth from this to empty on every reboot, and vice versa.


Working state:

Route:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.8.1     0.0.0.0         UG    0      0        0 eth1.2
192.168.1.0     *               255.255.255.0   U     1      0        0 br-lan
192.168.8.0     *               255.255.255.0   U     0      0        0 eth1.2

ARP:

IP address       HW type     Flags       HW address            Mask     Device
192.168.8.1      0x1         0x2         b8:08:d7:d3:06:5c     *        eth1.2
192.168.1.111    0x1         0x2         94:10:3e:30:b6:8d     *        veth0
192.168.1.101    0x1         0x2         28:3b:82:cb:48:93     *        veth0
192.168.1.101    0x1         0x2         28:3b:82:cb:48:93     *        br-lan
192.168.1.130    0x1         0x2         d4:a3:3d:29:43:ba     *        veth0
192.168.1.100    0x1         0x2         2c:4d:54:4d:b9:ff     *        br-lan
192.168.1.130    0x1         0x2         d4:a3:3d:29:43:ba     *        br-lan
192.168.1.111    0x1         0x2         94:10:3e:30:b6:8d     *        br-lan
192.168.1.249    0x1         0x2         ac:cf:23:66:f1:6a     *        br-lan
192.168.1.100    0x1         0x2         2c:4d:54:4d:b9:ff     *        veth0

(note 192.168.1.100 is my main windows client i do all testing on, but intermittent/bad routing also happens on Wireless so)

ifconfig:

root@OpenWrt:~# ifconfig
br-lan    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:101957 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1653 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5465158 (5.2 MiB)  TX bytes:307035 (299.8 KiB)

eth0      Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          inet6 addr: fe80::6038:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:100419 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42955 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:7113924 (6.7 MiB)  TX bytes:84470511 (80.5 MiB)
          Interrupt:37

eth0.1    Link encap:Ethernet  HWaddr 62:38:E0:10:AE:CF
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:99946 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42421 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:5253588 (5.0 MiB)  TX bytes:84066378 (80.1 MiB)

eth1      Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:83115 errors:0 dropped:0 overruns:0 frame:0
          TX packets:101665 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:88378022 (84.2 MiB)  TX bytes:7032264 (6.7 MiB)
          Interrupt:36

eth1.2    Link encap:Ethernet  HWaddr 60:38:E0:10:AE:CF
          inet addr:192.168.8.2  Bcast:192.168.8.255  Mask:255.255.255.0
          inet6 addr: fe80::6238:e0ff:fe10:aecf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:43242 errors:0 dropped:0 overruns:0 frame:0
          TX packets:101660 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:85278140 (81.3 MiB)  TX bytes:6625038 (6.3 MiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:180 errors:0 dropped:0 overruns:0 frame:0
          TX packets:180 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:16238 (15.8 KiB)  TX bytes:16238 (15.8 KiB)

veth0     Link encap:Ethernet  HWaddr 4A:67:F0:B2:05:7D
          inet6 addr: fe80::4867:f0ff:feb2:57d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:622 errors:0 dropped:0 overruns:0 frame:0
          TX packets:42121 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:70380 (68.7 KiB)  TX bytes:84996134 (81.0 MiB)

veth1     Link encap:Ethernet  HWaddr F2:0D:43:D8:0E:82
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:42121 errors:0 dropped:0 overruns:0 frame:0
          TX packets:622 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:84996134 (81.0 MiB)  TX bytes:70380 (68.7 KiB)

wlan0     Link encap:Ethernet  HWaddr 62:38:E0:10:AE:D1
          inet6 addr: fe80::6038:e0ff:fe10:aed1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2026 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2117 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:232750 (227.2 KiB)  TX bytes:1354977 (1.2 MiB)

wlan1     Link encap:Ethernet  HWaddr 62:38:E0:10:AE:D0
          inet6 addr: fe80::6038:e0ff:fe10:aed0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:259 errors:0 dropped:0 overruns:0 frame:0
          TX packets:388 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21900 (21.3 KiB)  TX bytes:57435 (56.0 KiB)