Why does all isp's here use MTU=1480 and not 1500 ?
http://n2.netalyzr.icsi.berkeley.edu/summary/id=36a470be-31160-6e4f1570-a314-4a48-a11c#tcp_connectivity
It might be because of something like pppoE ?
But if you see some ISP's use 1492 or 1500(jumbo frame) as MTU
That could be an IPv4 tunnel, as the IPv4's header length is 20 bytes...
1492 indicates PPPoE, as that header uses 8 Bytes. 1500 bytes indicates no obvious tunnel. Now, the thing to look at is the MTU/MSS into the internet, not only the MTU into the ISP's network.
The results indicate use of timestamps in the IP headers. There might be reasons to drop packets whose timestamp is too old, particularly in networks where congestion and delay are common.
Edit https://stackoverflow.com/questions/7880383/what-benefit-is-conferred-by-tcp-timestamp has more to do with large MSS and wrapping of a 16 bit counter. It's a TCP specific feature to prevent excessive retransmits etc
2 Likes
Ah, TCP timestamps take 12 Bytes, so maybe PPPoE plus timestamps?
This link does not show your results, but will be remeasured for each access, so I just see my own data there..
@dlakelan
I'm always have problems with TCP on all ISP's when browsing sometimes i get errors like:
error_TCP_reset
error_TCP_timeout
tcp_network_error
Network dropped connection on reset
sometimes when i open some sites i saw the spining circle near the page name on the page, it's just spins then stop then spin and page never load!
This is my test results:
Summary
« SpeedGuide.net TCP Analyzer Results »
Tested on: 2019.01.10 10:01
IP address: 185.51.xxx.xxx
Client OS/browser: Windows 10 (Chrome 71.0.3578.98)
TCP options string: 020405780402080a008f89c900000000
MSS: 1400
MTU: 1440
TCP Window: 64240 (not multiple of MSS)
RWIN Scaling: 0 bits
Unscaled RWIN : 64240
Recommended RWINs: 64400, 128800, 257600, 515200, 1030400
BDP limit (200ms): 2570kbps (321KBytes/s)
BDP limit (500ms): 1028kbps (128KBytes/s)
MTU Discovery: ON
TTL: 44
Timestamps: ON
SACKs: ON
IP ToS: 00000010 (2)
Precedence: 000 (routine)
Delay: 0 (normal delay)
Throughput: 0 (normal throughput)
Reliability: 0 (normal reliability)
Cost: 1 (low cost)
Check bit: 0 (correct)
DSCP (DiffServ): CS0 000000 (0) - class 0, default traffic (RFC 2474).
So TCP timestamps 12+ pppoe 8+ ????=at least 1480 but why it's 1440
Can i have a look at your test? (hide important info)
also how it's possible to have 1492 or 1500 on some isp's ?
ifconfig output seems fine:
Summary
pppoe-wan Link encap:Point-to-Point Protocol
inet addr:10.54.106.50 P-t-P:10.0.0.10 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1480 Metric:1
RX packets:6245456 errors:0 dropped:0 overruns:0 frame:0
TX packets:5057702 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3
RX bytes:6394391429 (5.9 GiB) TX bytes:1175822788 (1.0 GiB)
root@OpenWrt:~# ifconfig
br-lan Link encap:Ethernet HWaddr C0:4A:00:E7:23:46
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fd80:b841:8de6::1/60 Scope:Global
inet6 addr: fe80::c24a:ff:fee7:2346/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:320846 errors:0 dropped:3 overruns:0 frame:0
TX packets:263257 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:67901368 (64.7 MiB) TX bytes:151799892 (144.7 MiB)
eth0 Link encap:Ethernet HWaddr C0:4A:00:E7:23:46
inet6 addr: fe80::c24a:ff:fee7:2346/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6560721 errors:0 dropped:0 overruns:0 frame:0
TX packets:5083954 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2343733299 (2.1 GiB) TX bytes:1330843850 (1.2 GiB)
eth0.1 Link encap:Ethernet HWaddr C0:4A:00:E7:23:46
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:19722 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:1270029 (1.2 MiB)
eth0.4 Link encap:Ethernet HWaddr C0:4A:00:E7:23:46
inet addr:10.10.20.21 Bcast:10.10.20.255 Mask:255.255.255.0
inet6 addr: fe80::c24a:ff:fee7:2346/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6560711 errors:0 dropped:110870 overruns:0 frame:0
TX packets:5064216 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6468108881 (6.0 GiB) TX bytes:1288741553 (1.2 GiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:16 errors:0 dropped:0 overruns:0 frame:0
TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3748 (3.6 KiB) TX bytes:3748 (3.6 KiB)
pppoe-wan Link encap:Point-to-Point Protocol
inet addr:10.54.106.50 P-t-P:10.0.0.10 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1480 Metric:1
RX packets:6252374 errors:0 dropped:0 overruns:0 frame:0
TX packets:5061633 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:3
RX bytes:6400979740 (5.9 GiB) TX bytes:1177299962 (1.0 GiB)
veth0 Link encap:Ethernet HWaddr 16:DA:D1:25:D2:5E
inet6 addr: fe80::14da:d1ff:fe25:d25e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4724948 errors:0 dropped:3 overruns:0 frame:0
TX packets:6066176 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1169025488 (1.0 GiB) TX bytes:6459975362 (6.0 GiB)
veth1 Link encap:Ethernet HWaddr 82:64:6F:9D:6B:4C
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:6066164 errors:0 dropped:0 overruns:0 frame:0
TX packets:4724940 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6459960876 (6.0 GiB) TX bytes:1169019427 (1.0 GiB)
wlan0 Link encap:Ethernet HWaddr C0:4A:00:E7:23:48
inet6 addr: fe80::c24a:ff:fee7:2348/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4307006 errors:0 dropped:0 overruns:0 frame:0
TX packets:5513918 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1047341137 (998.8 MiB) TX bytes:6037823762 (5.6 GiB)
wlan1 Link encap:Ethernet HWaddr 78:44:76:BE:90:39
inet6 addr: fe80::7a44:76ff:febe:9039/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:719887 errors:0 dropped:0 overruns:0 frame:0
TX packets:809715 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:193034842 (184.0 MiB) TX bytes:677871570 (646.4 MiB)
Well the 12 Bytes timestamps seem confirmed to me, but about PPPoE I am unsure. For the rest, I would recommend to redo the TCP analyzer test with a fresh browser after not using the browser for a while. I sometimes see lower values for MTU and MSS that disappear after a while, I assume this to be path MTU discovery in action...
Sure:
« SpeedGuide.net TCP Analyzer Results »
Tested on: 2019.01.10 10:36
IP address: 77.189.xxx.xx
Client OS/browser: Mac OS (Safari 605.1.15)
TCP options string: 020405ac010303050101080a3ba74dc60000000004020000
MSS: 1452
MTU: 1492
TCP Window: 132480 (not multiple of MSS)
RWIN Scaling: 5 bits (2^5=32)
Unscaled RWIN : 4140
Recommended RWINs: 63888, 127776, 255552, 511104, 1022208
BDP limit (200ms): 5299kbps (662KBytes/s)
BDP limit (500ms): 2120kbps (265KBytes/s)
MTU Discovery: ON
TTL: 52
Timestamps: ON
SACKs: ON
IP ToS: 00000000 (0)
Pretty much as I expect, as I am using PPPoE and configured my computer to use RFC 1323 TCP timestamps.
1 Like
I have a great connection in general, and I get 1440 MTU as well. I wonder if some ISPs are using ipv4 in ipv6 tunnels to combat ipv4 exhaustion or other similar voodoo? It probably has more to do with something between you and the speedguide.net site than something right at your end of the connection.
I wonder if your DSCP rules about ACK are causing packet reordering and this is causing some of your problems?
@moeller0
I did a test on my android phone and got the same results.
Nah, ISP's here is always doing stupid things, then we have a big and great infrastructure.
They have a ton's of ipv4 but they never give subscriber an ip(they say we give public ipv4 and even ipv6 only for business, or if you get a band from them, blah blah).
Let me check,...
I just disabled the script and did a restart my router and my pc, same results!
It's not affecting anything at the moment, and the problems that i have exits before i start using SQM and DSCP marks!.
One more thing, when i use the https version of dslreports, bufferbloat(F) is too bad and ping is very bad too(above 400ms), but http is fine, dslreports is using TCP to measure ping!
Well, at least it's good to have checked the DSCP script.
Do your problems vary in frequency throughout the day? Like does it happen a lot when other people are home and trying to stream their favorite videos etc? Does it happen during highest temperatures of the day (thermal issues in electronics devices?).
No, it's not affected throughout the day.
At home i have only 2 phones using youtube at the same time(youtube is isolated from main download/upload), we can't stream videos with 1mbps(youtube is the only choice ATM).
ISP is providing iptv via a webpage(tv.box) and m3u8 file can be used on tv or video player.
It's not affected by high or low temperatures, usually i keep electronics cool in the summer!
If you want the truth, most people here is complain a lot about then internet and they have a lot of problems,
but only me have a few(negligible) problems cause i'm using openwrt!!! 
Hey guys, finally got a nice setup working!
Inbound and outbound packet DSCP QoS
And VPN ingress over veth as well, and VPN to redirect only certain games through the VPN to optimize routing for low latency!
I think i should tag DSCP on OpenVPN ports now too? Inbound and outbound?
Als curious about network layer size, i assume I should keep 18 bytes?
If your tagging happens in prerouting or postrouting it will affect all interfaces already. Since you use VPN for only games, you may want to tag all the encrypted packets high prio
Currently I have this:
IPT="iptables"
## set up a 2 pair of veth devices to handle inbound and outbound traffic
ip link show | grep veth0 || ip link add type veth
## get new veth interfaces up
ip link set veth0 up
ip link set veth1 up
## trun on promisc mode,sometimes it's needed to make bridge work
ip link set veth1 promisc on
## add veth1,veth3 to bridge
brctl addif br-lan veth1
## just to make sure there's nothing inside those 2 tables
ip rule del priority 100
ip rule del priority 101
ip route flush table 100
##ipset for bulk. They are bening filled by dnsmasq
ipset create bulk hash:ip
## flush mangle table
$IPT -t mangle -F PREROUTING
## add routing for veth0 this will handle all slow traffic
ip route add default dev veth0 table 100
ip rule add iif eth1.2 table 100 priority 100
ip rule add iif tun0 table 100 priority 101 ;For routing tun0 vpn traffic through veth as well.
# * Latency Sensitive (CS7, CS6, EF, VA, CS5, CS4)
# * Streaming Media (AF4x, AF3x, CS3, AF2x, TOS4, CS2, TOS1)
# * Best Effort (CS0, AF1x, TOS2, and those not specified)
# * Background Traffic (CS1)
#Clear interface dscp marks, we don't trust ISP/VPN marks(also to use our own marks) so basically becomes best effort.
$IPT -t mangle -A PREROUTING -i eth1.2 -j DSCP --set-dscp 0
$IPT -t mangle -A PREROUTING -i tun0 -j DSCP --set-dscp 0
#Others
$IPT -t mangle -A PREROUTING -p icmp -j DSCP --set-dscp-class CS6
#VPN Traffic (For gaming)
$IPT -t mangle -A PREROUTING -p all --dport 443 -j DSCP --set-dscp-class EF
$IPT -t mangle -A PREROUTING -p all --sport 443 -j DSCP --set-dscp-class EF
#MWO specific (ingress UDP)
$IPT -t mangle -A PREROUTING -p udp -m iprange --src-range 5.135.129.0-5.135.129.255 -j DSCP --set-dscp-class EF
$IPT -t mangle -A PREROUTING -p udp -m iprange --src-range 198.27.85.0-198.27.85.255 -j DSCP --set-dscp-class EF
#Clear all outgoing (egress) packets from anything but 192.168.1.100
$IPT -t mangle -A PREROUTING -p all -m iprange --src-range 192.168.1.101-192.168.1.255 -j DSCP --set-dscp 0
$IPT -t mangle -A PREROUTING -p tcp -m multiport --ports 80,853,3455,8080 -j DSCP --set-dscp-class CS3
$IPT -t mangle -A PREROUTING -p udp -m multiport --ports 80,853,3455,8080 -j DSCP --set-dscp-class CS3
$IPT -t mangle -A PREROUTING -m set --match-set bulk src -j DSCP --set-dscp-class CS1 ##set dscp tag for our bulk ipset
$IPT -t mangle -A PREROUTING -p tcp -m multiport --ports 21,80,8080 -m connbytes --connbytes 1048576: --connbytes-dir both --connbytes-mode bytes -j DSCP --set-dscp-class CS1
$IPT -t mangle -A PREROUTING -p udp -m multiport --ports 21,80,8080 -m connbytes --connbytes 1048576: --connbytes-dir both --connbytes-mode bytes -j DSCP --set-dscp-class CS1
$IPT -t mangle -A PREROUTING -p udp -m multiport --dport 11132 -j DSCP --set-dscp-class CS1
vpn-policy-routing service currently routes all connections have EF (46) DSCP to the VPN (So my games pretty much)
All works fine, but I cannot see EF (46) marks on my client end with Wireshark, they are in fact Unknown.
Is this because I tag the VPN on port 443, but only encapsulated in tunnel? As in... once they get decrypted and send on their way to my local PC they are 'reset'? I guess they are in fact QoS'ed at the router level?
Otherwise something weird is going on. If i set EF (46) on firefox.exe and restart it, when i do a speedtest I can clearly see the bandwidth capped to what I set on veth0. So data definately flowing through there.
Perhaps I should use this?
**âpasstos**
Set the TOS field of the tunnel packet to what the payloadâs TOS is.in openvpn so it copies tags, but im not sure if that requires server side support.
I'm not quite sure what you're seeing, or asking here, but remember that the mangle table operates on your router so if you capture on your PC you'll see incoming packets tagged by your router, but outgoing ones will not be tagged at the point they're captured they get tagged only after the router receives them and prepares to forward them.
Is that helpful?
1 Like