Deal with domain priority

storm1er · January 17, 2021, 8:54pm

Hi people,

I've got a weird problem and I do not know how to deal with it.

Let's say my SQM settings is set to 10000, up and down.

Let's say I can do speedtests, alone on my router, to this given speed without any problem.

Let's say, I try two download simultaneously, they get the same speed.

Until here, everything is okay.

Now, let's say I try a download, and beside, I open Facebook (website). Facebook load fast but the download slow down, a lot, by a lot, I mean it almost stop to failure...

Another try: I have a ping to 8.8.8.8 (Google dns) every 5 seconds (just a ping, not a download), and beside a speedtest download. No problem at all, the ping stays stable (around 60~).

And now, the ping is still enable and I go onto Facebook website: the ping goes MAAAAD like 50 then 100 then 2000, timeout BUT Facebook is loading on my screen ... Once Facebook page is fully loaded, ping goes back to normal. During the Facebook page loading, the bandwidth usage is way far from the 10000 SQM limit...

I don't know how to deal with this... Playing an online game, if someone is on the same wifi and goes to Facebook or Instagram, ping goes mad, unplayable... Bandwidth is not under load

I thought about installing a VPN to a VPS I could buy, bypassing my internet provider, but I don't even know if it could help ... Plus the downside of using a vpn.

Any idea ?

Thanks

schmaller · January 17, 2021, 10:45pm

I don't know exactly how facebook's website access is implemented in detail
but i could absolutely imaging that it's in an optimized (you may call it aggressive ;-)) way,
so that it opens several connections simultaneously to download parts of its content in parallel.
You can verify this by tracking the open connection count on the router status page when accessing facebook.

What discipline did you chose to be used by sqm?
CAKE is, i guess, recommended.
Maybe your problem is related to "external host isolation".
This Wiki section explains it further.
Mind the parts dealing with "triple-isolate".
Best regards , Dan

storm1er · January 18, 2021, 11:13am

Cake's original isolation mode was based on flows: each stream was isolated from all the others, and the link capacity was divided evenly between all active streams independent of IP addresses. More recently Cake switched to triple-isolate, which will first make sure that no internal or external host will hog too much bandwidth and then will still guarantee for fairness for each host. In that mode, Cake mostly does the right thing. It would ensure that no single stream and no single host could hog all the capacity of the WAN link. However, it can't prevent a BitTorrent client - with multiple connections - from monopolizing most of the capacity. And running speedtests from multiple internal hosts to the same speedtest server can give unpredictable results.

Cake now uses the true source/destination address information to create Per-Host Isolation, and dynamically distributes the available bandwidth fairly between the currently-active IP addresses. So a single Netflix stream to one host ideally gets just as much capacity as all the BitTorrent traffic destined to another.

To enable Per-Host Isolation Add the following to the “Advanced option strings” (in the Interfaces → SQM-QoS page; Queue Discipline tab, look for the Dangerous Configuration options):

For queueing disciplines handling incoming packets from the internet (internet-ingress): nat dual-dsthost

For queueing disciplines handling outgoing packets to the internet (internet-egress): nat dual-srchost

Well, I'll try with those nat options, we'll see if it's better

moeller0 · January 18, 2021, 12:56pm

These will only help if the FB browsing and the downloads are not happening on the same computer...

storm1er · January 18, 2021, 4:27pm

Well, that's mainly the case as most of the tiime facebook browsing is happening on a phone and impact everyone else in the house.

The worst scenario (and the most frequent) is when we are playing online and someone open a Facebook notification on his phone ... Every player are screwed for ~30 seconds ... then it goes back to normal until they scroll

Same happens with Instagram.

moeller0 · January 18, 2021, 4:40pm

Mmmh, could you post the following information please:

ifstatus wan | grep -e device
cat /etc/config/sqm
tc -s qdisc
tc -d qdisc

maybe that gives some insight into the issue?

storm1er · January 18, 2021, 5:22pm

I rebooted openwrt some hours ago stats may not be precise.

Shell result

$ ifstatus WAN | grep -e device
	"l3_device": "eth1",
	"device": "eth1",
$ cat /etc/config/sqm
config queue
	option interface 'eth1'
	option debug_logging '0'
	option verbosity '5'
	option qdisc 'cake'
	option script 'piece_of_cake.qos'
	option enabled '1'
	option linklayer 'atm'
	option overhead '44'
	option qdisc_advanced '1'
	option squash_dscp '0'
	option squash_ingress '1'
	option ingress_ecn 'ECN'
	option egress_ecn 'NOECN'
	option qdisc_really_really_advanced '1'
	option iqdisc_opts 'nat dual-dsthost'
	option eqdisc_opts 'nat dual-srchost'
	option download '8500'
	option upload '8500'
$ tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 2791270871 bytes 3209302 pkt (dropped 0, overlimits 0 requeues 95) 
 backlog 0b 0p requeues 95
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 473177495 bytes 486715 pkt (dropped 0, overlimits 0 requeues 15) 
 backlog 0b 0p requeues 15
  maxpacket 1384 drop_overlimit 0 new_flow_count 46 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 793134201 bytes 872509 pkt (dropped 0, overlimits 0 requeues 7) 
 backlog 0b 0p requeues 7
  maxpacket 1392 drop_overlimit 0 new_flow_count 50 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 439020507 bytes 504324 pkt (dropped 0, overlimits 0 requeues 54) 
 backlog 0b 0p requeues 54
  maxpacket 1384 drop_overlimit 0 new_flow_count 35 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 394084390 bytes 466689 pkt (dropped 0, overlimits 0 requeues 18) 
 backlog 0b 0p requeues 18
  maxpacket 265 drop_overlimit 0 new_flow_count 59 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 691854278 bytes 879065 pkt (dropped 0, overlimits 0 requeues 1) 
 backlog 0b 0p requeues 1
  maxpacket 1294 drop_overlimit 0 new_flow_count 41 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 800d: dev eth1 root refcnt 2 bandwidth 9750Kbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44 
 Sent 419635674 bytes 1872022 pkt (dropped 305, overlimits 971503 requeues 1) 
 backlog 0b 0p requeues 1
 memory used: 288960b of 4Mb
 capacity estimate: 9750Kbit
 min/max network layer size:           28 /    1378
 min/max overhead-adjusted size:      106 /    1590
 average network hdr offset:           14

                  Tin 0
  thresh       9750Kbit
  target            5ms
  interval        100ms
  pk_delay        807us
  av_delay         50us
  sp_delay          2us
  backlog            0b
  pkts          1872327
  bytes       420018004
  way_inds       154846
  way_miss        35968
  way_cols            0
  drops             305
  marks               0
  ack_drop            0
  sp_flows            4
  bk_flows            1
  un_flows            0
  max_len          5536
  quantum           300

qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
 Sent 2593432719 bytes 2878160 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 800e: dev ifb4eth1 root refcnt 2 bandwidth 9750Kbit besteffort dual-dsthost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44 
 Sent 2602597860 bytes 2831940 pkt (dropped 46220, overlimits 3388994 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 840Kb of 4Mb
 capacity estimate: 9750Kbit
 min/max network layer size:           46 /    1378
 min/max overhead-adjusted size:      106 /    1590
 average network hdr offset:           14

                  Tin 0
  thresh       9750Kbit
  target            5ms
  interval        100ms
  pk_delay       27.7ms
  av_delay       9.34ms
  sp_delay        219us
  backlog            0b
  pkts          2878160
  bytes      2664903763
  way_inds       128974
  way_miss        36705
  way_cols            0
  drops           46220
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len         20760
  quantum           300

$ tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2 
qdisc mq 0: dev eth0 root 
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
qdisc cake 800d: dev eth1 root refcnt 2 bandwidth 9750Kbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44 
qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
qdisc cake 800e: dev ifb4eth1 root refcnt 2 bandwidth 9750Kbit besteffort dual-dsthost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44

EDIT:

Just tested with the new Facebook Light app, it does not happens when scrolling with it >.<

moeller0 · January 18, 2021, 6:12pm

Mmmh, the bandwidth between tc -s and /etc/confg/sqm seem to be off.
Could you please post the output of:

SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start

storm1er · January 18, 2021, 6:51pm

I may know why:

I'm using a 4G modem, therefore the max bandwidth is not stable enough to set a given SQM limit and forget about it.

I have a script running which depending on bandwidth usage and ping (using stats module) it edit the SQM limit.

Here's a capture of what I get with it: (last hour, no sqm changes in it, but you can see ping going mad 2 times ... My wife going on Facebook precisely at this time each time)

Do you still need more details?