Traffic Shaping with HTB

Hi,

I'm trying to run the HTB Traffic Shaping Example from the Example from the OpenWRT Tutorial. It runs without an error, but doesn't actually do any traffic shaping. I have limited the total traffic to 10mbit/sec, the TV to 8mbit/sec & my laptop to 2 mbit/sec. But the limits aren't in affect. I pasted the script I'm running and attached a wireshark chart to this post so you can see. Did anyone else having this problem, and if so. How did you fix it?

Best,
Justin

#!/bin/bash

# Variables
#IF_DSL=pppoe-dsl
IF_DSL=br-lan
TC=/usr/sbin/tc
IPT=/usr/sbin/iptables
IPTMOD="$IPT -t mangle -A POSTROUTING -o $IF_DSL"
IP_USER1=192.168.8.106 #-- TV
IP_USER2=192.168.8.229 #-- Laptop
#IP_USER3=
#IP_USER4=

insmod sch_htb

$TC qdisc add dev $IF_DSL root       handle 1:    htb default 40
echo Added Root
$TC class add dev $IF_DSL parent 1:  classid 1:1  htb rate 10000kbit
echo Added Total Rate
$TC class add dev $IF_DSL parent 1:1 classid 1:10 htb rate 8000kbit #-- 80% to user1
echo Added Limit on User 1
$TC class add dev $IF_DSL parent 1:1 classid 1:20 htb rate 2000kbit #-- 20% to user2
echo Added Limit on User 2
#$TC class add dev $IF_DSL parent 1:1 classid 1:30 htb rate 350kbit #-- % to user3
#$TC class add dev $IF_DSL parent 1:1 classid 1:40 htb rate 150kbit #-- % to user4

$IPTMOD -s $IP_USER1 -j CLASSIFY --set-class 1:10
echo Applied Limit to User 1
$IPTMOD -s $IP_USER2 -j CLASSIFY --set-class 1:20
echo Applied Limit to User 2
#$IPTMOD -s $IP_USER3 -j CLASSI![HTB1|690x455](upload://rXXrqa7L2g3g2zRlSQTqlQELoO6.png) FY --set-class 1:30
#$IPTMOD -s $IP_USER4 -j CLASSIFY --set-class 1:40
                                                                                                                                                

I don't believe you can put a queue on a bridge and expect it to do anything, the bridge sends through the individual interfaces in the bridge, the bridge interface itself does no queueing.

to get around this limitation I have developed a method using a veth pair where one end is in the bridge and the other end has a queue on it. Then policy routing forces LAN traffic to go into veth0 be queued, and "come out" veth1 which is part of the bridge where it's forwarded. It's a bit more overhead but it allows you to fully shape "inbound" traffic.

That sounds like a good idea. Do you have an example of how to implement that?

discussed in detail here in recent thread

2 Likes

I just figured out that its applying the bandwidth shaping with br-lan on the upload and not the download. Is there an indicator in the script that tells it what way to limit/is it possible to limit both? I can't tell

Queing only happens on egress/transmit. Transmit towards LAN is more or less equivalent to receive from WAN as far as routers go.

I would like to limit users in specific groups to have specific rate limits and max-rates.
I tried SQM/piece_of_cake, which, to my understanding, is an optimised set of rules for the overall interface/WAN, but which does "fair" distribution etc for all users.

  • I want to implement UNFAIR distribution, where some users get more of the cake than others (maybe I'm just not-a-nice-person :sunglasses:).
    • I have ~3 groups/user_classes - slow/lo, medium/ok, fast/hi
    • I would like to limit both ave. rate and max rate if this is possible.
    • ideally in both directions
      • Is it "valid" to assume from the last comment by @dlakelan that setting the interface to br-lan would effectively limit WAN uploading
      • if that is correct, would I then need to implement 2 sets of TC commands with something like
$TC class add dev $IF_IN .......
$TC class add dev $IF_OUT .......

(IF_OUT=br-lan, IF_IN=pppoe-dsl)

  • I see that one could use ipset to group various things.
    • Is it possible to use something like IPSET_GROUP1 in place of the IP_USER1 in the example (& of course, if so, how would one do this)?

Lastly - and I apologise for my total ignorance on this - do I run this as a script once-off (and it gets saved into the system config), or on each boot via init.d or something?

Thanks

bridges don't do queueing only the individual interfaces in the bridge... so that throws a monkey wrench in to your plans.

Ah yes - thank you for that.
I don't mind monkeys too much - they teach me things (even though I am a slow learner).
In theory, would I be able to do that on the eth0[lan] interface?
(I could separate off the WiFi to a cheapo router to do the wireless bit, but still use OpenWRT as the DHCP/DNS/GW provider?)

Yes, things become a lot easier if you have a wifi free router at the border of your network, and then you can put qdiscs on egress towards the WAN and towards the LAN.

If you want to do this and retain the bridge, you have to use VETH devices... which you can search up on this forum where I've discussed them at length with some others. But if you can put wifi on a separate device it makes your life easy.

1 Like

If you want to do this and retain the bridge, you have to use VETH devices..

I have no particular allegiance to the bridge. I will do whatever is simplest. If you recommend veth I will go that route.

I tried to follow several related links, but you guys are obviously WAAAAAAY higher up on the intelligence ladder than I.
Is it OK so say that if I use a tweaked version (as best as I can grasp) of the latest ldir scripts that it'll produce something acceptable while I try to learn the details?

You mention in more recent posts that you are using nftables - is this above us mortals, or could it make things simpler for the lowly folk like me?

nftables is very nice, but it isn't compatible with the standard OpenWrt firewall, so you have to choose one or the other. If you choose nftables it lets you do some things more easily, but you lose the point and click LuCI editing of firewalls

Though LuCi/GUI is nice/useful, I prefer CLI and scriptable. Might I ask if there's any examples or sample configs with some explanation on each?

This does not do per internal-IP-fairness by default, you still need to configure it, have a look at https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details especially from the section titled " Making cake sing and dance, on a tight rope without a safety net (aka advanced features)" onwards. I would implore you to try that out, not because it will solve your challenge (as it will not) but it might be a good enough fudge/work-around to give you time to research and implement your bespoke solution.

1 Like

You should probably start with https://openwrt.org/docs/guide-user/firewall/misc/nftables

I also found the nftables wiki and the arch Linux wiki helpful.

one bit of advice is to make use of the iifname directive. you can get something like openwrt zones by matching on interface names.

Also for QoS purposes I use a table on the input hook and then I DSCP tag packets. Then I use the classify command to classify by DSCP.

1 Like

Thank you good sir's - I shall research/learn and revert

Here's an example nftables file that I use on my kids computers it might be helpful to get you started:

# A simple firewall based on nftables archlinux wiki
# https://wiki.archlinux.org/index.php/nftables

flush ruleset

table inet retag {
      chain dscptag {
      	    type filter hook output priority -150;


	    ## set cs3 for default because cs2 is throttled by ATT

	    ip dscp set cs3
	    ip6 dscp set cs3

## downgrade NFS traffic slightly
	    ip protocol tcp tcp dport 2049 ip dscp set cs2
	    ip6 nexthdr tcp tcp dport 2049 ip6 dscp set cs2

## upgrade ICMP and UDP game traffic for Steam games

	    ip protocol icmp ip dscp set cs5
	    udp dport {7000-9000, 27000-27200} ip dscp set cs5
	    udp sport {7000-9000, 27000-27200} ip dscp set cs5

	    ip6 nexthdr icmpv6 ip6 dscp set cs5

	    ip6 nexthdr udp udp dport {7000-9000, 27000-27200} ip6 dscp set cs5
	    ip6 nexthdr udp udp sport {7000-9000, 27000-27200} ip6 dscp set cs5

	    ## VOIP servers UDP traffic (sanitized, insert your own xxxx here)
	    ip saddr {xxxx} ip protocol udp ip dscp set cs6
	    ip daddr {xxxx} ip protocol udp ip dscp set cs6

	    ip6 saddr {xxxx} ip6 nexthdr udp ip6 dscp set cs6
	    ip6 daddr {xxxx} ip6 nexthdr udp ip6 dscp set cs6

## prioritize small packet udp flows on any ports:
   	  ip protocol udp ip dscp < cs5 udp dport != {http,https,domain} udp sport != {http,https, domain} ct avgpkt 0-450 counter ip dscp set cs5
   	  ip6 nexthdr udp ip6 dscp < cs5 udp dport != {http,https,domain} udp sport != {http,https,domain} ct avgpkt 0-450 counter ip6 dscp set cs5

	    # set hfsc class based on dscp

	    meta priority set 1:40 ## default

	    ip dscp {ef,cs6} meta priority set 1:10
	    ip dscp {cs5} meta priority set 1:20
	    ip dscp {af41, af42, af43} meta priority set 1:30
	    ip dscp cs2 meta priority set 1:50
	    ip dscp {cs1} meta priority set 1:60

	    ip6 dscp {ef,cs6} meta priority set 1:10
	    ip6 dscp {cs5} meta priority set 1:20
	    ip6 dscp {af41, af42, af43} meta priority set 1:30
	    ip6 dscp cs2 meta priority set 1:50
	    ip6 dscp {cs1} meta priority set 1:60

	    
    }     
}


table inet filter {

      chain input {
		type filter hook input priority 0; policy drop;

		# established/related connections
		ct state established,related accept

		# invalid connections
#		ct state invalid log prefix "DROP INVALID: " drop
		
		# loopback interface
		iifname lo accept

		# ICMP & IGMP
		ip6 nexthdr icmpv6 icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, mld-listener-query, mld-listener-report, mld-listener-reduction, nd-router-solicit, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, ind-neighbor-solicit, ind-neighbor-advert, mld2-listener-report } accept
		ip protocol icmp icmp type { destination-unreachable, router-solicitation, router-advertisement, time-exceeded, parameter-problem } accept
		ip protocol igmp accept

		# SSH (port 22)
		ct state new ip protocol tcp tcp dport ssh  meter ssh-meter4 {ip saddr limit rate 10/minute burst 10 packets} accept
		ct state new ip6 nexthdr tcp tcp dport ssh  meter ssh-meter6 {ip6 saddr limit rate 10/minute burst 10 packets} accept

		# HTTP (ports 80 & 443)
#		tcp dport { http, https } accept


		udp dport bootps drop # quietly
		udp dport 5353 accept # local mDNS traffic
		udp dport 1900 accept # UpNP/service announcements

		# steam ports, accept some, others drop quietly https://support.steampowered.com/kb_article.php?ref=8571-GLVN-8711
		# https://steamcommunity.com/discussions/forum/10/359543951704487251/
		
		udp dport 27000-27030 accept
		udp dport 27031-27036 drop
		tcp dport 27036-27037 drop

		## rocket league from the LAN
		ip protocol udp ip saddr xxxx/24 accept
                ip6 nexthdr udp ip6 saddr xxxx::/64 accept
                ip protocol tcp ip saddr xxxx/24 tcp dport > 4096 accept ## minecraft?

		udp dport 10999 accept ## don't starve together

		counter log prefix "DROP INPUT: " drop
	}

	chain forward {
	    type filter hook forward priority 0; policy drop;
	}

	chain output {
		type filter hook output priority 0; policy accept;
	}

}

Obviously since the machines this is used on are not routers, they only deal with tagging output traffic, but if you put this kind of thing on a router, you can tag incoming traffic with DSCP and prioritize it as well.

Also my custom QoS has 6 separate channels.... channel 10 and 20 are for VOIP and game traffic and use a real-time scheduler, whereas channels 30,40,50,60 are for high, medium, low, and bulk traffic.

I use the same system for my router, where I do more sophisticated stuff, like down-prioritize large transfers into cs1, and up prioritize small udp flows based on their statistics (packet size and total transfer rates)

Basically video streaming goes into high, medium is normal browsing, low is NFS, and bulk is things like large downloads (things that have transferred more than 1 second of gigabit speeds) or torrents

WOW - thank you Daniel - this will certainly get my brain working and figuring. I may have to bother you again with more silly questions once I start getting my head around it all.

Sure, if you want to talk nftables and QoS let's create a new topic for that! Would be interesting to discuss, and I would probably learn something about how to use nftables on OpenWrt... my router is actually running Debian, only my access points have OpenWrt