I followed this post Ultimate SQM settings: Layer_cake + DSCP marks
Purpose? As the above post says to take most of the advantage of layer.cake
Why did you setup an ipset - what does it do after a DNS lookup?
I surmise you could flag traffic to the IP of the server, instead of port?
As explained in the post it creates hash list of IP addresses after DNS lookup as per traffic.
Port based DSCP marking is old & less scalable. Having ipset is better choice in my case I can just put the most common IPs in dnsmasq.conf file and done with it. No need to waste time setting up unreliable, unpredictable ports e.g. Discord.