After realising that the HTB qdisc is part of the problem for R7800 I looked into the original sources of HTB at Linux upstream and noticed that there are a bunch of performance patches that have been implemented in Linux stable after 4.4 that we use here.
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/log/net/sched/sch_htb.c?h=linux-4.8.y
Everything after Feb2016 is not in 4.4, about 10 commits in total. I looked at the commit list and one patch especially looks interesting as it promises to fix a major performance issue in dual-core processors (like ipq8065 in R7800).
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?h=linux-4.8.y&id=a9efad8b24bd22616f6c749a6c029957dc76542b
I found a serious performance bug in packet schedulers using hrtimers. sch_htb and sch_fq are definitely impacted by this problem. ... This issue is particularly visible when multiple cpus can queue/dequeue packets on the same qdisc, as hrtimer code has to lock a remote base.
I compiled a firmware with that patch backported to 4.4, and SQM simple's download throughput improved by 1.5-2.0 Mbit/s. So, a partial fix, but not yet perfect.
I will look also into the other htb patches, but good to know that at least a partial reason has been found.