Wouldn't 600 ms make VoIP unusable? Imagine a queue of 1000 deep and the VoIP packet which just joined at the end of a full queue had to wait 600 ms just to be sent out into the wire.
Linux routers have TCP/UDP receive and transmit buffers, which can be adjusted to hold received packets before being sent into the transmit queue.
My limited understanding of Linux networking is as follows:
[1] Socket buffer -> [2] QDisc Queue -> [3] Network interface queue -> [4] Wire/Phy
So I would think that [2] and [3] should be as short as possible, i.e. just enough to saturate the wire/phy. [1] should be as big as possible to avoid dropped packets.
QDisc can be implemented in most consumer routers, but it would not be able to scale effectively.
IMHO, most consumer routers just do not have enough grunt and resources to effectively shape traffic. Also, it only ever make sense to shape uplink traffic in a home setup, as there's no effective way to shape downlink, since it's controlled by your ISP.
From my experience with my 50/20 mbps link, VoIP, video streaming and app updates downloading which saturates the downlink does not really present major issue, when the uplink is relatively free.
My router gets an A from DSLReports.com test as well without any QoS enabled, so I guess ISP plays a part as well? It could also be because my links are not considered fast. Anyway, I would think networking is more an art than science
so probably have to trial and error until we get a setup that works for our own use.