I experience the same as you have described since moving to DSA. I've got 910/110 FTTH and there is a speed reduction compared to swconfig used in 19.07.
A good article exists which explains that the kernel doesn't support multiple CPU at the moment due to current attempts not being accepted upstream.
I haven't been able to find a patch that enables multi CPU for 5.10