The openNDS package can do this, but usually via some login, where upload and download quotas and rates are specified.
But if you have a list of mac addresses to be restricted you can add a dnsmasq dhcpscript option that runs a custom script whenever a device is allocated an ip address. This will authenticate devices on the list with specified quotas and rates, and devices not on the list getting full access or some default predefined allocation.