Anyone knows why none of the limits take effect and the containers still get access to all available CPUs?
So I have no idea why it is not working via configuration, but the kernel support is present. I was able to achieve it all by modifying cgroups parameters directly.
The values I needed are under /sys/fs/cgroup/cpu/lxc.payload.* per each LXC container.