I've set up a build system and added a few pr_info statements to find the cause of the error and it seems that it breaks here: https://github.com/torvalds/linux/blob/v5.4/net/ethernet/eth.c#L579.
I went down the rabbit hole and eventually found that the reason for the failure originates here: https://github.com/torvalds/linux/blob/v5.4/drivers/nvmem/core.c#L86
It's like partition is not registered as mtd device anymore but it has not been added to nvmem devices at the same time - or I'm reading it wrong?