How to customize location of "rootfs_data" partition?

I have an additional SPI NOR flash on my custom board that i would like to use for the "rootfs_data" partition. Even if i set the label to rootfs_data, the partition created by mtdsplitter will be first in the list in /proc/mtd and therefore i guess it gets picked up while mounting. I can use an extroot but while flashing a new image, the mtdsplitter needs a significant amount of time (2+mins) which i'm trying to avoid.