Disabling the hardware in devicetree only prevents the driver from speaking with it and it's considered non-existent -- if the bootloader enabled all 5 FE switch ports, you will need to enable ethernet and esw in devicetree and set portmap and disablemap. Example from rt5350_hootoo_ht-tm02.dts:
&esw {
mediatek,portmap = <0x10>;
mediatek,portdisable = <0x2f>;
};
In that way, the registers of the switch to disable unused ports will actually get written.