There are two major issues that you might find on some managed switches:
-
during boot-up of the switch, it is possible that all ports will be assigned the same VLAN, basically as if it was just an unmanaged switch. Once the system is booted, it will apply the VLAN configuration to the ports as is expected. This 'unmanaged' boot-up is not common on switches, but may still happen in some low end models (it is also seen in a few all-in-one wifi routers where the wan port is part of the same switch as the lan).
-
Some low end switches such as the TP-Link TL-SG1xxE series does not have a way to specify the management VLAN. This means that the switch can and will take an address from any DHCP server (including that of an ISP if wan exposed) when in DHCP client mode. Further, the switch's management interface is reachable from any VLAN which drastically reduces the security posture of the device.
While what you want to do is possible, and it is a valuable approach in some specific situations (such as when there are cabling limitations and the like), I think you're best served by continuing to use the USB ethernet adapter you already have.