The RTL8231 driver should configure all pins as inputs, so it's strange that would trigger the hard reset. According to the info on the wiki, that would be connected to GPIO 5 on the RTL8231. The current driver does change the pins to GPIO before changing them to an input, maybe that's triggering something...
A custom build disabling gpio1 in the DTS should leave the GPIO chip alone. Maybe that's a good thing to test first.