I have been doing a little research and it seems likes this has happened before. Archive Openwrt
I think this has already been suggested somewhere that dwc2 driver has some sort of unidentified issue that causes the mounted drive to panic and give errors sometimes. Can not get to the root cause of the issue though because it is random most of the time when the drive is being used.
I think it may be possible to remove this driver and use other drivers in place of this and it can also help improve the situation for you.
Synopsys Designware (that's where the term "dwc" comes from) is a popular source for IP blocks used in various SOCs - if yours is using this IP block and therefore requires the dwc2 driver, you can't replace it with another (there's only a single driver supporting these IP blocks plus some SOC specific glue code).