I did it on the router itself. I used this link to do it: https://9to5answer.com/disable-ipv6-with-openwrt
Your results may be a little different than mine, but with the router set as a subnet router I was able to get it to work properly. The only issue that I'm having right now is though my solution works, after a couple of hours Tailscale crashes on that device, not sure why. Worse than that, it doesn't crash my internet when Tailscale goes down, so in a real world scenario, it would leak my IP. I think the issue is most likely something simple like my router just can't handle that amount of processing and it crashes.
I think I am going to buy one of these: https://store-us.gl-inet.com/collections/travel-routers they make travel routers with a custom build of OpenWrt pre-installed on them with Tailscale support built in that you can use through a UI. They also have a built in killswitch feature which will solve my issue with the crashing leaking my IP on my current router. They are also native setup to be able to be used as a wifi bridge so if you are in an AirBnb or something that only has wifi and no modem, you can connect this to the wifi signal and this will still route all your internet traffic through Tailscale through your exit node. Here is the page that has all of their routers that support Tailscale: https://docs.gl-inet.com/router/en/4/tutorials/tailscale/