External Geolocation with GPS-enabled LTE Modem inaccurate

I'm running 22.03.2 with an LTE modem which has an integrated GPS, controlling the device with ModemManager.

When I query the modem for GPS information via ModemManager, it provides an accurate positioning.

If I use a web-based location detection tool, such as Google Maps, it places me around 1000 miles away from my current location. Even if I use speedtest at the console on the router, so the modem's GPS while present doesn't seem to be respected for one reason or another.

If I take this SIM and plop it into a phone, it gets accurate location.

The reason this is important to me is because it is severely impacting my latency when gaming. Sometimes I'm East Coast, sometimes I'm West Coast - and those shifts impact my ping significantly. I just recently got placed on the PACNW and my latency is terrible. I want to utilize the modem's positioning so I can have normalize latency.

How can I get accurate location for this device either by using the modem's GPS info or just by fudging it?

Doesn't this use your location based on IP ?
Does google location place you in the same location as https://www.iplocation.net/ (click Details on IP location)

Phones have a GPS built in.

Yeah it sure does get the same exact location.

My modem also has a GPS built-in, in fact it's the same type of modem that high end 4g Android devices used to connect to the exact same type of network I'm using- a Snapdragon X20. That's the reason I believe I should be able to improve my location accuracy. The ISP supports precise GPS location on a phone and that directly impacts my latency- I believe I need to learn how to configure my router to share the GPS location effectively probably with the ISP so that the ISP has my traffic emerge from their network into the internet in a location close to where I am... just like a phone would work.

/edit:

until recently my assigned IP has tended to be on the East Coast via these IP location checks. When that was the case, I pinged ~80ms to East Coast servers. Now that I'm moved to an IP that appears to be damn near right next to the Pacific my ping to East Coast servers is up at least 100 ms, while my ping to West Coast servers has dropped 50-70.

The address on my WAN interface is not public, my public address is entirely different

The traffic being generated by my router is being tunneled on the ISPs network to another region of the nation and then emerging to the internet.

How can I get the ISP to respect the GPS data from an OpenWRT/ModemManager router in the same way it respects GPS data from a phone?

You probably can't, and you have to destinguish between location based on GPS vs IP.

They might route traffic based on IMEI or something else, but it's only a guess.

See what happens if you swap the SIMs, but disable the GPS in the phone, before you make the swap.

Or try using some GPS location spoofing app.