Check my QOS config. (Some SQM issues.)

Will do.

hmm... I'll just test about this with winmtr for a bit and if the spikes occur at the gateway then It is very likely It's just my ISP or my Line if not then... It'll either still be my ISP (just at some other destination) or in case of non-local servers it could be other stuff like bad routing. (by local I don't mean my city but rather my country in this sense... so there are still 4-6 internal hops through my isp)

Couldn't agree more. Well, I can just hope at this point that the Fiber cables are set up quicker so I can get VDSL and higher internet speeds.

Point to note although, When using gargoyle which required me to manually create classes for my games and set up rules with the proper ports... the same thing happened even though I set it so the game ports get half of my bandwidth which is 2mbit download and 0.5mbit upload at all times... and the game uses 0.6-1mbit download and 0.1-0.2mbit upload at most... plus it had this ACC thing where it would constantly ping a target to automatically figure out the best downlink speed... I noticed that If I set it to my modem's IP (192.168.1.1) or gateway the ping didn't spike no matter what... If I choose the hop after the gateway it seemed to work fine and started to lower my downlink speed. my point here is the ping to the gateway didn't spike but after that is where the spikes started to happen. I'll test with winmtr but I'm highly positive I'll get the same results. Gateway pings won't spike but after that hop pings do go up and down as shown in all my tests so far.

I think HFSC with a custom queue for your game would give the best game performance. It sounds like you don't need a ton of bandwidth but you need v low latency. HFSC can do that very well.

Does your game put AF31 in both directions?

That might indicate that your ISP's transit/peering is simply too narrow...

well, that seems like the type of issue that I can do basically nothing about... Changing ISP's is not an option unless I move away from this city.

I'll still do the winmtr tests and will try to fine-tune the downlink/uplink a bit more with some general tests. All I can do for now.

Thank you for all your help the past ~1.5weeks.

Sure, but the first thing would be to actually try to localize the point in the network path where the latency spikes actually occur; if you can convince yourself that it is the ISP's network, maybe, if you are really lucky and get access to a technician over the phone (as compared to a marketing person, not to diss marketing, but this is a technical question, so...) they might be able to tell you more about the root causes. Or if you are really really truly lucky might be willing to investigate and fix the bottleneck.
This is also relevant as you will never be able to sqm you way around a variable bandwidth/latency issue further upstream (I believe not even gargoyle's ACC would be able to help you as it assumes fixed bandwidth upload).

Great, I am always curious about these things, so please post any results here, thanks.