@regae After some fiddling looks like I found a solution.. I stumbled upon https://www.mail-archive.com/ath10k@lists.infradead.org/msg13386.html , and it says that using a different ath10k firmware solves the issue, I downloaded it from https://github.com/kvalo/ath10k-firmware/blob/master/QCA4019/hw1.0/3.6/firmware-5.bin_10.4-3.6-00140 , replaced the one under /lib/firmware/ath10k/QCA4019/hw1.0/ and rebooted the device, and it started working.
Note: I tested it on 22_03 but I am sure that it will work on earlier version as well (as the mentioned thread is from a year ago).