Large flash and qtnfmac driver for R7500v1

Hello there,

i would like to ask, if is it possible to make bigger flash space available in netgear R7500v1 like it is in R7800 or R7500v2 ?
is it possible to use driver https://github.com/torvalds/linux/tree/master/drivers/net/wireless/quantennt/netgear-rt/netgear-r7800-only-19mb-flash-available/11157800-only-19mb-flash-available/1115a/qtnfmac in 19.07 build?

thank you for an answer
mk

[your link is obviously broken]

Technically qtnfmac should cover the QSR1000/ "Topaz" chipsets in your router (not in 19.07.x, but in master). However, given that support for "Topaz" is very new (October 2018) and wasn't available before, there are no preparations for qtnfmac at all. This means quite some work will be needed to get it working, maybe DTS additions, maybe wi-fi calibration data extraction/ loading, packaging up qtnfmac, definitely sourcing the necessary firmware (which is not officially published by Quantenna, so you have to search for it, …somewhere), quite possibly some firstboot setup (MAC extraction, etc.).

Yes, it should be possible for someone motivated with the device on their desk to get it working, but don't expect it to be a walk in the park - and only for master or newer (19.07.x is too old).

hi there

I use the current situation just to deal with the R7500v1, too.

After a lot of research I came to 2 results:

  1. increase UBI partition
    for the larger UBI partition I need to customize the *.dts file. this can only be done with the full BUILD environment.

  2. 5 Ghz wifi
    since the end of 2018, the Driver for the 5 Ghz quantenna NIC should be included in the Linux kernel.


to 1:
From a previous operation to the R7800 I was able to see how it works and could apply it successfully.
(Netgear R7800 - only 19MB Flash available?)
With the R7500v1, however, the factory partitioning is different, so that only deleting the "netgear" partition does not bring much memory. The reserve partition is much larger here.
ubi: 22 MB - netgear: 12 MB - reserve: 70 MB
I would like to use both.

git diff for target/linux/ipq806x/files-5.4/arch/arm/boot/dts/qcom-ipq8064-r7500.dts
(and target/linux/ipq806x/files-4.19/arch/arm/boot/dts/qcom-ipq8064-r7500.dts)

                        ubi@1740000 {
                                label = "ubi";
-                               reg = <0x1740000 0x1600000>;
-                       };
-
-                       netgear@2d40000 {
-                               label = "netgear";
-                               reg = <0x2d40000 0x0c00000>;
-                               read-only;
-                       };
-
-                       reserve@3940000 {
-                               label = "reserve";
-                               reg = <0x3940000 0x46c0000>;
-                               read-only;
+                               reg = <0x1740000 0x68C0000>;
                        };
                };
        };

The post said that openwrt always wants to keep the way back open. With the R7800 everything was probably restored from the Stock ROM. I don't have the skills to verify the r7500v1, but I created an mtd block backup with the backup script from the docu before I first used the entire memory in the *.dts. After many attempts and also a fully written flash I replayed to stock and made a backup again.

=> could someone with the skills check that? (link to onedrive below - look in R7500v1/Backup - Netgear and Reserve Partition)
==> maybe i destroyed something? ^^
===> the default wlan passwort from stickers on the back was still alive after

If everything were restored here, could partitioning be included in the global *.dts? Would be the only way I could work with the official openwrt and full flash size.


to 2:
I am unfortunately absolutely not a professional in the linux world ... this is my first attempt ever to build something ... :smile:
first I wanted to create the driver as an extra package, which I couldn't do. I don't really understand that thing with the make command ^^
but then I found in the openwrt doku the hint that i could add missing kernel packages via the modules/*.mk files.
(https://openwrt.org/docs/guide-developer/packages in topic "Creating packages for kernel modules")
I tried that but I don't think it's working properly yet. I can't find a *.ko file, so I think there's no more on the router later ...

I think the additions in wireless.mk are not yet correct or something in the ipq806x specific kernel config?

Test 1 (wireless.mk):

define KernelPackage/qtnfmac_pcie
  SUBMENU:=$(WIRELESS_MENU)
  TITLE:=Quantenna QSR1000/QSR2000/QSR10g PCIe support (PCIE)
  DEPENDS:=@PCI_SUPPORT +@FW_LOADER @kmod-cfg80211 +@kmod-lib-crc32c
  KCONFIG:=\
	WLAN_VENDOR_QUANTENNA \
	CONFIG_QTNFMAC_PCIE
  FILES:= \
	$(LINUX_DIR)/drivers/net/wireless/quantenna/qtnfmac/qtnfmac_pcie.ko
  AUTOLOAD:=$(call AutoProbe,qtnfmac_pcie)
endef

define KernelPackage/qtnfmac_pcie/description
This option adds support for wireless adapters based on Quantenna 802.11ac QSR10g (aka Pearl) and QSR1000/QSR2000 (aka Topaz) FullMAC chipsets running over PCIe.
endef

$(eval $(call KernelPackage,qtnfmac_pcie))



define KernelPackage/qtnfmac
  SUBMENU:=$(WIRELESS_MENU)
  TITLE:=Quantenna QSR1000/QSR2000/QSR10g PCIe support
  DEPENDS:=@kmod-qtnfmac_pcie
  KCONFIG:=\
	CONFIG_QTNFMAC
  FILES:= \
	$(LINUX_DIR)/drivers/net/wireless/quantenna/qtnfmac/qtnfmac.ko
  AUTOLOAD:=$(call AutoProbe,qtnfmac)
endef

define KernelPackage/qtnfmac/description
This option adds support for wireless adapters based on Quantenna 802.11ac QSR10g (aka Pearl) and QSR1000/QSR2000 (aka Topaz) FullMAC chipsets running over PCIe.
endef

$(eval $(call KernelPackage,qtnfmac))

Test 2 (wireless.mk):

define KernelPackage/qtnfmac
  SUBMENU:=$(WIRELESS_MENU)
  TITLE:=Quantenna QSR1000/QSR2000/QSR10g PCIe support
  DEPENDS:=@PCI_SUPPORT +@FW_LOADER @kmod-cfg80211 +@kmod-lib-crc32c
  KCONFIG:=\
	CONFIG_WLAN_VENDOR_QUANTENNA=y \
	CONFIG_QTNFMAC_PCIE \
	CONFIG_QTNFMAC
  FILES:= \
	$(LINUX_DIR)/drivers/net/wireless/quantenna/qtnfmac/qtnfmac_pcie.ko \
	$(LINUX_DIR)/drivers/net/wireless/quantenna/qtnfmac/qtnfmac.ko
  AUTOLOAD:=$(call AutoProbe, qtnfmac)
endef

define KernelPackage/qtnfmac/description
This option adds support for wireless adapters based on Quantenna 802.11ac QSR10g (aka Pearl) and QSR1000/QSR2000 (aka Topaz) FullMAC chipsets running over PCIe.
endef

$(eval $(call KernelPackage,qtnfmac))

Test 3 (tryed to add in target/linux/ipq806x/config-5.4):

CONFIG_WLAN_VENDOR_QUANTENNA=y
CONFIG_QTNFMAC_PCIE=y
CONFIG_QTNFMAC=y

Collected error while different trys:

modules/wireless.mk:61: recipe for target '/home/user/pp-mod-openwrt/bin/targets/ipq806x/generic/packages/kmod-qtnfmac_5.4.34-1_arm_cortex-a15_neon-vfpv4.ipk' failed
Collecting package info: doneERROR: module '/home/user/pp-mod-openwrt/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.4.34/drivers/net/wireless/quantenna/qtnfmac/qtnfmac_pcie.ko' is missing.

for testing I have activated the other 2 packages from the wireless.mk in menuconfig. ... same..

WARNING: kmod-net-prism54 is not available in the kernel config - generating empty package
WARNING: kmod-net-rtl8192su is not available in the kernel config - generating empty package
WARNING: kmod-qtnfmac is not available in the kernel config - generating empty package

OneDrive Link to files:
https://1drv.ms/u/s!ApgFWNTXIXFBjLxKlNZFakncKFvQrw?e=ihmxV7
Only that I said, DO NOT use the firmware or other packages from the folders from my share to flash your router! The files should only help able people to see where I stand.


@slh Can you do something with this informations and give a few tips on how to keep going? what do I overlook?

... without understanding it better, I am unfortunately no longer able to make any progress at this point ...

stay healthy

Just if anyone care, here are my changes regarding flash size and qtnfmac

Known issues:
ath10k - mac is not set properly even with correct caldata
qtnfmac - missing firmware (driver loads correctly, detects chipset and even loads incompatible firmware correctly)

Do I understand it correctly that the firmware for the Quantenna Topaz chips is missing? I don't have any experience with wifi drivers, but I found an old driver package with the firmware while trying to make my R7500 usable again: https://github.com/ILOVEPIE/qts1kpcie_wifi_driver/tree/BuildScripts/kernel/quantenna_qts1kpcie/files/lib/firmware

It's absolutely huge but maybe it works.

Edit: I compiled OpenWrt and put the firmware files from the latest stock sources (130) at the locations specified in the kernel log. It uploads everything correctly but shows a firmware execution failure ~50 seconds later.

As you already know this is the incompatible firmware I mentioned earlier.
Actually it boots correctly and you can even enable 5G using TTL console. This firmware is I think so called softmac used in genuine Netgear firmware.

I even contacted guys from Quantenna and AFAIK the firmware is ready but stuck in legal department.
The other problem is acquisition of Quantenna but other company, so the firmware release it rather low priority and no one know when and if it happens.

Ok, thank you for the info.

Hopefully the new firmware also means that the connection issues with ac get resolved.

It is really sad :cry: that they developed a new full mac driver but not published the firmware for it...better to remove it from the kernel and commit it as closed code than give a false hopes...

Hi,

I saw the patch that increases flash size of the R7500v2 (by using the Netgear partition).
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=57b1e951b52010e42c64ee617e0b5b9d547d3a0d
Is there a reason why this was not done for the R7500v1, too?

OK, in case of the R7500v1 the Netgear partition would only add 12M, but the "reserve" partition would add another 70MB. Is there a reason not to use "reserve"?

On my R7500v1 the "reserve" partition is filled with 0xFF.

One reason is that the developers with capability to merge the change probably do not have a r7500v1 to test on.

If you or another user does the work, tests, and submits a PR like what was done for the r7800 and r7500v2, then it is possible it will get merged. There is forum thread describing what was done for the r7500v2 (including testing) here.

Note in the linked post above, I found that settings made while using the "stock" image were preserved even after flashing openwrt followed by reflashing stock. The speculation is that the "reserve" partition is where the stock images save these settings. If the reserve partition is overwritten, some extra effort might be required to go back to stock - perhaps reformatting the partition? I don't know and I don't have an r7500v1 so you or another user will have to experiment.

HTH

EDIT: if you do decide to experiment, make a backup of any partition you overwrite... and keep in mind that you might not be able to go back.

@nmrh Thanks for your reply and for the link to your post in the R7500v2 thread. I actually missed that.
When I got the router I reset it to factory defaults. So this may explain why "reserve" is empty. OTOH it does not matter - even if the OEM firmware stores stuff there, rolling back to factory default is all we need to do, right? So the only partition I would have to worry about is "netgear" and I don't see why or how this should be different from the R7500v2 or R7800.
Remains the proof...
Alas, in the age of high-speed unlimited internet it may seem strange to you but I have very limited bandwidth and pay for download volume right now (being in a very remote area close to the arctic circle...). If I'm correct your repo on git is 186MB, plus all the feeds...
So this would have to wait. :cry:

If in doubt, it's better to be very conservative about repurposing flash partitions. Especially considering that the r7500 v1 only comes with 256 MB RAM (which means you can't install/ run everything but the kitchen sink anyways).

1 Like

my repo isn't relevant or necessary for the r7500v1 and is out of date. Don't waste your download budget on it - just clone/download the official openwrt git repo and work with that. You will only have to edit a few lines of the dts file anyway.

if you remove/overwrite the "reserve" partition, there might be a chance you won't be able to use the OEM firmware even after re-flashing it. The r7800 and r7500v2 OEM firmware have code to check and rebuild the "netgear" partition if it missing/damaged. @hnyman did not find similar logic for the "reserve" partition which is why he (and I) avoided removing it.

If I lived in a remote area, could not easily replace the device, and risked losing something I needed (like 5GHz wifi), I would not take the risk. Use your own judgement.

EDIT: based on the post of another user above in this thread, it does look like OEM firmware will at least boot and run after having used an openwrt image with the "reserve" partition removed. How well the OEM firmware runs and if it can still save settings are some of the things I suspect you would have to experiment with and demonstrate to get the change merged into openwrt.

Please correct me if I am wrong, but my understanding of flash memory was that an mtd partition is just a bunch of flash cells from address x to address y. There is no "boot sector", no partition table, no backup partition tables. The boundaries are set arbitrarily and hard coded.
As I wrote before, on my router the "reserve" partition was filled with 0xFF (= unprogrammed flash). So there would be nothing to be restored or even repaired in the "reserve" address space.

Meanwhile I compiled OpenWRT 19.07.04 with a changed dts file.
(I did this on a VPS server I almost forgot I still had. :wink: That helped me to avoid downloading the sources to my computer up here.)
Result:
The full flash is available now, 86.8Mb in overlay, and I can still use the official packages.

So I got everything I wanted, but...
how to give back?

I filled the overlay partition with a recognizable pattern:
yes "" | tr "\n" "1" | dd of=/overlay/work/test
Then I rolled back to the OEM image and played a bit with it. Everything I tried worked.
Finally I installed the stock OpenWRT image (with the limited flash), backed up mtd6 ("netgear") and mtd7 ("reserve") and compared to the backups I had made before.

Result:
mtd6 ("netgear") has been overwritten. The pattern has vanished. I quickly screened the contents for strings. At the beginning of "netgear" there is still (or again?) some OpenWRT stuff. The rest is OEM.

mtd7 ("reserve") still contains UBI tags, the pattern "1", and OpenWRT stuff (like USB IDs). So it seems the OEM firmware has not touched "reserve" at all. Not at first OEM installation, when it all read 0xFF, not after rolling back from OpenWRT. Apparently the OEM firmware completely ignores it.

Question is: Is this a problem?
The only issue I can think of would be if any later version of the OEM firmware makes use of "reserve" and does so by writing to an erase block without first erasing it. But wouldn't that be utterly... unprofessional?

First off, my complements on successfully making the change.

Regarding the reserve partition, I'm really not the person to ask.

If your stock /dev/mtd7 partition really was blank and unformatted (contained no ubifs volumes/data) then I don't see how overwriting it is a problem. In this case, if netgear sends out an update that makes use of it, then they would have to format/prepare it.

Regardless, I hope you kept a copy of your stock /dev/mtd7 partition as you may need it later to convince the dev's it's really blank. (I'm not the one you need to convince).

The r7500v2 "reserve" partition does contain 3 ubifs volumes, "pot", "config", and "language_volume". "pot" and "config" are empty, but my "language_volume" does contain some data. Nothing that looks terribly important but I'm not the best person to judge that.

I can post the tools/commands I used to backup and examine my /dev/mtd7. If you have a backup of your stock "reserve" partition, you can use something similar to take a look at yours. (EDIT: please ask if you wan't this - I won't otherwise post it as it seems the r7500v1 and the r7500v2 are different and it is somewhat lengthy).

HTH

Hi HTH,

Thank you for your response.

I still have the backed up mtd7 and I did a
xxd -p R7500v1.mtd7.bin | grep -v f
which yielded - nothing.
Thus, thank you for your offer, but I am not sure what else I could possibly do to convince the devs that it was empty.

OTOH I found this box on ebay. Second hand. It had the OEM firmware on it, but how would I know how the previous owner messed with it and whether it really was in OEM condition?

Anyways, it is not a big change I made and it's not rocket science. But it will add value to those who picked this box because the toh lists it with 128MB flash and who are surprised to find only 8MB but shy away from re-compiling OpenWRT.

So if the devs approve I can provide a diff. Towards OpenWRT 19.07.4 or current master.
Or I can clone the repo to GitHub and send a pull request.
I am just not sure how to proceed or if anyone is interested at all, especially since this router seems to be considered as "outdated and unfit" already:

It's all a matter of perspective. :wink:
The router I replaced with it had only 64MB RAM. And 256MB should be plenty for a wireguard tunnel, so that I can safely control this place via the internet when I am not here, even though my ISP does all he can to prevent me from doing so. :wink:

1 Like

Begin EDIT:

dd if=R7500v1.mtd7.bin 2>/dev/null | tr -d '\377' | wc -c

I think should return 0 if every value in the partition is decimal 255/0xFF/octal 377.

Don't spend too much time trying to prove a negative. Submit at PR at github (for just the expanded flash - link this forum thread and specifically your posts in which you did the testing), post a link to the github PR here and encourage other r7500v1 users to comment (at the github PR and not just this forum thread) about the contents of their /dev/mtd7 aka "reserve" partition.

At least one user above has posted the contents of their /dev/mtd7 partition so you can use that as "evidence" to start.

Be patient, courteous, and responsive with the dev's (just like you have been here) as it may take a while to get accepted into master and longer to make it into a "stable" build.

that will be up to the dev's - and I wouldn't worry too much about having got the box on ebay - likely the "devs" will ask other r7500v1 users to check.

the xxd command you used does give some non-zero output for my "reserve" partition but it is mostly zeros...

other things you or other users might want to try on a mtd backup include strings R7500v1.mtd7.bin, "ubireader_display_info R7500v1.mtd7.bin", ubireader_extract_images R7500v1.mtd7.bin, then binwalk -e any ubifs volumes that get extracted with a non zero size which in your case should be none..
End EDIT

Looks like you are on your way... good luck.

Thank you for reminding me - I forgot he had made his files available.
I checked his mtd7 backup and it is like mine: all 0xFF = unprogrammed flash.

Just to avoid any confusion: Yours is a R7500v2, right?
The xxd command does a hex dump of the entire file, he -p switch drops the leading '0x', and the result is piped through grep, checking for an inverse match of the letter 'f'. So any byte that is not all '1' (binary) will show up. The return code of the command last executed (which is grep) is '1', which means "no match".