Howto: Updating the Lantiq XRX200 devices with latest DSL vectoring firmware

wgqoufsn · August 12, 2018, 1:09pm

Because i had to struggle around some time i would like to release a howto for updating the firmware. Most information is available somewhere over the internet. Here a more combined version.
This howto is for GNU-Linux OS. I dont care about closed source OS like Windows, MacOS, ...

In general i would recommend to use the AVM Fritzbox 7490 firmware to get the firmware from. Reason for this is because this Router get most frequent firmware updates. It also have Annex A and Annex B firmware inside. Both Annex A and Annex B version support vectoring.
The howto is written to be most time reusable. So the firmware file to the AVM 7490 is not directly linked.
In general: You get the latest firmware from here: https://ftp.avm.de/ There are stable and beta (even more up to date) releases. You can choose on yourself what you prefere.

download the firmware from the website. If you download a version in an zip, extract it. We need the *.image file.
Install p7zip package if not installed and run "7z e [filename_of_firmware].image -r filesystem.image" and then "7z e filesystem.image filesystem_core.squashfs"
Now we need software from freetz fo extract further. You have to first install the prequirements for the build process. Check https://www.freetz.org/wiki/help/howtos/common/install.en . When you have installed the required software from your OS packaging system, run "svn checkout http://svn.freetz.org/trunk freetz-trunk" followed by "cd freetz-trunk" followed by "make menuconfig". In make menuconfig you dont have to change anything. Just press esc and confirm saving. Now build the required tools for extraction - "make tools"
In freetz-trunk/tools/ you should now find unsquashfs4-avm-be. Run "unsquashfs4-avm-be filesystem_core.squashfs -e lib/modules/dsp_vr9/" followed by "cd lib/modules/dsp_vr9/" followed by " bspatch vr9-B-dsl.bin vr9-A-dsl.bin vr9-A-dsl.bin.bsdiff" .
Done. Now you have your two files with the latest DSL firmware. One is named "vr9-A-dsl.bin" and the second is named "vr9-B-dsl.bin".
Copy the Annex A oder Annex B file on your router: "scp vr9-*-dsl.bin root@192.168.1.1:/lib/firmware/"
Go with your webbrowser to luci(by default 192.168.1.1) and configure in Network-> Interfaces the line "Firmware File". You have to enter there for example "/lib/firmware/vr9-B-dsl.bin". Then press on save&apply. Your dsl connection should restart and resync. Done

I have asked some people at intel few times to finally release a new firmware file that can be included into the openwrt image by default. They dont answer to my emails.

Sources of the knowledge:

ahmar16 · August 12, 2018, 3:39pm

If the firmware is updated it means it can be better than the recent firmware we have on devices already? Maybe better functionality and operation? Please also link your post to Wiki page.

[Edit] I see the most recent firmware seems to be Fritzbox 7530? Is that also supported?

ahmar16 · August 12, 2018, 4:52pm

So I have a problem here, I installed everything that I could find. It seems some packages have been deleted or obsolete now in Ubuntu 18.04. I am getting this error:

ERROR: The header file sys/acl.h was not found in /usr/(local/)include. ERROR: The header file sys/capability.h was not found in /usr/(local/)include. WARNING: The header file readline/readline.h was not found in /usr/(local/)include. Makefile:124: *** Some build prerequisites are missing! Please install the missing packages before trying again. See http://freetz.org/wiki/help/howtos/common/install#NotwendigePakete for installation hints. Stop.

What else do I need to do?

tmomas · August 12, 2018, 4:59pm

???

You are trying to build freetz, not OpenWrt?

ahmar16 · August 12, 2018, 5:09pm

Yes, I am trying to build Freetz.

tmomas · August 12, 2018, 5:19pm

Please see http://freetz.org/ and ip-phone-forum.de for help regarding freetz.

wgqoufsn · August 12, 2018, 7:06pm

You are probably missing the package libacl1-dev (maybe its named different on your os). But this is really not openwrt topic, its more freetz topic.
It would be great if you could upgrade the freetz wiki and drop the outdated and unspoorted OS and replace those with typical modern ones.

ahmar16 · August 13, 2018, 4:59am

Update: I used Freetz-Linux VM to setup everything. Now I see unsquashfs-avm-be in tools folder but when I execute the command. It says command not found. How do I execute it now?

freetz@freetz-linux:~/freetz-trunk/tools$ ls ash freetz_download remove-nmi-vector avm_kernel_config.bin2asm freetz_functions scriptpatcher.sh avm_kernel_config.extract freetz_mklibs sfk avm_kernel_config.extract.sh freetz_patch sha1sum avm-rle-decode fwdu sha256sum avm-rle-stream-length fwdu-opensrc sha512sum blkid genext2fs strip-trailing-whitespaces.sh build genin sum build_download_toolchains genin.cat tar bunzip2 gunzip tar-gnu busybox imagename2id tichksum bzimage2eva images tune2fs check_prerequisites lib_report uncompress cksum lunzip unlzma config lzma unpack-kernel debugfs lzma2eva unsquashfs3-multi depmod.pl make unsquashfs4-avm-be developer makedevs unsquashfs4-avm-le device_table.txt md5sum unxz e2fsck mke2img unzip eva2bzimage mksquashfs2-lzma verify-dls.sh eva2lzma mksquashfs3-multi visualise_make external mksquashfs4-avm-be visualise_make.pl extract-images mksquashfs4-avm-le xz filesystem_core.squashfs parse-config yf find-squashfs push_firmware freetz_bin_functions recover-eva freetz@freetz-linux:~/freetz-trunk/tools$

mbo2o · August 13, 2018, 5:18am

If it is not in your path you need to type ./unsquashfs-avm-be or the full path instead of just unsquashfs-avm-be on its own.

ahmar16 · August 13, 2018, 9:42am

I only thought ./ was for scripts only otherwise I knew I had to use that to run scripts. Thanks though. Much thanks to OP.

M3m3nt0 · December 24, 2018, 10:45am

I apologize for the trivial question...
How to know the current version of the firmware used on OpenWrt?

Mijzelf · December 24, 2018, 12:24pm

/etc/init.d/dsl_control status

Corphish · September 23, 2020, 4:57am

Sorry for resurrecting an old thread, but I need someone to help extracting the dsl firmware out of the latest frtizbox 7940 firmware.

After I issued this line:
7z e filesystem.image filesystem_core.squashfs

I got that error => There are some data after the end of the payload data.

I also tried to jump over to step 3 about acquiring that fretz software. fretz-linux? but the link unfortunately is dead, so now even if we sorted out that 7zip error, there will be not anyway to extract that dsl firmware?

jesanor · September 23, 2020, 6:11am

AFAIK freetz moved to github. i also have problems with 7z and 07.21.image, but 7z does not complaint with 07.12.image, all works.

edit: https://github.com/xdarklight/lantiq-xdsl-firmware-info/issues/25 but no details given.

Corphish · September 23, 2020, 6:18am

Yeah, thanks for lightening me up! I believe the problem isn't related to 7z as I faced the same issue when trying PeaZip.

I will try to get in touch with nouman8, gonna see how he managed to extract the firmware, I'll let you know if I get something.

moeller0 · September 23, 2020, 7:43am

Could you let us know here in the thread:
a) if you were successful, how you managed
b) how stable this works on your link (there are some doubts about matching relative modern FB firmware blob versions with the relative old DSL driver API OpenWrt still uses, and since the proof is in the pudding it would be great to learn of issues (or better lack of issues) in spite of the version mismatch, as always in theory there are no differences between theory and practice, in practice however...)

Corphish · September 23, 2020, 9:00am

Yeah of course! I'm sharing my full experience once I get it done.

Hmm, Actually I wasn't aware of the API thing, I was just imaging it as a separate modem bridged to a router and I was naively wondering why would problems arose in the first place.
I was talking with xdarklight lately about fritzbox 7490 firmware, and he said "personally I don't choose the AVM firmwares because they have a special version number and I hear that they are also using a special driver version in the vendor firmware"
what do you think? are his words related to the matter you are addressing here?

by the way, two days ago I extracted Netgear DM200 DSL firmware version 1.0.0.66 until now everything seems to be great
tplink,tdw8980
OpenWrt 19.07.4 r11208-ce6496d796
ATU-C Vendor ID: Broadcom 192.34
Firmware Version: 5.7.11.5.0.7
Annex: B
Line Mode: G.993.2 (VDSL2)
Line Uptime: 2d 14h 31m 4s

that DSL firmware version 5.7.11.5.0.7 is not logged in https://xdarklight.github.io/lantiq-xdsl-firmware-info/
I don't know if Netgear updated it since I've seen @pythonic saying that they are using the same one since firmware version 1.0.0.52 which has a different file version 5.7.B.5.0.7-5.7.5.4.0.1

edit: nevermind, have just compared both firmwares SHA-1 and they're identical.

pythonic · September 23, 2020, 12:19pm

"B" in hexadecimal is 11 decimal, so 5.7.B.5.0.7 == 5.7.11.5.0.7 (OpenWrt formats using decimal conversion but others format using hex conversion).

Corphish · September 24, 2020, 5:40pm

Hey! nouman8 has just replied to me, he used binwalk to extract it, I got the 0.squash file but I can't make any progress from there.
check this out: https://github.com/xdarklight/lantiq-xdsl-firmware-info/issues/25

After installing all of the required packages, I cloned the freetz repository, now whenever I try to pull off any of these

make menuconfig
make tools

It returns something about setting umask to 0022, I'm not a Linux user and was using a live image so it really gave me a hard time missing with folders permissions.

Also, there's something confusing me, 'unsquashfs4-avm-be' isn't there in the freetz repo tools folder, so even if we managed to solve the aforementioned problem, how I'm supposed to run that tool if it's not there?

moeller0 · September 24, 2020, 8:15pm

I got this from https://github.com/PeterPawn/yf_bin/ not sure why it does not seem to be there anymore...