Ok, I've tested a patched build on my 7530 and can change MTU to 1508 without error, but as noted I can't test actual traffic as my connection is IPoE rather than PPPoE and 1500 bytes is the max MTU for it.
Guides to the build system: overview, setup and usage.
For convenience I'd suggest creating a files subdirectory in your checkout and populating it with the DSL firmware per the 7530 installation guide so that it gets built in to your images.
I use the following very basic diffconfig file for my 7530 - you can save some mucking around in menuconfig by saving it as the .config in your checkout and then run make defconfig to get a usable .config for building:
CONFIG_TARGET_ipq40xx=y
CONFIG_TARGET_ipq40xx_generic=y
CONFIG_TARGET_MULTI_PROFILE=y
CONFIG_TARGET_DEVICE_ipq40xx_generic_DEVICE_avm_fritzbox-7530=y
CONFIG_TARGET_DEVICE_PACKAGES_ipq40xx_generic_DEVICE_avm_fritzbox-7530=""
CONFIG_TARGET_PER_DEVICE_ROOTFS=y
CONFIG_PACKAGE_luci-ssl=y
CONFIG_PACKAGE_luci-mod-dashboard=y
CONFIG_PACKAGE_luci-mod-dsl=y
If you want to add packages, run make menuconfig after make defconfig.
I'd suggest starting by building either main or the 23.05 branch from a normal openwrt checkout and get comfortable with building and running an unpatched image; once you've got the build environment sorted, you can then build from my patch branch:
- repo: https://github.com/aimacintyre/openwrt/
- branch: vrx518_max_mtu
If that works for you, I'll create a pull request to see whether the OpenWrt devs will accept it. The patch applies to 23.05 cleanly as well as main, so I'd be hopeful for a backport.