Testing out a few EVPN concepts at home now that OpenWRT has the excellent FRR toolset available.
Hitting a blocker when it comes to routing due to the lack of VRF support. I can see that some kernel options were enabled back in the 22.03 release but there's no sign of a kmod-vrf or similar. Is this something that is expected to be added or do VRFs require a custom kernel build?
Yes, looks like the problem is that it depends on the new NET_L3_MASTER_DEV symbol which defaults to n. So the kmod-vrf package won't be built by the builders.
But it should work fine if you build your own images and select "L3 Master device support" under " Global build settings -> Kernel build options", and then enable the "kmod-vrf" package showing up under "Kernel modules -> Network Support"
I don't know why NET_L3_MASTER_DEV ended up disabled by default on all targets, but I assume it's because it adds too much size overhead for a very specialized use case.
Looks like the functionality is purely in a kmod, because when I enable that option I get exactly the same sized image built. Can you validate that finding and perhaps we can include it by default if the size is not increased?
Actually - the sysupgrade is the same size but initramfs grew a small amount:
$ ls -al bin/targets-default/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-squashfs-sysupgrade.bin
-rw-r--r-- 1 100999 100999 6423349 Dec 19 09:35 bin/targets-default/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-squashfs-sysupgrade.bin
$ ls -al bin/targets-vrf/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-squashfs-sysupgrade.bin
-rw-r--r-- 1 100999 100999 6423349 Dec 19 10:49 bin/targets-vrf/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-squashfs-sysupgrade.bin
$ ls -al bin/targets-default/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-initramfs-kernel.bin
-rw-r--r-- 1 100999 100999 5617238 Dec 19 09:35 bin/targets-default/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-initramfs-kernel.bin
$ ls -al bin/targets-vrf/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-initramfs-kernel.bin
-rw-r--r-- 1 100999 100999 5617320 Dec 19 10:49 bin/targets-vrf/ath79/generic/openwrt-23.05-snapshot-r23672-fd6831731b-ath79-generic-glinet_gl-ar750-initramfs-kernel.bin
Nice. Then there should be no reason to disable this option. Maybe send a PR enabling KERNEL_NET_L3_MASTER_DEV by default? Or even removing it and simply set CONFIG_NET_L3_MASTER_DEV in target/linux/generic/config-*?
Looking at the kernel I see that the option hides code in a large number of files, but it's mostly a single line. The most significant code contribution is
but it's not that big either so you're numbers are probably correct:
bjorn@miraculix:/usr/local/src/git/linux$ ls -l net/l3mdev
total 16
-rw-r--r-- 1 bjorn bjorn 298 Feb 24 2023 Kconfig
-rw-r--r-- 1 bjorn bjorn 6743 Jul 13 08:43 l3mdev.c
-rw-r--r-- 1 bjorn bjorn 96 Feb 24 2023 Makefile
I could be wrong, but it doesn't seem like this causes any major increase in object sizes either. It does add some fields here and there but nothing big.
Editing the default kernel file for 5.15 in target/linux/generic/config-5.15 and running defconfig doesn't seem to result in the option appearing in the .config file and I verified that installing the resulting image doesn't enable that option in kernel 5.15.
## select my architecture and target, do nothing else.
$ make menuconfig
*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.
$ grep -R L3_MASTER_DEV target/
target/linux/generic/config-5.15:CONFIG_NET_L3_MASTER_DEV=y
$ grep L3_MASTER_DEV .config
# CONFIG_KERNEL_NET_L3_MASTER_DEV is not set
Is the kernel looking elsewhere for build options? Docs seem to indicate that there's three sources of options - built-in Linux defaults, generic config and per-target/subtarget config. Seems like the generic config isn't being inserted into the .config file.
Sorry for asking so many questions - which is the correct place to make the kernel config change for a PR? I have been reading the many pages of build instructions but I'm not entirely sure how my local builds are differing from the release builds. If you can link to documentation that would be great.