I am building my own OpenWrt based on 22.03 branch. I noticed a strange behavior that seems like a bug…
In kernel_menuconfig, I do enable the netfilter nftables modules dup (nft_dup_netdev, nft_dup_ipv4 and nft_dup_ipv6) and the supporting drivers (nf_dup_netdev, nf_dup_ipv4 and nf_dup_ipv6).
I do see that these files are indeed compiled and produce the kernel modules in the build directory, but only dup_netdev is making it to the staging dir (and therefore to the produced image); dup_ipv4 and dup_ipv6 modules are missing.
Here is a search result for the libraries nf_dup in the openwrt folder on my building machine (the result is similar with nft_dup):
BUT as stated earlier, the ipv4 and ipv6 modules are not there, only the netdev one is present:
root@openwrt:~# ls -l /lib/modules/5.10.159/*dup*
-rw-r--r-- 1 root root 4136 Dec 15 01:26 /lib/modules/5.10.159/nf_dup_netdev.ko
-rw-r--r-- 1 root root 4824 Dec 15 01:26 /lib/modules/5.10.159/nft_dup_netdev.ko
Why are the modules missing? A bug somewhere between the compilation itself and the staging before generating the final image?
I will copy the missing modules from the building directory in my building system into my router and see if the compiled modules are working and report here.
I did copy the compiled modules in the building directory into the /lib/modules/(…)/ in the running openwrt on my router.
I was successfully able to load them with insmod, then use some nftables rules with dup without any error:
but nft_dup_netdev failed
root@wrt3200acm:~/dup# insmod nft_dup_netdev
failed to insert /lib/modules/5.10.146/nft_dup_netdev.ko
Maybe the but was the kernel version... stable is 146 and git is 159.
How did you run your dup ?
I try this but I have the following error
root@wrt3200acm:~/dup# nft add rule mangle prerouting dup to 8.8.8.8
Error: No such file or directory; did you mean table ‘fw4’ in family inet?
add rule mangle prerouting dup to 8.8.8.8
Because they are modules, rather than built-in, the build system needs to be told to include them.
Have a look at the file that defines the netfilter packages:
You want to define a new KernelPackage for nf-dup using these two files, then select that newly added kmod-nf-dup package in your OpenWrt config. Have a look through the history of netfilter.mk to find an example of another netfilter module being added as a starter.
I believe the module has to be compiled on the same build system for the same exact target, kernel version and configs. If you mixed modules of 159 with kernel 146, this is why it does not load properly.
As for how I run dup… There are 3 different dup:
1 for for the netdev family only, that duplicates to devices.
1 for the ip family only, that duplicates to an IPv4 address (with possibility to define on which device).
1 for the ip6 family only, that duplicates to an IPv6 address (with possibility to define on which device).
The dup rule do not work on the inet family, it has to be one of netdev, ip or ip6 (which the appropriate modules loaded in kernel).
For your example, this should work for you:
add table ip my_mangle
add chain ip my_mangle my_prerouting { type filter hook prerouting priority -1; }
add rule ip my_mangle my_prerouting dup to 8.8.8.8
Please, note that this create a table outside of fw4 (that is inet family).
If you have rules in fw4, this dup one will be read before, after or in between the fw4 ones depending on the hook priority you set for the chain my_mangle in comparison to the priorities set in fw4 for the prerouting hook.
I thought that enabling or disabling modules in the kernel_menuconfig was what telling the build system what is going to be compiled and included or not in the final build.
So I discover this other stage with kernel packages.
And indeed, there is, as for now, absolutely no package that includes nf*_dup_ipv(4|6).ko
I will probably start with the way @maurer suggested as it is an instant solution, but I might define a new KernelPackage as you described, particularly if it is to be included in future releases for all to benefit from it and solve this issue (because that means that at this point, the only way to use dup_ipv4 and/or dup_ipv6 is either to compile them builtin, or to manually add the modules in the final build (and also include them somewhere in /etc/modules.d/ for them to autoload at startup).
I did use the method suggested by @maurer, and it works right away.
For that it is the easiest and simplest way of solving the problem, without further work, I will mark it as the solution.
However, I really want to thank @johnth for his detailed explanation on why is was not working, and how to solve it the kernel module way. This hopefully will help anyone reading this thread and looking up for explanations (and me in the future if I encounter such an issue again).
To finish, I wanted to precise that unlike what @nicdesrochers experienced, I had to use insmod to use the modules when I was going this way, they would not load automatically for me.
