Setup with 4G passthrough and VLAN

Installing and Using OpenWrt Network and Wireless Configuration
1

Hello,

I need some help in setting up my configuration with OpenWRT on RaspberryPi4 in conjunction with OPNsense as my main router. I am using this setup because OPNsense is not working properly together with my 4G modem EM160R-GL. Now I have the modem connected to the RaspberryPi and setup together with modemmanager in OpenWRT. It is working quite acceptable so far in OpenWRT.

My intention is to passthrough the modem and public IP to OPNsense but still be able to access the OpenWRT LuCi webinterface from my home net. I separated my network in 2 VLANs (Vlan10 (home net) and Vlan100 (wan net)) which are also my LAN and WAN networks in OPNsense (configured via VLAN). Then I created 2 VLAN devices (also Vlan10 and Vlan100) in OpenWRT and configured the LAN interface under Networks to use the Vlan10 device with a static IP. Under Devices tab I changed the 'br-lan' device to use Vlan100 and wwan0 as bridge ports. I disabled all firewall rules and removed the wan zone from 'allow forward to destination zone' in the lan zone settings to cut the connection between lan and wan.

So far I can login to LuCi weinterface from my home net. But I can't manage to get internet.
What can I do or what is the correct setup for my intention?

I'd be very thankful for help.
Kind regards!

2

So does this mean that you have tested internet access (from the Pi) through the 4G modem and all is working properly?

If not, I think this is the first thing you should be checking.

But assuming that everything is good there, let's see your config files:

Please copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

cat /etc/config/network
cat /etc/config/firewall
3

so, excuse me for my belated answer. I could find some answers for my problem that led me to the solution to recompile openwrt with a modified dnsmasq dhcp.c file. This is meant to pass through the WWAN0 Ip to OPNsense as main router. Else a default bridge setup won't work because the modem is using raw IP packets. A bridge mode is introduced into the /config/dhcp settings file that will realize this. So far so good.

But I still have problems with internet connectivity. Before I do a dnsmasq reload to forward the wwan0 IP address to the router I can ping e.g. 8.8.8.8 just normally. But as soon as the router is using the wwan0 IP a ping is not possible anymore and I dont know why. Do you have any clue why this happens?
Before IP address forwarded to router:

root@OpenWrt:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether ...:a1 brd ff:ff:ff:ff:ff:ff
    inet6 ...:40a1/64 scope link
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether ...:a3 brd ff:ff:ff:ff:ff:ff
4: wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN qlen 1000
    link/ether ...:58 brd ff:ff:ff:ff:ff:ff
    inet ....247/28 brd ....255 scope global wwan0
       valid_lft forever preferred_lft forever
    inet6 ...:c758/64 scope link
       valid_lft forever preferred_lft forever
5: eth0.10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether ...:a1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.2/24 brd 192.168.10.255 scope global eth0.10
       valid_lft forever preferred_lft forever
    inet6 fe80...40a1/64 scope link
       valid_lft forever preferred_lft forever
6: eth0.100@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether ...:a1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.2/24 brd 192.168.100.255 scope global eth0.100
       valid_lft forever preferred_lft forever
    inet6 ...:40a1/64 scope link
       valid_lft forever preferred_lft forever


root@OpenWrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         ....248  0.0.0.0         UG    0      0        0 wwan0
....240  0.0.0.0         255.255.255.240 U     0      0        0 wwan0
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.10
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0.100

root@OpenWrt:~# ip rule list
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

and after IP given to router:

root@OpenWrt:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether ...:a1 brd ff:ff:ff:ff:ff:ff
    inet6 ...:40a1/64 scope link
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether ...:a3 brd ff:ff:ff:ff:ff:ff
4: wwan0: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN qlen 1000
    link/ether ...:58 brd ff:ff:ff:ff:ff:ff
    inet ....247/28 brd ....255 scope global wwan0
       valid_lft forever preferred_lft forever
    inet6 ...:c758/64 scope link
       valid_lft forever preferred_lft forever
5: eth0.10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether ...:a1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.2/24 brd 192.168.10.255 scope global eth0.10
       valid_lft forever preferred_lft forever
    inet6 ...:40a1/64 scope link
       valid_lft forever preferred_lft forever
6: eth0.100@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether ...:a1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.2/24 brd 192.168.100.255 scope global eth0.100
       valid_lft forever preferred_lft forever
    inet ....248/28 scope global eth0.100
       valid_lft forever preferred_lft forever
    inet6 ...:40a1/64 scope link
       valid_lft forever preferred_lft forever

root@OpenWrt:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         ....248  0.0.0.0         UG    0      0        0 wwan0
....240  0.0.0.0         255.255.255.240 U     0      0        0 eth0.100
192.168.10.0    0.0.0.0         255.255.255.0   U     0      0        0 eth0.10
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0.100

root@OpenWrt:~# ip rule list
0:      from all lookup local
142:    from all iif eth0.100 lookup 42
143:    from all iif wwan0 lookup 43
32766:  from all lookup main
32767:  from all lookup default

the network config:

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix '...::/48'

config device
	option type '8021q'
	option ifname 'eth0'
	option vid '10'
	option name 'eth0.10'

config device
	option type '8021q'
	option ifname 'eth0'
	option vid '100'
	option name 'eth0.100'

config interface 'mgmt'
	option proto 'static'
	option netmask '255.255.255.0'
	option ipaddr '192.168.10.2'
	option delegate '0'
	option device 'eth0.10'

config interface 'lan'
	option device 'eth0.100'
	option proto 'static'
	option ipaddr '192.168.100.2'
	option delegate '0'
	option netmask '255.255.255.0'

config interface 'wan'
	option proto 'mbim'
	option device '/dev/cdc-wdm0'
	option apn 'internet'
	option pincode '...'
	option auth 'none'
	option pdptype 'ipv4v6'

dhcp config:

config dnsmasq
	option boguspriv '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option nonwildcard '0'
	option localservice '1'

config dhcp 'lan'
	option interface 'lan'
	option leasetime '2m'
	option bridge_mode '1'
	option bridge_interface 'wan'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

and firewall config:

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option synflood_protect '1'
	option forward 'REJECT'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'
	list network 'mgmt'

config zone
	option name 'wan'
	option output 'ACCEPT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	option input 'ACCEPT'
	option forward 'ACCEPT'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config forwarding
	option src 'wan'
	option dest 'lan'

I hope you can help me. Thanks again!

4

You cannot bridge an ethernet port with non-ethernet port, that was already discussed here.

5

thanks andrew. That is the reason for the fix to give the eth port of the router (eth0.100) the ip of the wwan0 port of the modem by fixing dnsmasq. There is no problem for the opnsense router to pull the IP and gateway of the modem now. But I just can not reach the internet. I do not have problems with the so called "bridge".

6

Do you have ECM mode available on your modem and do you have ADB interface exposed?
If yes, we can try something.

7

nope. ECM is possible but not working properly (only returning link local address). ADB is not available.
The problem is not the modem. It is working in mbim mode.

8

Believe me or not, but I'm familiar with the problem for some years already.
The problem is with encapsulation used by the most modern modems and not with this particular modem. You can easily achieve what you want with ancient Huawei E3372 running in NCM mode - in this case you should be able to bridge your ethernet interface (physical or logical) with wwan, just because this modem uses ethernet framing. You should be able to find some posts about this here.
Found this for you: Using a router as "dumb" LTE gateway?

9

thank you andrew, but I'm quite sure it is not a problem of the modem. I did all steps based on this post (in german) IP passthrough. And this is the principle behind the method Bridge interface
it should not be a problem but I think my openwrt setup is not really correct. I am not even sure if the firewall is completely opened in my current config... as I said, I have no internet and connectivity problem just before passing the modem IP to the router.