Rpi4, managed switch configuration woes

What @dlakelan said! :grinning:

But a more detailed map of what is connected on each port would be useful, as would the following g config files:
/etc/config/network
/etc/config/firewall
/etc/config/dhcp

rpi4 connected to port1 of tp-link managed switch.
switch is configured as;
802.1Q VLAN Enabled

  • VLAN1 ports 1-7, tagged 1, untagged 2-7
  • VLAN2 ports 1,8 tagged 1, untagged 8

OH!!! didn't realize what PVID was. I set port 8 to pvid 2. Plugging the modem into port 8 and the rpi4 pulls an IP for eth0.2, but I lose connection to port 1.

putting the modem to port 7 (or any on vlan1) and I get connectivity back.

seems I'm making some headway, just need that last link (in the rpi4) and things should be good.

I also installed the SQM package, but I'll have to read on this...first i've heard of sqm. :frowning:

thanks.

firewall config

root@OpenWrt:~# cat /etc/config/firewall

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan lan1'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan6 wan'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config include
        option path '/etc/firewall.user'

network

root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdfe:0b57:20c5::/48'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option proto 'dhcp'
        option ipaddr '192.168.3.1'

config interface 'lan1'
        option proto 'dhcp'
        option ifname 'eth0.1'

config interface 'wan'
        option ifname 'eth0.2'
        option proto 'dhcp'

DHCP

root@OpenWrt:~# cat /etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option boguspriv '1'
        option filterwin2k '0'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option nonegcache '0'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
        option nonwildcard '1'
        option localservice '1'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv6 'server'
        option ra 'server'
        option ra_slaac '1'
        list ra_flags 'managed-config'
        list ra_flags 'other-config'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'
  • change
config interface 'lan'
	...
        option ifname 'eth0'
	...
        option proto 'dhcp'
	...

to

config interface 'lan'
	...
        option ifname 'eth0.1'
	...
        option proto 'static'
	...
  • remove:
config interface 'lan1'
	option proto 'dhcp'
	option ifname 'eth0.1'
  • remove lan1 from firewall zone lan
1 Like

will do.

Thanks!

That seems to have been the ticket. I can connect my desktop to one of the open ports and have an internet connection.

But seems to be an issue with the R6700v2 as an AP. Connects to the rpi just fine, but when I connect wirelessly says, "connected, but there is no internet available" I'll have to test with the c5 and see if this is an issue with r6700.

Thanks for the help!

Well, seems I'm not fully up just yet. Pulling internet from the c5 to the rpi4 works, but going from the cable modem to Port 8 and rpi4 won't pull an IP address. Looking at the configs from another OpenWRT router I am currently using (C5), I don't see anything in the settings to explain why an IP address from the modem is not getting assigned to the rpi4. all devices were powered down during the rpi4 swap. as soon as I plug in the C5, I get a wan IP via dhcp, just not with the rpi4.

root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdfe:0b57:20c5::/48'

config interface 'lan'
        option type 'bridge'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ifname 'eth0.1'
        option proto 'static'
        option ipaddr '192.168.10.1'

config interface 'wan'
        option ifname 'eth0.2'
        option proto 'dhcp'

dhcp

root@OpenWrt:~# cat /etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option boguspriv '1'
        option filterwin2k '0'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option nonegcache '0'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
        option nonwildcard '1'
        option localservice '1'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv6 'server'
        option ra 'server'
        option ra_slaac '1'
        list ra_flags 'managed-config'
        list ra_flags 'other-config'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'

firewall

root@OpenWrt:~# cat /etc/config/firewall

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan lan1'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan6 wan'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option proto 'tcp udp'
        option name 'notricardo'
        option src_ip '192.168.10.132'
        option dest 'lan'
        option target 'REJECT'
        option src 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config include
        option path '/etc/firewall.user'

Is your Pi's network configuration from comment 6 still current? If so...

your main LAN (eth0) is set as untagged and is properly configured to be aLAN, while LAN1 (VLAN 1 on eth0.1) is set as a DHCP client (as is the WAN on eth0.2).

Try the following:

  • remove LAN1 from your network configuration.
  • Change port 1 of your switch to have vlan 1 untagged (PVID 1); keep VLAN 2 tagged on that port.

There are some people on this forum who feel that all networks should be tagged on a trunk port, so my suggestion above may potentially prompt a discussion on that topic. That said, try it to see if this resolves the issue. If it does, you have two options -- keep it as is (vlan1 untagged, vlan2 tagged on port 1), or if you want all networks tagged on trunk ports, change the ifname in lan to eth0.1 and then change the switch back to have VLAN1 tagged on port 1.

Let me see if I can make a little more clear.
Modem >router (openwrt 18.06)>desktop =internet connection.

Modem>router> sw prt8 > sw prt1 > rpi4 > sw prt2-7 > desktop = internet connection.

Router is on 192.168.3 subnet, rpi4 on 192.168.10

No config changes to switch or rpi4.

Modem> sw prt8 > sw prt1 > rpi4 > sw prt2-7 > desktop = NO internet connection.

Not sure if suggestion to change switch will have any affect, but will try.

Thanks for the help.

If the first config works, then probably what's going on is that your modem has bonded to the first router's MAC address...

you should do "the dance" (for the config without the extra router)

  1. Turn off the modem, turn off the PI, turn off the switch.
  2. Turn on the switch, wait 30 seconds
  3. Turn on the Pi, wait 30 seconds to ~ 1min
  4. Turn on the modem, wait about 1 minute

Voila, you hopefully will have internet.

3 Likes

Make sure your computer(s) get new DHCP leases. Typically, if the interface doesn't go down, the computer will not try to negotiate a new/renewed lease. Unplugging the ethernet cable between the switch and the desktop and then plugging it back in again is one way to ensure DHCP requests happen.

1 Like

+1, except in DOCSIS, the modem will need to give a DCHP lease to connected devices, and most modems will only hand out new DHCP-leases to new MACs in a short period of time after booting up, so @dlakelan is spot on with his recommendation to do "the dance" :wink: and @psherman with his recommendation to get a new dhcp lease (from the modem0.

Turns out I was partly right and partly wrong, sure the cable modem (CM) does do a DHCP dance with the CMTS with the CM's MAC address (https://specification-search.cablelabs.com/CM-SP-MULPIv3.1):

5.2.7.2 Initialization, Provisioning and Management of CPEs

DOCSIS assumes the use of DHCP for provisioning of CPE devices. To that end the CMTS supports a DHCP relay agent which allows the operator to associate a CPE IP Address request with the subscriber Cable Modem MAC Address. This feature is also used as the basis of a mechanism that prevents spoofing of IP Addresses.

DOCSIS 3.0 gives operator the option to provision CPE devices with an IPv4 or an IPv6 or both types of IP Addresses simultaneously.

but the real issue is the bridging table, the modem learns which internal MACs to bridge to the DOCSIS segment with a default of a single MAC, this bridging table is not persistent and hence will be refilled/relearned after a modem reboot, see:

9.1.2.1 MAC Address Acquisition

The CM maintains a forwarding database (bridging table) including entries for the CM's own MAC address and CPE MAC addresses.

The CM MUST acquire CPE Ethernet MAC addresses, either from the provisioning process or from learning, until the CM acquires its maximum number of CPE MAC addresses (the lesser of the Max CPE from the config file, Max CPE or a device-dependent value), see subsection Maximum Number of CPEs in Annex C. Once the CM acquires its maximum number of CPE MAC addresses, then newly discovered CPE MAC addresses MUST NOT replace previously acquired addresses. The CM MUST support acquisition of at least 64 CPE MAC addresses.

The CM MUST NOT learn any MAC addresses for its forwarding database prior to registration. The CM MUST allow configuration of CPE MAC addresses during the provisioning process (up to its maximum number of CPE addresses) to support configurations in which learning is not practical, nor desired. The CM MUST give provisioned addresses precedence over learned addresses when adding entries to the forwarding database. The CM MUST NOT age out CPE MAC addresses. The CM MUST place all acquired CPE MAC addresses in its forwarding database [RFC 1493].

In order to allow modification of user MAC addresses or movement of the CM, addresses are not retained in non- volatile storage. On a CM reset (e.g., power cycle), the CM MUST discard all provisioned and learned addresses.

In addition, a CM can be configured such that it will discard any dynamically learned MAC addresses associated with a CMCI port if it has determined that the link has been lost for that port or that the port has been disabled (interface status changed from 'UP' to 'DOWN'). This behavior is controlled via the MAC Address Learning configuration file TLV as defined in (see subsection C.1.2.18 in Annex C). When the MAC Address Learning Control sub-TLV is set to 'Remove', if the CM determines that a CMCI link has been lost or that the interface has been administratively disabled, the CM MUST initiate the MAC Address Learning Holdoff timer and perform the following for dynamically learned MAC addresses associated with the CMCI.

  • If the link is re-established on the interface or the interface status is transitioned back to 'UP' before the timer expires, the modem clears the timer and no further action is taken.
  • If the timer expires without re-establishing link or without the interface status transitioning back to 'UP', the CM removes all learned MAC addresses associated with the interface on which link was lost, and transmits a CM-Status Message indicating the MAC addresses that were removed (if such reporting has been enabled).Once a MAC address has been removed, the CM is able to continue acquiring MAC addresses up to the maximum permitted as noted above. The MAC address learning configuration TLV is not applicable to a statically provisioned MAC address or eSAFE MAC addresses, and therefore does not affect the learning and retention of those addresses in any way.

C.1.1.7 Maximum Number of CPEs

The maximum number of CPEs which can be granted access through a CM during a CM epoch. The CM epoch is the time between startup and hard reset of the modem. The maximum number of CPEs MUST be enforced by the CM.

NOTE: Type

This parameter should not be confused with the number of CPE addresses a CM may learn. A modem may learn Ethernet MAC addresses up to its maximum number of CPE addresses (from the subsection MAC Address Acquisition in Section 9). The maximum number of CPEs that are granted access through the modem is governed by this configuration setting.

Length Value

18 1

The CM MUST interpret this value as an unsigned integer. The non-existence of this option, or the value 0, MUST be interpreted by the CM as the default value of 1.

NOTE: This is a limit on the maximum number of CPEs a CM will grant access to. Hardware limitations of a given modem implementation may require the modem to use a lower value.

This however, while good advise for normal DHCP set-ups, will probably not work for the DOCSIS modem case, just because the modem will not give out new leases after a shoer while. (That is why in the past people went great length to clone the MAC address from the first device connected to the modem into their OpenWrt routers, in that case the modem will still re-new the lease).

Was going to try the dance this morning. Logging into Luci shows only lan connections, no wan.

Hoping it's a bonding issue.

I was referring to the computers themselves since the op has a different subnet for the pi lan vs the old router lan. With that in mind, a quick cable down/up will force the computer to initiate a dhcp request.

I agree that the “dance” with the cable modem is also likely necessary. But that had already been stated by @dlakelan.

Meanwhile, since it seems it is still not working, please post the latest config files (network, firewall, dhcp).

1 Like

It was the dance. Not really understanding why the Mac was bound when the router was powered down for a good ten minutes. But letting everything bit up and become ready before connecting the modem to the switch was the ticket.

Thanks for all the help on getting this system up and running!

Sure, can you please do some testing with it and report back some performance numbers? I will suggest some tests later, working on a data analysis project at the moment, but I'd really like to see how the RPi 4 behaves as a router using some real world numbers.

For a start though, can you run a dslreports speed test and then link to the results page? http://www.dslreports.com/speedtest

It would be great if you do it while logged into the router and running top -d 1 from the shell and watching the idle % to see what the lowest it gets is during the test.

1 Like

probably because the binding is done at the head-end or some local office, not in the modem itself.

2 Likes

I'll have to connect the other router and run the test at some later date to get results. Speed wise, is about the same. There is some slowdown when running 2 consecutive speed test from different devises (mobile/desktop) But I don't know where to look to find the bottleneck. could be the switch, could be the modem, could be the rpi4. So long as we're no longer crashing the router during heavy(er) use.

I'll have to do more reading/research on how to setup SQM QoS properly.

Test Results

CPU & Idle both remain at 0%, while the nic is at @ 99% Might be missing some install to be able to report those numbers.

you are probably seeing something like this:

Mem: 188552K used, 323412K free, 1188K shrd, 2852K buff, 10688K cached
CPU:   0% usr   0% sys   0% nic 100% idle   0% io   0% irq   0% sirq

nic = "nice" or "low priority" user process... it's probably at 0 (the number is to the left of the label) while idle is probably at 99%.

is that true even during 100Mbps download? I'd expect idle to drop to something, maybe 75% or like that during the peak download phase. Perhaps you can re-look now that you oriented to the fact that the number is to the left of the name

idle % drops to a low of 97% during download, hovers at 98%. idle stays steady at 99% during upload.

every test site renders the same 97-98% idle and close to the same download speeds..

1 Like

Wow, that's a significant amount of processor power if routing and firewall of multiple TCP streams at 100Mbps total costs only 3 or 4% of CPU!

Since I'm guessing you have DOCSIS 3.0 / 3.1 with the built-in queue management and therefore get reasonable bufferbloat stats without any SQM you might be all set as is. I do imagine SQM will increase the CPU usage a bit, but I'd still expect it to be 90% or 85% idle or something (part of it is there are 4 cores, and only one gets involved in queueing at any given time).

Enjoy your setup, I suspect we need someone with a 300+ Mbps stream and need for SQM before we will get numbers that help us estimate the performance envelope of the little Pi 4