Homenet Install on ER-X with OpenWRT 19.07.3 questions

Hello, I'm not new to embedded systems, but I am just now realizing there's a lot I don't understand about networking in general and IPv6 specifically.

After some weeks on reading and learning at OpenWRT.org and related sites, it seemed that HomeNet might be the answer for my situation.

Learning to load OpenWrt onto a factory configured ER-X was greatly helped by a recent build provided by @stman a few weeks ago.

(Also helpfully posted on the ER-X device page.)

After loading this version over the EdgeOS factory firmware, the load of 19.07.3 worked flawlessly. The detailed explanation of the build by stman was very helpful for the learning process.

Now with running OpenWRT firmware, the HomeNet install began. And there were a few speed bumps on this path.

Out of the gate these commands were recommended:

# opkg update
# opkg install luci-ssl-openssl
# /etc/init.d/uhttpd restart
# opkg install ip strace tcpdump
# opkg install ipset
# opkg install hnet-full
  1. installing ip doesn't seem to actually install "ip" ... "ip-full" is installed
  2. hnet-full does not install because of a conflict with odhcpd-ipv6only

The following gets past that issue:

# opkg remove odhcpd-ipv6only
# opkg install hnet-full
  1. going into luci gui admin interface via https now the interface type for HNCP is not available

These commands help here:

# opkg install luci-compat
# opkg install luci-app-hnet
# opkg install luci-proto-hnet
# reboot

The computer I am connected from is a Debian-based linux and there were many games to play with the "help" of the Network Manager. Physical ethernet cable unplug/plug iterations, deleting connection profiles/making new "auto" profiles, rebooting the router, and sometimes just waiting for it to "self-heal." If manual configure of connection didn't make connection, finding the IPv6 port for dropbear could get an address to go in via ssh.

$ ping6 -I enp0s25 ff02::1 (address for "all routers")
$ ssh root@fe80::7a8a:20ff:fe09:d00c%enp0s25

After a couple of rounds involving the use of the router's reset button to return to baseline configuration of OpenWRT, I had one router that was configured similar to Tore Anderson's tutorial from 2015.

Some important differences would be the ER-X does not have any wireless to use for backup when converting the lan bridge to VLAN and the ER-X architecture only has a single eithernet port which is managed via switch configuration. Device (internal:external) mapping is (eth0.1:LAN1, eth0.2:LAN2, eth0.3:LAN3, eth0.4:LAN4, eth0.5:WAN).

Nicely enough, it you create a couple of VLANs for LAN ports 3 and 4 first, you can then move your computer connection to them to complete the changes for the existing 'lan' interface without losing the ability to get into the router.

There were points where the browser interface seemed to setup the configuration better than direct edits to the underlying files: dhcp, network, firewall. I demonstrated this to myself by starting from a fresh baseline configuration and modifying these files to match the ones on a working router and yet it did not have the same result. Maybe luci does something "more" for configuration.

In the end using luci had the better result, but I've not yet been able to load "https://openwrt.home." directly after making the configuration changes. I suspect dhcp/dnsmasq is still not quite right.

In addition, there are some complaints from babeld about not having any interfaces configured. Not sure it's needed.

/etc/config/dhcp:


config dnsmasq
	option domainneeded '1'
	option filterwin2k '0'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option nonegcache '0'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option noresolv '1'
	option strictorder '1'
	list server '64.6.64.6#5353'
	list server '64.6.65.6#5353'
	list server '64.6.64.6#53'
	list server '64.6.65.6#53'
	list server '209.244.0.3#5353'
	list server '209.244.0.4#5353'
	list server '209.244.0.3#53'
	list server '209.244.0.4#53'
	option nonwildcard '1'
	option boguspriv '0'
	option localservice '1'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv6 'server'
	option ra 'server'
	option ra_default '1'
	option ra_management '1'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'
	option maindhcp '1'

Perhaps the interfaces 'lan' and 'wan' need to be changed to the new hnet1/2/3/4 and hnet0 names?

From the hnetd source readme there were explanations about the various types of connections in HNCP. Maybe auto would work, but when things were going sideways, I choose to manually set the WAN to 'external' and the VLANs to 'internal' in hopes of a smoother ride.

/etc/config/network:

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'

config interface 'hnet0'
	option ifname 'eth0.5'
	option dnsname 'hnet0'
	option proto 'hnet'
	option mode 'external'
	option ip6assign '64'
	option ip4assign '24'

config interface 'hnet1'
	option ifname 'eth0.1'
	option dnsname 'hnet1'
	option proto 'hnet'
	option mode 'internal'
	option ip6assign '64'
	option ip4assign '24'
	list ip6class 'local'

config interface 'hnet2'
	option ifname 'eth0.2'
	option dnsname 'hnet2'
	option proto 'hnet'
	option mode 'internal'
	option ip6assign '64'
	option ip4assign '24'
	list ip6class 'local'

config interface 'hnet3'
	option ifname 'eth0.3'
	option dnsname 'hnet3'
	option proto 'hnet'
	option mode 'internal'
	option ip6assign '64'
	option ip4assign '24'
	list ip6class 'local'

config interface 'hnet4'
	option ifname 'eth0.4'
	option dnsname 'hnet4'
	option proto 'hnet'
	option mode 'internal'
	option ip6assign '64'
	option ip4assign '24'
	list ip6class 'local'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option vid '1'
	option ports '1 6t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option vid '2'
	option ports '2 6t'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option vid '3'
	option ports '3 6t'

config switch_vlan
	option device 'switch0'
	option vlan '4'
	option vid '4'
	option ports '4 6t'

config switch_vlan
	option device 'switch0'
	option vlan '5'
	option vid '5'
	option ports '0 6t'

Some options in the network file were added after the basic functionality was in place after reading another discussion about protecting internal IPv6 addresses from leaking over to WAN side. Not sure that is a real thing or not.

/etc/config/firewall:

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option network 'hnet1 hnet2 hnet3 hnet4'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option network 'hnet0'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config include 'miniupnpd'
	option type 'script'
	option path '/usr/share/miniupnpd/firewall.include'
	option family 'any'
	option reload '1'

And now to the actual question that pushed me into posting:

I have two of these routers that I've done my best to configure in an identical manner. Either one connected to my current WRT54GL/Tomato gateway router will allow connection to the internet from a computer on one of their LAN ports. HOWEVER, if they are connected in series with the upstream WAN of one connected to a VLAN port on the other, all access to the internet is gone. Can't ping gateway router or any internet level address (via IPv4 addressing at least).

I have logs and such, just let me know what would be useful for diagnosing this.

Thank you for anyone that has had the patience to read this far.
Any ideas are welcome.