Before I state my issue, I'll describe my network topology on the wan side of my router.
I have two internet links coming into my house. At the point of entry, the two modems are connected to the input taps and then plugged into a switch. One of these ports is a hardware tagged VLAN ID 101 and the other is the default VLAN, untagged.
A single cat6 cable runs from this switch to another switch in my office, where my main openwrt router is located. I have two physical wan ports on my main router which plug into this external network switch. One port is untagged and the other port is tagged with VLAN 101.
These represent my wan and wan2 interfaces. Both modems are operating in bridge mode.
I have private IP addresses setup on both modems and corresponding aliases on the wan interfaces so that I can access the modems' management interfaces. This all works fine and both modems can be accessed from my internal network.
I've also got a separate openwrt router running on a linksys WRT1200AC with its lan port plugged into the switch at the point of entry, ie., on the external network. Firewall switched off, all interfaces except lan disabled. It also has an IP address on the same private network aliased to the wan interface and the modems, default VLAN - so no tagging.
I use this box to run an iperf server so I can test various settings / qos / loads etc at gigabit speeds through my main router. It has two open ports that are being listened on: port 80 for luci and port 22 for ssh.
The curious thing is that most of the time when I restart my main openwrt router, then connections to this linksys box cannot be made. With tcpdump on the wan interface, I can see the packet go out and I can also see the linksys box reply, but the client gets a connection refused error.
Accidentally, while trying to see if these ports were actually open, I discovered that if login to my main openwrt and run a single nmap against the linksys box, all of a sudden the ports that are open become visible and I can connect to them from the client machine on my lan.
I am somewhat puzzled as to why this is happening. I'm wondering if maybe it is some kind of arp related issue.
Anybody have any bright ideas?