Issue trying to subnet using openWRT

Hi,
I'm trying to setup a server which i want to use a different subnet for. First of some info.
Im using a ASUS RT-AC57U running OpenWrt 22.03.5.
I have a modem from my isp connected on the WAN port in bridge mode. Internet works fine. And i dont think this should affect things. In my OpenWrt router i then have my PC and my server connected on a lan port each. I have a single interface on br-lan with 2 ip adresses 192.168.1.1/24 and 192.168.100.1/24.
Skärmbild 2023-10-09 205618

What im trying to do is to set my server to 192.168.100.2/24 and my PC and the rest of my network is on 192.168.1.0/24 including my PC. I've added two routing rules:


My firewalls setting is to accept forwards in the LAN zone.

I can ping from my PC to the Server but i can see in wireshark that i get ICMP redirects with the goal gateway set to the same as the host adress as my server and it works. But i cannot ping from my server to my PC. I can see that the messages reaches my PC and that it responds to the host address but does not seam to route it to the other gateway so that my PC shows.

I can SSH into the server without issues but i cannot reach a website hosted by it. I can see on my PC when i try to reach the website that i get alot of TCP retransmissions. Sometimes i reach the site but i cant interact with it and i see the retransmissions again.

Anyone have any clues what the issue could be?

The hope was to set these two subnets in different firewall zones so that i can manage them. I later realized this might not be possible. But i still just want it to work on different subnets. I would perfer not to put it on separate VLAN's since i dont know the interface i would use it on at this time.

You need separate LAN interfaces for each subnet, but not necessarily VLANs. Add the appropriate port to each interface. Put them both in the LAN firewall zone (or different zones if that's what you're after).

With Linux you can put as many layer 3 networks as you want on an interface. Also the tcp IP model does not prevent or forbid to have multiple networks on a link layer in general...

...but no security can be enforced between those "networks" as they are all on the same layer 2 link.

2 Likes

Doese this mean i have to remove one of my 4 ports from my br-lan and use that port as a separate interface that i then put in the same firewall zone?
I tried before with a alias interface that was an alias to @lan but that did not work either.

I have some trouble to parse your thoughts. Could you please post some examples like the redirects or tcpdumps. What OS does your server use?
Ping from PC to server works and SSH too? But http not?

Http works sporadicaly. I can sometimes load the login page but never login.
Same with ping from my server. Actually the first ping in a while goes well and then it stops.
Example ping from server -> PC

root@proxmox1:~# ping 192.168.1.20
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
64 bytes from 192.168.1.20: icmp_seq=1 ttl=127 time=1.05 ms
From 192.168.100.1: icmp_seq=2 Redirect Host(New nexthop: 192.168.1.20)
64 bytes from 192.168.1.20: icmp_seq=2 ttl=127 time=1.32 ms
64 bytes from 192.168.1.20: icmp_seq=3 ttl=127 time=1.18 ms
64 bytes from 192.168.1.20: icmp_seq=4 ttl=127 time=1.13 ms
^C
--- 192.168.1.20 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 1.050/1.168/1.317/0.097 ms
root@proxmox1:~# ping 192.168.1.20
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
^C
--- 192.168.1.20 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4078ms

The server is Debian 12 base. It's a fresh install of Proxmox.

when i try to reach the HTTP webpage i get this:

where the redirect message tells my PC to go directly to the host adress.

Have you checked the proxmox manual about icmp redirect and other related sysctl settings as well as their default firewall rules?

I will dig into that tmrw. It’s a good point. But from what I understand there should not be any redirects sent from the router in the first place? They’re literally in the same router and from what I understood it’s just needed if there’s another gateway that’s more direct?

Yes I think this is correct. My thoughts were based on the idea that maybe the proxmox default config drops/rejects some ICMP messages, or do not act based on them.
But I'm not entirely sure why the router sends the redirect in the first place at all. Because the PC in Subnet A sends to the Gateway A and the PC has no address in Subnet B, so it would not be able to reach Gateway/Router B directly. (Yes the PC can reach it directly because its layer 2 but it has no IP config for the other network. But linux is really liberal what addresses are accepted / will be answered. This could play into it, too. TL;DR; from a "network design" perspective its still not a good idea. Try to rule out that proxmos stands in your way, an then lets see...)

While @_bernd is correct that you can have multiple subnets on the same interface, the reality is that this is not going to work properly. Not only does it mean that there is a lack of security enforcement, but it also means that routing may not work as expected and if a DHCP server is used on both of the networks, this could lead to unpredictable results.

Yes. You would simply split out the port, then create an interface that is attached to that port. Provided that the zone level 'forward' rule is set to accept, your two networks will be routed to each other seamlessly.

3 Likes

Okay it's a new day and i have made new discoveries.
I set it up as @psherman said. That is, removed the physical lan port my Proxmox is on from the br-lan and made a new interface on that physical port. The new Interface now has a IP on the 192.168.100.0/24 subnet.I put that new Interface on the same FW zone as my LAN. And now everything works as it should. No more unexpected redirects. Pinging works as expected and the web config gui works as expected.

@_bernd Maybe there is a issue since the br-lan tries to act as a layer 2 switch while the router is trying to act as a router, leading to a confilict. That together with that my Proxmox setup might not listen to ICMP redirects. That would kind of make sense since from what i read it's a security risk.

Is there at this point a difference between putting this new interface on a separate FW zone and adding rules as i wish or making this interface a separate VLAN and then doing the same?

I also want to change this server to a another physical interface later on. Might be via a switch or via wireless. in the case of via a switch am i then going to have to use a layer 3 switch through which i can pass VLAN's? And in the case of Wireless i assume i will have to make a separate SSID?

Also i would like to thank you guys for putting in time to answer these questions. You are amazing.

Not really.

Depends what else you plan to connect to the switch. If all the devices are within the same subnet then an unmanaged switch will be fine. However, if you'll be connecting devices on multiple subnets then a managed switch would be better suited.

I believe so.

1 Like

That is exactly what a bridge is... a simple unmanaged layer 2 switch, but defined in software rather than hardware.

This isn't where the conflict is... rather, it's the fact that you were trying to put two subnets on the same L2 interface (both untagged), which can cause unexpeced behaviors. Further, while I can't describe the reasons for all of the odd results you experienced, I can say that L3 (routing) only happens between two interfaces/subnets. L2 (switching) is what happens between hosts on the same subnet, so the router isn't involved. In the unusual case of having a single interface with 2 subnets, routing is not guaranteed to work properly because you would not need to cross between two interfaces.

If you use VLANs on this switch, you only need a smart/managed/VLAN-aware switch. These are usually L2. L3 switches do exist, but they incorporate special hardware for actual routing between subnets (as compared to a switching fabric). Switches with L3 capabilities are typically expensive, high end devices.

2 Likes

Thanks again for your guys(or girls) input. I've learned alot through this.

I will mark this solved now.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.