Fresh ESXI Installation - No Internet

Hello everyone!

I have done a fresh installation of OpenWRT21 using the OVA provided in the community section of this page.

Everything appears to work correctly from the LAN side (Luci works, I can SSH, devices on the network get local IPs) however I don't appear to get an internet connection.

My setup is HFC/DOCSIS Cable Modem ---> x86 Virtual Openwrt ---> TP-Link Wireless AP/Switch.

My home internet is a static IP in Australia through Launtel. As far as I know there is no MAC lock and should just be DHCP from the WAN side. The TP-Link device when in Router mode is picked up instantly and works correctly, which makes me believe this is an issue specific to OpenWRT (or my x86 machine).

What can I do to diagnose this issue?

Thanks.

Why are you using an old and unsupported version? 23.05 is available and supported.

Most of the challenges when using VMs is the supervisor/hypervisor mapping of the physical interface through to OpenWrt.

How many physical network ports do you have in the x86 machine and are they directly mapped?

Why are you using an old and unsupported version? 23.05 is available and supported.

I can try a newer version, but the OVA on the vmware page is only up to version 21 as of now and they're very fast and easy to install hence why I used that.

How many physical network ports do you have in the x86 machine and are they directly mapped?

There are three NICs on the machine. 1 is the built-in motherboard NIC I am using as the management interface and the other two are USB -> Gigabit ethernet adapters which appear to be up and connected in ESXI. They're both on their own vSwitches.

I don't know much about ESXI's specific means of handling the NICs, but in VB you want to use a bridge adapter -- this basically transparently presents the hardware NIC to the VM.

Can you verify that this is the case?

I am no expert either, but as far as I am aware this is the default behaviour for how the NICs are setup in ESXI.

Ok... let's take a look at your current config files (we'll go ahead and use what you have now, but you should really upgrade to 23.05):

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/dhcp
cat /etc/config/firewall
root@OpenWRT-21:~# ubus call system board
{
	"kernel": "5.4.124",
	"hostname": "OpenWRT-21",
	"system": "Intel(R) Core(TM) i5-6500T CPU @ 2.50GHz",
	"model": "VMware, Inc. VMware Virtual Platform",
	"board_name": "vmware-inc-vmware-virtual-platform",
	"release": {
		"distribution": "OpenWrt",
		"version": "21.02.0-rc3",
		"revision": "r16172-2aba3e9784",
		"target": "x86/64",
		"description": "OpenWrt 21.02.0-rc3 r16172-2aba3e9784"
	}
}
root@OpenWRT-21:~# cat /etc/config/network

config interface 'loopback'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'
	option device 'lo'

config globals 'globals'
	option ula_prefix 'fdab:620f:3d62::/48'

config interface 'lan'
	option proto 'static'
	option device 'eth1'
	option ipaddr '192.168.1.1'
	option netmask '255.255.254.0'

config device
	option name 'wan'
	option proto 'dhcp'
	option device 'eth0'

config interface 'wan'
	option proto 'dhcp'
	option device 'eth0'
	option hostname 'ArcherAX73'

config interface 'wan6'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'
	option device 'eth0'

config device
	option name 'eth0'
	option macaddr ''

Note the hostname for WAN and MAC for Eth0 I added to try and spoof a MAC lockout by matching the TP-Link device, I have ommitted the MAC addy

root@OpenWRT-21:~# cat /etc/config/dhcp

config dnsmasq
	option domainneeded '1'
	option boguspriv '1'
	option filterwin2k '0'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option nonegcache '0'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option nonwildcard '1'
	option localservice '1'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'

config dhcp 'lan'
	option interface 'lan'
	option leasetime '12h'
	option dhcpv6 'server'
	option ra 'server'
	option ra_management '1'
	option start '10'
	option limit '300'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'
	list ra_flags 'none'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config domain
	option name 'pornhub.com'
	option ip '0.0.0.0'
root@OpenWRT-21:~# cat /etc/config/firewall
config defaults
	option syn_flood	1
	option input		ACCEPT
	option output		ACCEPT
	option forward		REJECT
# Uncomment this line to disable ipv6 rules
#	option disable_ipv6	1

config zone
	option name		lan
	list   network		'lan'
	option input		ACCEPT
	option output		ACCEPT
	option forward		ACCEPT

config zone
	option name		wan
	list   network		'wan'
	list   network		'wan6'
	option input		REJECT
	option output		ACCEPT
	option forward		REJECT
	option masq		1
	option mtu_fix		1

config forwarding
	option src		lan
	option dest		wan

# We need to accept udp packets on port 68,
# see https://dev.openwrt.org/ticket/4108
config rule
	option name		Allow-DHCP-Renew
	option src		wan
	option proto		udp
	option dest_port	68
	option target		ACCEPT
	option family		ipv4

# Allow IPv4 ping
config rule
	option name		Allow-Ping
	option src		wan
	option proto		icmp
	option icmp_type	echo-request
	option family		ipv4
	option target		ACCEPT

config rule
	option name		Allow-IGMP
	option src		wan
	option proto		igmp
	option family		ipv4
	option target		ACCEPT

# Allow DHCPv6 replies
# see https://dev.openwrt.org/ticket/10381
config rule
	option name		Allow-DHCPv6
	option src		wan
	option proto		udp
	option src_ip		fc00::/6
	option dest_ip		fc00::/6
	option dest_port	546
	option family		ipv6
	option target		ACCEPT

config rule
	option name		Allow-MLD
	option src		wan
	option proto		icmp
	option src_ip		fe80::/10
	list icmp_type		'130/0'
	list icmp_type		'131/0'
	list icmp_type		'132/0'
	list icmp_type		'143/0'
	option family		ipv6
	option target		ACCEPT

# Allow essential incoming IPv6 ICMP traffic
config rule
	option name		Allow-ICMPv6-Input
	option src		wan
	option proto	icmp
	list icmp_type		echo-request
	list icmp_type		echo-reply
	list icmp_type		destination-unreachable
	list icmp_type		packet-too-big
	list icmp_type		time-exceeded
	list icmp_type		bad-header
	list icmp_type		unknown-header-type
	list icmp_type		router-solicitation
	list icmp_type		neighbour-solicitation
	list icmp_type		router-advertisement
	list icmp_type		neighbour-advertisement
	option limit		1000/sec
	option family		ipv6
	option target		ACCEPT

# Allow essential forwarded IPv6 ICMP traffic
config rule
	option name		Allow-ICMPv6-Forward
	option src		wan
	option dest		*
	option proto		icmp
	list icmp_type		echo-request
	list icmp_type		echo-reply
	list icmp_type		destination-unreachable
	list icmp_type		packet-too-big
	list icmp_type		time-exceeded
	list icmp_type		bad-header
	list icmp_type		unknown-header-type
	option limit		1000/sec
	option family		ipv6
	option target		ACCEPT

config rule
	option name		Allow-IPSec-ESP
	option src		wan
	option dest		lan
	option proto		esp
	option target		ACCEPT

config rule
	option name		Allow-ISAKMP
	option src		wan
	option dest		lan
	option dest_port	500
	option proto		udp
	option target		ACCEPT

# include a file with users custom iptables rules
config include
	option path /etc/firewall.user


### EXAMPLE CONFIG SECTIONS
# do not allow a specific ip to access wan
#config rule
#	option src		lan
#	option src_ip	192.168.45.2
#	option dest		wan
#	option proto	tcp
#	option target	REJECT

# block a specific mac on wan
#config rule
#	option dest		wan
#	option src_mac	00:11:22:33:44:66
#	option target	REJECT

# block incoming ICMP traffic on a zone
#config rule
#	option src		lan
#	option proto	ICMP
#	option target	DROP

# port redirect port coming in on wan to lan
#config redirect
#	option src			wan
#	option src_dport	80
#	option dest			lan
#	option dest_ip		192.168.16.235
#	option dest_port	80
#	option proto		tcp

# port redirect of remapped ssh port (22001) on wan
#config redirect
#	option src		wan
#	option src_dport	22001
#	option dest		lan
#	option dest_port	22
#	option proto		tcp

### FULL CONFIG SECTIONS
#config rule
#	option src		lan
#	option src_ip	192.168.45.2
#	option src_mac	00:11:22:33:44:55
#	option src_port	80
#	option dest		wan
#	option dest_ip	194.25.2.129
#	option dest_port	120
#	option proto	tcp
#	option target	REJECT

#config redirect
#	option src		lan
#	option src_ip	192.168.45.2
#	option src_mac	00:11:22:33:44:55
#	option src_port		1024
#	option src_dport	80
#	option dest_ip	194.25.2.129
#	option dest_port	120
#	option proto	tcp

You have 2 wan interfaces... delete one of them and the restart the VM.

Please run two tests:

  1. plug a client computer into the port that is mapped to eth1 and configure it to get an IP via DHCP. Does it successfully obtain an IP address?
  2. I assume you have a router with a DHCP server currently in your network. Please connect a cable between your upstream LAN and the port mapped to eth0 on your VM. Does your VM wan interface get an address (via DHCP as configured)? Don't worry about actual routing yet... we just need to see if it's getting an address.
  1. plug a client computer into the port that is mapped to eth1 and configure it to get an IP via DHCP. Does it successfully obtain an IP address?

This worked fine, I used my macbook and it picked up immediately.

  1. I assume you have a router with a DHCP server currently in your network. Please connect a cable between your upstream LAN and the port mapped to eth0 on your VM. Does your VM wan interface get an address (via DHCP as configured)? Don't worry about actual routing yet... we just need to see if it's getting an address.

I don't believe this was working, but with that said I'm not sure how to make this test work in the way you want as the VM won't get DHCP from the other server until its active/online. However, once it is online OpenWRT is trying to broadcast DHCP from 192.168.1.1 which seems like a conflict?

Ok... so we know eth1 works.

I'm guessing you haven't tested this yet in the manner that I was suggesting -- please confirm.

Do you have a existing router currently in use to provide services for your network? I'm assuming you do since you're actively online.

All we need is a device (such as a router) that has an active DHCP server that is not controlled by the ISP (ISP router lan connection is fine, but not the upstream ISP connection). That's why I'm hoping you have a router currently in use that you would be able to connect to eth0.

Yes... and no.

A conflict/overlap of the subnets on the upstream and downstream networks will be a problem, but only insofar as routing will not function.

If a DHCP server can issue a DHCP lease to your VM router, and your VM router is properly configured, you will see an address show up on the wan. This can be done typically with a router's lan connection (i.e. whatever router is currently in use, or a router you have sitting in a junk drawer probably has a DHCP server enabled)... we don't care what address or subnet it's using, we just need to prove that the VM gets an address.

So... for this test:

  • Connect the a cable between the lan port of your existing router and your macbook. Your macbook should get an address via DHCP.
  • Then disconnect the macbook and plug the cable into the VM host machine -- into the port you've associated with eth0 in the VM. What happens?

Okay I tried your test, no DHCP.

Ok... but I assume the first test (from the upstream DHCP server to your macbook) did work, correct?

Yes that is correct. DHCP from the other server (the TPLink Router) works fine.

Ok... that suggests that the port you've designated as eth0 is not properly mapped through from the hardware > host OS > VM host supervisor/hypervisor > VM.

Check to see what the differences are in the configuration between the first and second USB ethernet adapters (you said that the built-in NIC is for management, and the other two -- USB adapters -- should theoretically be mapped to eth0 and eth1 on your VM, correct).

VMs are tricky for this reason, and it's generally not a good design to use a VM as your main -- it's always better to run bare metal.

That said, we can make a quick change to prove that it the port that is at issue here:

  • Swap eth0 and eth1 in the lan and wan configurations so that it looks like this:
config interface 'lan'
	option proto 'static'
	option device 'eth0'
	option ipaddr '192.168.1.1'
	option netmask '255.255.254.0'

config device
	option name 'wan'
	option proto 'dhcp'
	option device 'eth1'
  • Then reboot the VM and run the experiment again with the upstream DHCP server now connected to eth1 (and also test the downstream between eth0 and your mac)

Beyond that, fixing the port mapping through the chain described above is outside the scope of these forums -- you may wish to check the documentation or help forums for the VM environment itself.

I have not tried swapping the ports to verify the issue yet as I ran out of time today, but I will update this thread if I find a solution. If I cannot get a working solution I think I might swap to ProxMox due to the current licensing problems impacting free use of ESXI.

Thank you for your help.

In general, I advise against running a primary router in vm environments. Bare metal is the way to go for about a dozen reasons.

That said, you could try virtual box - it’s free and easier to use than some of the other virtualization environments.