Need help making VXLAN work

Installing and Using OpenWrt
1

Can someone give me a simple example configuration for vxlan?

A simple requirement of mine is to be able to send untagged packets to port 1 and output the same packets but with vxlan id from port 2. So port 1 and port 2/vxlan will need to be bridged.

I have tried the examples in [OpenWrt Wiki] Tunneling interface protocols and it didn’t work. The vxlan interface connectivity status on the bridge vlan tab is never connected like the example says.

I have installed all the necessary packages for vxlan configuration via luci and am configuring the vxlan using luci.

I am using openwrt 24.10.0.

No matter the configuration, the vxlan interface never seem to send out anything.

How to bridge VXLAN to Ethernet
2

After reading various posts on the forum, I have noticed that my vxlan device is missing despite having

  1. created a vxlan interface using luci

  2. created a bridge device bridging one ethernet adapter to the vxlan interface name using luci

  3. created an interface for the bridge device using luci

  4. created an interface for the vxlan to bind to using luci

/etc/config/network:

config interface 'vxlan100'
option proto 'vxlan'
option peeraddr '192.168.3.2'
option ipaddr '192.168.3.1'
option port '4789'
option learning '0'
option vid '100'
option tunlink 'vxlan'

config device
option name 'br-vlan'
option type 'bridge'
list ports 'lan2'
list ports 'vxlan100'

config interface 'vlan'
option proto 'static'
option device 'br-vlan'
option ipaddr '192.168.3.1'
option netmask '255.255.255.0'

config interface 'vxlan'
option proto 'static'
option device 'lan0'
option ipaddr '192.168.2.1'
option netmask '255.255.255.0'

ip -d addr did not show a vxlan device.

EDIT:

Based on Make config change -> vxlan adapter vanishes - Installing and Using OpenWrt - OpenWrt Forum, UCI no longer supports vxlan parameters. I have tried the ip link add command similar to the one in that post but I got an error Unknown device type.

Command:

ip link add "vxlan100" type vxlan id "100" dev "lan0" remote "192.168.3.2" local "192.168.3.1" dstport 4789
3

I answered here:
Other thread

But probably better to continue here.

4

This is "vlan" thinking. You need a new mindset for vxlan!
Vxlan is a tunnelling protocol and does not use tags like vlans.
Vxlan creates virtual lan ports (secret is in the name :grin: ).
These virtual ports are linked to virtual ports somewhere else, via a layer 3 tunnel.

Hope this helps.

What we should do is review what you are actually trying to achieve. Do you really need a vxlan tunnel network?

5

I tried the following command based on your code:

ip link add "vxlan100" type vxlan id "100" dev "lan0" remote "192.168.3.2" local "192.168.3.1" dstport 4789

However, I got the error Unknown device type.

Someone suggested I remove all the double quotes:

ip link add vxlan100 type vxlan id 100 dev lan0 remote 192.168.3.2 local 192.168.3.1 dstport 4789

This resulted in the same error.

Any idea?

6

I am trying to encapsulate the traditional vlan tagged traffic within vxlan packets. This allows me to uniquely identify the traffic when I duplicate an almost identical network setup.

For example:

there are 4 senders node1-4 which I have no authority to modify
network 1 receives traffic tagged for vlan10 (from node1) and vlan20 (from node2)
network 2 also receives traffic tagged for vlan10 (from node3) and vlan20 (from node4)
the networks are connected together (vxlan)
one monitor is connected to network 1 and another is connected to network 2
the monitors need to be able to differentiate the traffic from each nodes.

The vxlan packets also need to be unicast due to a restriction of the connection to be used for tunneling.

7

I'll summarise what I have found:

  1. The Luci/UCI config for vxlan in OpenWrt, along with the wiki docs, seems to be based on very outdated vxlan support. Some people have had limited success using it, but I most definitely have not.
  2. I have found that using ip-full, you can follow the vxlan man pages and get it to work.
  3. On OpenWrt, I have only had success using ipv6 and multicast, I did not need anything else so did not look in depth after initial failure with ipv4 and unicast. I am pretty sure though it should work with ipv4 and unicast.

You are making it sound more complicated than it is.
Vxlan is short for "Virtual Extended Lan". It provides what looks like a virtual ethernet layer 2 network tunnel between "vxlan" peers.
ie Think of it as functioning like a piece of ethernet cable transparently passing traffic at layer 2.
It builds all this on an existing layer 3 network.
As long as the carrying network has a large enough mtu, a traditional vlan trunk network will be passed transparently through the vxlan tunnel to other peers.

This reads like a specification for a contracted task, or even an academic practical test question.
It does not give any useful detail about the underlying infrastructure.

You really need to give some detail if you want people on this forum to be able to help.

8

What I am trying to achieve is as “simple” as what I described. I am not trying to setup a home router. There is no dhcp, no dns, no firewall requirement, no internet access. I simply need two devices that can help distinguish identical traffic.

If what I described does not provide any useful detail, then may I know what is considered useful detail?

EDIT:

I managed to get the vxlan interface after upgrading to v25.12.1

Some things I did differently for this clean upgrade was not installing ip-bridge along side ip-full. ip-bridge was listed as a dependency for luci-proto-vxlan, which was why I had it installed, before installing ip-full after reading your post. I also did not install luci-proto-vxlan.

So either

  1. it was a bug with v24.10.0 or
  2. ip-bridge cannot be installed with ip-full or
  3. luci-proto-vxlan prevents creation of vxlan device
9

Nope. I have it working just fine.

The ip-bridge package provides utilities for managing a bridge.
It does not do anything that would upset ip-full or vxlan.
See:

It does seem to be very outdated, so this would not surprise me.

I wonder why? But it does not effect vxlan, regardless.

If it is now working for you, consider closing this thread, unless you have anything more to add.

10

Unfortunately, despite having created the vxlan device, my device still does not seem to be sending vxlan packets.

sender:

vxlan setup:

ip link add vxlan100 type vxlan id 100 dev lan0 remote 192.168.3.2 local 192.168.3.1 dstport 4789
ip link set vxlan100 up

network setup:

config device
option type 'bridge'
option name 'br-vlan'
list ports 'lan2'
list ports 'vxlan100'

config bridge-vlan
option device 'br-vlan'
option vlan '10'
list ports 'lan2:t'
list ports 'vxlan100:t'

config interface 'vlan'
option proto 'static'
option device 'br-vlan.10'
option ipaddr '192.168.3.1'
option netmask '255.255.255.0'
option multipath 'off'

receiver:

vxlan setup:

ip link add vxlan100 type vxlan id 100 dev lan0 remote 192.168.3.1 local 192.168.3.2 dstport 4789
ip link set vxlan100 up

network setup:

config device
option type 'bridge'
option name 'br-vlan'
list ports 'lan2'
list ports 'vxlan100'

config bridge-vlan
option device 'br-vlan'
option vlan '10'
list ports 'lan2:t'
list ports 'vxlan100:t'

config interface 'vlan'
option proto 'static'
option device 'br-vlan.10'
option ipaddr '192.168.3.2'
option netmask '255.255.255.0'
option multipath 'off'

Traditionally tagged (vlan 10) traffic arrives at sender port lan2. This has been verified via the realtime bandwidth graph of device br-vlan.10.

Ideally, vxlan packet encapsulating this traffic should exit sender port lan0. However, neither the realtime graph of device vxlan100 nor device br-vlan.10 show any outbound traffic.

The ports lan0 of both sender and receiver are connected physically.

11

I have had some success with this a while ago...

There are a few things which are very important the first one is that the DSA device needs to be added in luci and not being greyed.

Then also a full router restart is needed.

Then there is igmp snooping disable it, igmpproxy may be a interfering package too.

The MTU must not exceed, but also be not too low, try first 1280 or a little above this, 1280 is minimum for ipv6 you may need to add a extra length for vxlan to this.

I found that often when a change is made both routers need a full restart.

And of course a firewall rule is needed so that vxlans work.

I stopped using them btw because of the mtu overhead issues I had, cascaded wireguard + vxlan on 1500 isp mtu is a bit too much :face_savoring_food:, I learned it is possible to get some multicast working on the wireguard tunnel for Chromecast so vxlan are no longer needed for me.

it is worth to note that I only made end to end vxlans, but I know that vxlans also have peer to peer functionality I'm not sure if that works via luci app I have had some issues before when leaving the local endpoint empty also the option to bind on a interface can be breaking you may choose to listen on all interfaces to check and then you can try to set it.

12

Lets look at what you have so far.

Presumably, when you run your vxlan setup script, the interface actually does come up, and you checked with ip link to see if vxlan100 is listed - at both ends. If so, this is a good starting point.

Now, I am sure there are many ways to use the vxlan interface, but your network config looks wrong to me.
As I have mentioned before, the concept of vxlan is very different to that of vlan.
There is no "packet tagging" in the vlan sense, where vxlan is concerned.

Look at your vxlan setup script. Here you are setting up the end points of a fixed point to point tunnel using 192.168.3.1 and 192.168.3.2 on the backhaul network between the two nodes, whatever that may be (simplistically an ethernet cable, but could be fibre, wireless, wet-string, anything). Before even starting the vxlan, you must be able to ping 192.168.3.2 from 192.168.3.1 and vice versa. This is a pre-requisite for the setup of the vxlan tunnel.

The 192.168.3.x/24 subnet will be the backhaul layer 3 network that will carry the vxlan tunnel.

But you are vlan-tagging the vxlan interface and assigning the same ipv4 address you are using for the vxlan tunnel! This cannot work.
It is important not to conflate vxlan with vlan, they are entirely different things.

Looking at your configs:

This is the vxlan bridge, for your simple point to point tunnel it should at this stage only have the vxlan interface as a port. It is useful to give it a uci section name in addition to a bridge name, it helps to clarify what is going on to the human mind.

I would replace this section as follows:

config device 'vtunnel'
	option name 'br-tun100'
	option type 'bridge'
	option bridge_empty '1'
	list ports 'vxlan100'

it is important to have the bridge_empty option set as the network will be started before your vxlan-startup script gets run. ie on network startup, vxlan100 won't exist yet.

In your config, you go on to create a vlan port with the very same ip address you are using for the vxlan endpoint - very wrong.

You should remove both these sections.
Restart the network at both ends and run your vxlan setup scripts.

At this point, the vxlan layer 2 tunnel should be up. The entry points will be the br-tun100 bridge.

Then and only then should you add your vlans to the tunnel bridge.

13

Actually, I have seen an example somewhere with a bridge having only the vxlan interface as a port. What does this actually do? Don’t I need the vxlan interface bridged with another physical port (lan2) for the vxlan interface to be able to receive any traffic at all before forwarding through the device set (lan0)?

I never truly understood where the local vtep IP should go. Based a few examples that I have seen, I had assumed that there needs to be a bridge interface assigned with the local vtep IP. So what exactly has this IP? The vxlan device that I created? But do IPs not belong to “interfaces” instead of “devices”? Or, do I need to create an interface of br-tun100 and assign that the local vtep IP?

If I only have a bridge br-tun100 which only has vxlan100 listed as a port, how do I encapsulate traffic arriving at lan2 with vxlan and forward it to my remote vtep via lan0?

I have tried the following to no avail…

Sender

network setup:

config device
option type 'bridge'
option name 'br-vlan'
list ports 'lan0'
list ports 'lan2'

config device
option type 'bridge'
option name 'br-vxlan'
list ports 'vxlan100'
option bridge_empty '1'

config interface 'vlan'
option proto 'static'
option device 'br-vlan'
option ipaddr '192.168.2.1'
option netmask '255.255.255.0'
option multipath 'off'

config interface 'vxlan'
option proto 'static'
option device 'br-vxlan'
option ipaddr '192.168.3.1'
option netmask '255.255.255.0'
option multipath 'off'

vxlan setup (manually entered after router powered on with above setup):

ip link add vxlan100 type vxlan id 100 dev lan0 remote 192.168.3.2 local 192.168.3.1 dstport 4789
ip link set vxlan100 up

Receiver

network setup:

config device
option type 'bridge'
option name 'br-vlan'
list ports 'lan0'
list ports 'lan2'

config device
option type 'bridge'
option name 'br-vxlan'
list ports 'vxlan100'
option bridge_empty '1'

config interface 'vlan'
option proto 'static'
option device 'br-vlan'
option ipaddr '192.168.2.2'
option netmask '255.255.255.0'
option multipath 'off'

config interface 'vxlan'
option proto 'static'
option device 'br-vxlan'
option ipaddr '192.168.3.2'
option netmask '255.255.255.0'
option multipath 'off'

vxlan setup (manually entered after router powered on with above setup):

ip link add vxlan100 type vxlan id 100 dev lan0 remote 192.168.3.1 local 192.168.3.2 dstport 4789
ip link set vxlan100 up
14

You are very much missing the point that a vxlan tunnel provides a new layer 2 network that is carried on an existing layer 3 network.

So before even starting the vxlan (with your vxlan setup script), you must be able to ping each end from the other. This is a pre-requisite for the vxlan tunnel to be active.

It is still not clear what you are trying to achieve.

If this a single point to single point and is not a wireless link, then why do you need a vxlan tunnel?

Are you trying to create a vlan-trunk connection between the two devices over wireless?

15

Yes, I am able to ping.

I am using it because:

  1. As I have mentioned multiple times in this thread, I need to encapsulate traditionally tagged traffic, and I need this because I have identical traffic that are also identically vlan tagged. So vxlan is used to differentiate between these identically vlan tagged identical traffic.
  2. I am also using vxlan to send my traffic that are primarily multicast as unicast through lan0. Because lan0 , though a simple physical wired connection for now, will be equiped with hardware packet modifiers that only work with unicast.

Do not try to overthink my scenario. It is exactly as I have described, at least as a test for now. It may not seem to make sense as a normal use case because my setup, though simple, is not a normal use case.