Trouble getting batman-adv working

emike · May 22, 2019, 8:28pm

I'm using:

https://openwrt.org/docs/guide-user/network/wifi/mesh/batman

to set up a simple mesh network on two Linksys WRT1900ACS routers.

I can't get past these two parts in the instructions:

Once applied, mesh0 should be seen in the output of ip link . The operation of the mesh can be confirmed with iw dev mesh0 station dump . You should see the other peers in the listing with mesh plink: ESTAB indicating that the peering has been successful. This can be checked prior to the changes in /etc/config/network being made by commenting-out option network 'nwi_mesh0

iw dev mesh0 station dump return nothing.

If I continue and setup /etc/config/network, I never get a 'bat0' interface, and 'batctl o' replies:

Error - interface bat0 is not present or not a batman-adv interface

I've check the instructions many many times, but still can't get past here.

My 'wireless':

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '36'
        option hwmode '11a'
        option path 'soc/soc:pcie/pci0000:00/0000:00:01.0/0000:01:00.0'
        option htmode 'VHT80'
        option country 'US'
        option legacy_rates '1'

config wifi-device 'radio1'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
        option htmode 'HT20'
        option country 'US'
        option legacy_rates '1'

config wifi-iface 'mesh0'
        option device 'radio0'
        option ifname 'mesh0'
        option network 'nwi_mesh0'
        option mode 'mesh'
        option mesh_fwding '0'
        option mesh_id 'stlmesh'
        option encryption 'psk2+ccmp'
        option key 'xxxx'

My network:

config interface 'loopback'
        option ifname 'lo'    
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'                                     
                             
config globals 'globals'   
        option ula_prefix 'fd3e:ed24:18e9::/48'

config interface 'lan'     
        option type 'bridge'  
        option ifname 'eth0.1 bat0'
        option proto 'static'
        option netmask '255.255.255.0'                                 
        option ip6assign '60'
        option ipaddr '192.168.0.10'
                               
config interface 'wan'
        option ifname 'eth1.2'
        option proto 'static' 
        option ipaddr '192.168.1.10'
        option netmask '255.255.255.0'
        option gateway '192.168.1.1'
        option dns '75.75.75.75'
                                
config interface 'wan6'              
        option ifname 'eth1.2' 
        option proto 'dhcpv6'
 
config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'
 
config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 5t'
 
config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '4 6t'
 
config interface 'nwi_mesh0'
        option ifname 'mesh0'
        option proto 'batadv'
        option mesh 'bat0'
        option mtu '2304'

Kind of running out of ideas ...

jeff · May 22, 2019, 8:44pm

Have you confirmed that you can get an unencrypted mesh connected?

Does iw phy show that your chips and their drivers are capable of 802.11s operation?

jeff@office:~$ iw phy | fgrep mesh
		 * mesh point
		 * #{ managed } <= 2048, #{ AP, mesh point } <= 8, #{ P2P-client, P2P-GO } <= 1, #{ IBSS } <= 1,
		 * mesh point
		 * #{ AP, mesh point } <= 8, #{ managed } <= 1,

If not, and you're running the -CT drivers, you may need to install the "upstream" (non-CT) ones for older chips (as is the case on my Archer C7v2, shown above).

emike · May 22, 2019, 9:05pm

iw phy | fgrep mesh

	 * mesh point
	 * #{ AP } <= 16, #{ mesh point } <= 1, #{ managed } <= 1,
	 * mesh point
	 * #{ AP } <= 16, #{ mesh point } <= 1, #{ managed } <= 1,

jeff · May 22, 2019, 9:16pm

OK, that's a good sign that your chips/drivers support 802.11s

Checking unencrypted mesh and then confirming that wpad-mesh (and not wpad-full, last I checked) is installed would be my next steps.

emike · May 24, 2019, 8:38pm

A little more information. I followed the instructions here:
https://www.open-mesh.org/projects/batman-adv/wiki/Batman-adv-openwrt-config#Batman-adv-201300-till-20190-2

But when I look at the interfaces, I get this:

Unfortunately, I don't know if this is even wrong, as I don't have a working one anywhere to compare to.

Again, any help would be appreciated.

jeff · May 24, 2019, 10:42pm

Any instructions written prior to March, 2019 are outdated as the configuration approach of batman-adv changed dramatically

Edit: Looks like your link was already updated

There are some good details in the commit message there, as well as "amendments" to the procedure on the wiki page linked just below.

I haven't updated https://openwrt.org/docs/guide-user/network/wifi/mesh/batman yet, but the steps for getting the mesh up and encrypted are still the basically the same. I might switch over to the EA8300s this weekend and, if I do, that will involve "new" configuration of batman-adv.

https://openwrt.org/docs/guide-user/network/wifi/mesh/batman now updated as of 2019-05-25

Fresh-flashed EA8300 from a build of master at commit f21c8577bb (HEAD as of tonight). Added a slew of packages that I use for management and debug to the build. Notable for 802.11s and batman-adv (and it's monitoring and debug) include:

CONFIG_PACKAGE_kmod-batman-adv=y
CONFIG_PACKAGE_batctl-full=y
# CONFIG_PACKAGE_wpad-basic is not set
CONFIG_PACKAGE_wpad-mesh-openssl=y

ath10k-firmware-qca4019-ct - 2018-10-10-d366b80d-1
ath10k-firmware-qca9888-ct - 2018-10-10-d366b80d-1
kmod-ath - 4.14.120+4.19.32-1-2
kmod-ath10k-ct - 4.14.120+2019-05-08-f98b6dc4-1
batctl-full - 2019.2-0
kmod-batman-adv - 4.14.120+2019.2-0
libopenssl1.1 - 1.1.1b-5
wpad-mesh-openssl - 2018-12-02-c2c6c01b-6

Edited /etc/config/wireless to include

config wifi-device 'radio5pci'
	option type 'mac80211'
	option channel 'X'
	option hwmode '11a'
	option path 'soc/40000000.pci/pci0000:00/0000:00:00.0/0000:01:00.0'
	option htmode 'VHT80'
	option require_mode 'ac'
#	option disabled '1'

config wifi-iface 'mesh0'
        option device 'radio5pci'
        option ifname 'mesh0'
        option network 'nwi_mesh0'
        option mode 'mesh'
        option mesh_id '<redacted>'
        option mesh_fwding '0'
        option encryption 'psk2+ccmp'
        option key '<redacted>'

With no other changes, not even in /etc/config/network it connects to my existing mesh (three other stations)

root@test:~# iw dev mesh0 station dump
Station c6:6e:1f:xx:xx:xx (on mesh0)
	inactive time:	50 ms
	rx bytes:	40640
	rx packets:	439
	tx bytes:	966
	tx packets:	7
	tx retries:	0
	tx failed:	0
	rx drop misc:	2
	signal:  	-87 [-88, -92, -95, -95] dBm
	signal avg:	-86 [-88, -91, -95, -95] dBm
	Toffset:	366695690570 us
	tx bitrate:	6.0 MBit/s
	rx bitrate:	6.0 MBit/s
	rx duration:	0 us
	last ack signal:-95 dBm
	mesh llid:	0
	mesh plid:	0
	mesh plink:	ESTAB
	mesh local PS mode:	ACTIVE
	mesh peer PS mode:	UNKNOWN
	mesh non-peer PS mode:	ACTIVE
	authorized:	yes
	authenticated:	yes
	associated:	yes
	preamble:	long
	WMM/WME:	yes
	MFP:		yes
	TDLS peer:	no
	DTIM period:	2
	beacon interval:100
	connected time:	17 seconds
Station 1a:d6:c7:xx:xx:xx (on mesh0)
	inactive time:	50 ms
	rx bytes:	38909
	rx packets:	418
[...]

8: mesh0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DORMANT group default qlen 1000
    link/ether 32:23:03:xx:xx:xx brd ff:ff:ff:ff:ff:ff

(working through this step-by-step here)

Add pretty much "stock" batman-adv config from the git commit

config interface 'bat0'
        option proto 'batadv'
        option routing_algo 'BATMAN_IV'
        option aggregated_ogms 1
        option ap_isolation 0
        option bonding 0
        option fragmentation 1
        #option gw_bandwidth '10000/2000'
        option gw_mode 'off'
        #option gw_sel_class 20
        option log_level 0
        option orig_interval 1000
        option bridge_loop_avoidance 1
        option distributed_arp_table 1
        option multicast_mode 1
        option network_coding 0
        option hop_penalty 30
        option isolation_mark '0x00000000/0x00000000'

config interface 'nwi_mesh0'
#       option ifname 'mesh0'
        option mtu '2304'
        option proto 'batadv_hardif'
        option master 'bat0'

Links look reasonable:

8: bat0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether ea:cb:a5:xx:xx:xx brd ff:ff:ff:ff:ff:ff
9: mesh0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2304 qdisc noqueue master bat0 state UP mode DORMANT group default qlen 1000
    link/ether 32:23:03:xx:xx:xx brd ff:ff:ff:ff:ff:ff

Neighbors present

root@test:~# batctl n
[B.A.T.M.A.N. adv openwrt-2019.2-0, MainIF/MAC: mesh0/32:23:03:xx:xx:xx (bat0/ea:cb:a5:xx:xx:xx BATMAN_IV)]
IF             Neighbor              last-seen
        mesh0	  c6:6e:1f:xx:xx:xx    0.730s
        mesh0	  32:b5:c2:xx:xx:xx    0.950s
        mesh0	  1a:d6:c7:xx:xx:xx    0.140s

Add an interface that uses the mesh to transport IP traffic

config interface 'vlanNNNN'
        option type 'bridge'
        option stp '1'
        option ifname 'eth0.NNNN bat0.NNNN'
        option proto 'static'
        option ipaddr 'X.X.X.X'
        option netmask '255.255.255.0'
        option delegate '0'

        option gateway 'X.X.X.1'
        option dns 'X.X.X.Y'

works

root@test:/etc# ping www.google.com
PING www.google.com (172.217.5.100): 56 data bytes
64 bytes from 172.217.5.100: seq=0 ttl=53 time=14.152 ms
64 bytes from 172.217.5.100: seq=1 ttl=53 time=14.574 ms
64 bytes from 172.217.5.100: seq=2 ttl=53 time=15.158 ms

markbirss · May 25, 2019, 12:24pm

@jeff I will also try again encrypted mesh with your notes. Thanks

emike · May 28, 2019, 2:05pm

Thanks Jeff! Lots of information.

I was running the latest stable release 18.06.2, which as a date of January 2019.

To use your instructions, I need to get the latest development snapshot?

I was using openwrt-18.06.2-mvebu-cortexa9-linksys-wrt1900acs-squashfs-factory.img, so I'll download:
openwrt-mvebu-cortexa9-linksys_wrt1900acs-squashfs-sysupgrade.bin
and try again.

Thanks again.

jeff · May 28, 2019, 2:21pm

Likely, yes, to get the current versions of the batman-adv and its configuration scripts.

http://downloads.openwrt.org/releases/18.06.2/packages/arm_cortex-a9_neon/routing/Packages.manifest suggests that those packages are being built from a package feed that uses the 2018.1 drop of the B.A.T.M.A.N. source (why did they put in all those annoying dots in the name?? )

emike · May 29, 2019, 6:36pm

Thanks again for you help Jeff! Updating to the latest snapshot, and following your directions have helped.

However, I can't get past this point: After updating /etc/config/wireless, my 'ip link' shows "mesh0" as down:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 532
    link/ether 24:f5:a2:bf:20:23 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 532
    link/ether 26:f5:a2:bf:20:23 brd ff:ff:ff:ff:ff:ff
7: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 26:f5:a2:bf:20:23 brd ff:ff:ff:ff:ff:ff
8: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether 26:f5:a2:bf:20:23 brd ff:ff:ff:ff:ff:ff
9: eth1.2@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 24:f5:a2:bf:20:23 brd ff:ff:ff:ff:ff:ff
10: nwi_mesh0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 06:b1:52:03:dc:72 brd ff:ff:ff:ff:ff:ff
11: mesh0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master br-lan state DOWN qlen 1000
    link/ether 00:25:9c:14:72:e1 brd ff:ff:ff:ff:ff:ff
12: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP qlen 1000
    link/ether 26:f5:a2:bf:20:24 brd ff:ff:ff:ff:ff:ff
13: wlan0-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP qlen 1000
    link/ether 26:f5:a2:bf:20:25 brd ff:ff:ff:ff:ff:ff

Never mind the nwi_mesh0, I continued on "just in case". "Ifconfig mesh0 up" returns:
ifconfig: SIOCSIFFLAGS: Operation not permitted

/etc/config/wireless:

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '36'
        option hwmode '11a'
        option path 'soc/soc:pcie/pci0000:00/0000:00:01.0/0000:01:00.0'
        option htmode 'VHT80'
        option country 'US'
        option legacy_rates '1'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option macaddr '26:f5:a2:bf:20:25'
        option ssid 'stlmanet'
        option encryption 'psk2+ccmp'
        option key 'Shadow2012'

config wifi-device 'radio1'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
        option htmode 'HT20'
        option country 'US'
        option legacy_rates '1'

config wifi-iface 'default_radio1'
        option device 'radio1'
        option network 'lan'
        option mode 'ap'
        option macaddr '26:f5:a2:bf:20:24'
        option ssid 'stlmanet'
        option encryption 'psk2+ccmp'
        option key 'Shadow2012'

config wifi-iface 'mesh0'
        option device 'radio0'
        option ifname 'mesh0'
        option network 'nwi_mesh0'
        option mode 'mesh'
        option mesh_id 'stlmesh'
        option mesh_fwding '0'
        option key 'Shadow2012'
        option encryption 'psk2+ccmp'

jeff · May 29, 2019, 11:38pm

Two ways to check what might be going on with your mesh0 interface that I would try are shown below, with output from one of my devices where mesh1 is the one associated with bat0.

I'd also check that there is a wpa_supplicant running. (Note that I have "full" versions of many utilities installed, ps w gives "wide" output for the "default" version.)

jeff@office:~$ iwinfo 
[...]
mesh1     ESSID: unknown
          Access Point: 1A:D6:C7:xx:xx:xx
          Mode: Mesh Point  Channel: 149 (5.745 GHz)
          Tx-Power: 30 dBm  Link Quality: 36/70
          Signal: -74 dBm  Noise: -108 dBm
          Bit Rate: 263.2 MBit/s
          Encryption: unknown
          Type: nl80211  HW Mode(s): 802.11nac
          Hardware: 168C:003C 0000:0000 [Qualcomm Atheros QCA9880]
          TX power offset: none
          Frequency offset: none
          Supports VAPs: yes  PHY name: phy0
[...]

and

jeff@office:~$ iw dev mesh1 station dump | sed -e 's/..:..:.. (/xx:xx:xx (/'
Station 32:b5:c2:xx:xx:xx (on mesh1)
	inactive time:	70 ms
	rx bytes:	620348208
	rx packets:	4942308
	tx bytes:	377665481
	tx packets:	969382
	tx retries:	0
	tx failed:	4
	rx drop misc:	3673
	signal:  	-66 [-80, -66, -85] dBm
	signal avg:	-66 [-80, -66, -86] dBm
	Toffset:	18446741337357090778 us
	tx bitrate:	351.0 MBit/s VHT-MCS 8 80MHz VHT-NSS 1
	rx bitrate:	650.0 MBit/s VHT-MCS 7 80MHz short GI VHT-NSS 2
	rx duration:	296428040 us
	mesh llid:	0
	mesh plid:	0
	mesh plink:	ESTAB
	mesh local PS mode:	ACTIVE
	mesh peer PS mode:	ACTIVE
	mesh non-peer PS mode:	ACTIVE
	authorized:	yes
	authenticated:	yes
	associated:	yes
	preamble:	long
	WMM/WME:	yes
	MFP:		yes
	TDLS peer:	no
	DTIM period:	2
	beacon interval:100
	connected time:	159998 seconds
Station c6:6e:1f:xx:xx:xx (on mesh1)
	inactive time:	80 ms
	rx bytes:	97648368
	rx packets:	1005861
	tx bytes:	59605855
	tx packets:	48778
	tx retries:	0
	tx failed:	3
	rx drop misc:	543
	signal:  	-75 [-83, -77, -82] dBm
	signal avg:	-75 [-81, -77, -84] dBm
	Toffset:	18446741217709822209 us
	tx bitrate:	263.4 MBit/s VHT-MCS 2 80MHz VHT-NSS 3
	rx bitrate:	195.1 MBit/s VHT-MCS 2 80MHz short GI VHT-NSS 2
	rx duration:	53478228 us
	mesh llid:	0
	mesh plid:	0
	mesh plink:	ESTAB
	mesh local PS mode:	ACTIVE
	mesh peer PS mode:	ACTIVE
	mesh non-peer PS mode:	ACTIVE
	authorized:	yes
	authenticated:	yes
	associated:	yes
	preamble:	long
	WMM/WME:	yes
	MFP:		yes
	TDLS peer:	no
	DTIM period:	2
	beacon interval:100
	connected time:	40348 seconds

jeff@office:~$ ps aux | fgrep mesh1
root      1681  0.6  2.1   4164  2644 ?        Ss   Apr26 333:36 /usr/sbin/wpa_supplicant -B -P /var/run/wpa_supplicant-mesh1.pid -D nl80211 -i mesh1 -c /var/run/wpa_supplicant-mesh1.conf -C /var/run/wpa_supplicant -H /var/run/hostapd/wlan0-2

15: mesh1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 2304 qdisc noqueue master bat0 state UP mode DORMANT group default qlen 1000
    link/ether 1a:d6:c7:xx:xx:xx brd ff:ff:ff:ff:ff:ff

emike · May 30, 2019, 7:20pm

Thanks. More helpful stuff!

Output of iwinfo:

mesh0     ESSID: unknown
          Access Point: 00:25:9C:14:72:E1
          Mode: Mesh Point  Channel: unknown (unknown)
          Tx-Power: 23 dBm  Link Quality: unknown/70
          Signal: unknown  Noise: -89 dBm
          Bit Rate: unknown
          Encryption: unknown
          Type: nl80211  HW Mode(s): 802.11nac
          Hardware: 11AB:2A55 11AB:0000 [Marvell 88W8864]
          TX power offset: none
          Frequency offset: none
          Supports VAPs: yes  PHY name: phy0

NOTE: no Signal, Bit Rate info. It does know hardware, etc.

wpa_supplicant is not running.
'logread' shows this:

Thu May 30 14:11:04 2019 daemon.notice netifd: radio0 (1778): ip: SIOCSIFFLAGS: Operation not permitted
Thu May 30 14:11:04 2019 daemon.notice netifd: radio0 (1778): Interface 1 setup failed: IFUP_ERROR
Thu May 30 14:11:04 2019 daemon.notice netifd: Network device 'wlan0-1' link is up
Thu May 30 14:11:04 2019 daemon.notice netifd: Network device 'wlan1' link is up

I'm assuming that the radio0 errors are for 'mesh0', as 'wlan0-1' and 'wlan1' continue with link up.

jeff · May 30, 2019, 7:47pm

Does it support 802.11s at all? (Edit: I don't know, I don't have any Marvell wireless)

You may be able to use "ad-hoc" if it doesn't

emike · June 5, 2019, 2:52pm

Sorry for being such a nube
I tried this:

config wifi-iface 'adhoc0'
option device 'radio0'
option ifname 'adhoc0'
option mode 'adhoc'
option bssid '02:CA:FE:CA:CA:40'
option ssid 'stlmesh'

readlog gives me this:

user.notice mac80211: Failed command: iw phy phy0 interface add adhoc0 type adhoc

When I run it manually I get this:

iw phy phy0 interface add adhoc0 type adhoc

command failed: Not supported (-95)

I don't have the proper packages installed? (I installed wpad-full)
I don't have the config right?
I can't do this on a WRT1900ACS?

Sorry, and thanks.

mk24 · June 5, 2019, 2:55pm

Run iw list, see if "IBSS" (which means adhoc) is one of the supported interface types. I don't think it is on that hardware.

In that case you'd use mode mesh instead. All the mesh points have to use the same mode. This is fine if you're setting up your own mesh. If you're trying to interact with an existing adhoc mesh, it can't be done.

jeff · June 5, 2019, 3:18pm

If you run iw phy you can look for two sections to know what each radio supports with the installed driver and firmware, below from a GL-AR300M

	Supported interface modes:
		 * IBSS
		 * managed
		 * AP
		 * AP/VLAN
		 * monitor
		 * mesh point
		 * P2P-client
		 * P2P-GO
		 * outside context of a BSS

	valid interface combinations:
		 * #{ managed } <= 2048, #{ AP, mesh point } <= 8, #{ P2P-client, P2P-GO } <= 1, #{ IBSS } <= 1,
		   total <= 2048, #channels <= 1, STA/AP BI must match, radar detect widths: { 20 MHz (no HT), 20 MHz, 40 MHz }

emike · June 5, 2019, 8:11pm

Thanks! I run iw list, and for phy0 and phy1, get:

Supported interface modes:
	 * managed
	 * AP
	 * AP/VLAN
	 * monitor
	 * mesh point

emike · June 5, 2019, 8:14pm

I'm getting closer. Thanks for the help.
I tried:

config wifi-iface 'mesh0'
option device 'radio0'
option ifname 'mesh0'
option mode 'mesh'
option bssid '02:CA:FE:CA:CA:40'
option ssid 'stlmesh'

Now 'iw link' shows 'mesh0' but it's DOWN. And logread shows:

Failed command: iw dev mesh0 set channel 36 VHT80

emike · June 5, 2019, 8:28pm

Got past the failed command by editing /etc/config/wireless, and changing VHT80 to HT20 for radio0 and radio1.

'ip link' still shows 'mesh0' as down:

11: mesh0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master br-lan state DOWN qlen 1000

emike · June 5, 2019, 8:36pm

iw mesh0 mesh join stlmesh
command failed: Network is down (-100)