Creating a Mesh Network (with OLSR Protocol) for multiple Access Points

philipdaely · October 20, 2019, 11:24am

Hello everyone, above is a simple diagram of project. I use several Raspberry Pi 3s as mesh nodes for this project. I have made each of them connected to each other with OLSR protocol using this tutorial https://justingoetz.net/display/PB/2019/04/10/Comprehensive+guide+to+running+OLSR+over+WPA2+on+OpenWRT. Below is a Wireless Configuration Overview of one RPi that is connected to internet via ethernet port.

config fi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'platform/soc/3f300000.mmc/mmc_host/mmc1/mmc1:0001/mmc1:0001
        option htmode 'HT20'
        option disabled '0'
        option legacy_rates '1'
        option country 'KR'

config wifi-iface
        option device 'radio0'
        option mode 'adhoc'
        option ssid 'MeshBackhaul'
        option network 'mesh meshnetw'
        option encryption 'psk2'
        option key 'meshnetw'

What I want to do next is how to make all those interconnected RPis provide internet access to client devices in their coverage. I have tried to create a regular access point, so the /etc/config/wireless became as below

config fi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'platform/soc/3f300000.mmc/mmc_host/mmc1/mmc1:0001/mmc1:0001
        option htmode 'HT20'
        option disabled '0'
        option legacy_rates '1'
        option country 'KR'

config wifi-iface
        option device 'radio0'
        option mode 'adhoc'
        option ssid 'MeshBackhaul'
        option network 'mesh meshnetw'
        option encryption 'psk2'
        option key 'meshnetw'

config wifi-iface
        option device 'radio0'
        option mode 'ap'
        option encryption 'none'
        option ssid 'SimpleWIFI'
        option network 'lan'

But, when I connect my phone to that "SimpleWIFI" wifi, my phone detected no internet connection.
When the access point was created, in LuCI, the mesh network mode changed from AdHoc to Client, but from SSH it was still AdHoc, which is weird. What should I do to get my phone internet connection through "SimpleWIFI"?

mk24 · October 20, 2019, 1:38pm

The chip and driver likely does not allow combined interfaces like that. The open source driver for Broadcom chips, which is part of OpenWrt is very limited. Other chipsets especially Atheros work better for doing more than one thing at once.

lleachii · October 20, 2019, 1:48pm

Well wait...why does your adhoc mesh link have encryption?

I'm not sure about the combination yet...running iw phy0 info should provide that.

philipdaely · October 20, 2019, 4:38pm

The iw phy0 info command gave output below

Wiphy phy0
        max # scan SSIDs: 10
        max scan IEs length: 2048 bytes
        max # sched scan SSIDs: 16
        max # match sets: 16
        max # scan plans: 1
        max scan plan interval: 508
        max scan plan iterations: 0
        Retry short limit: 7
        Retry long limit: 4
        Coverage class: 0 (up to 0m)
        Device supports T-DLS.
        Available Antennas: TX 0 RX 0
        Supported interface modes:
                 * IBSS
                 * managed
                 * AP
                 * P2P-client
                 * P2P-GO
                 * P2P-device
        Band 1:
                Capabilities: 0x1020
                        HT20
                        Static SM Power Save
                        RX HT20 SGI
                        No RX STBC
                        Max AMSDU length: 3839 bytes
                        DSSS/CCK HT40
                Maximum RX AMPDU length 65535 bytes (exponent: 0x003)
                Minimum RX AMPDU time spacing: 16 usec (0x07)
                HT TX/RX MCS rate indexes supported: 0-7
                Frequencies:
                        * 2412 MHz [1] (13.0 dBm)
                        * 2417 MHz [2] (13.0 dBm)
                        * 2422 MHz [3] (13.0 dBm)
                        * 2427 MHz [4] (13.0 dBm)
                        * 2432 MHz [5] (13.0 dBm)
                        * 2437 MHz [6] (13.0 dBm)
                        * 2442 MHz [7] (13.0 dBm)
                        * 2447 MHz [8] (13.0 dBm)
                        * 2452 MHz [9] (13.0 dBm)
                        * 2457 MHz [10] (13.0 dBm)
                        * 2462 MHz [11] (13.0 dBm)
                        * 2467 MHz [12] (13.0 dBm)
                        * 2472 MHz [13] (13.0 dBm)
                        * 2484 MHz [14] (disabled)
        valid interface combinations:
                 * #{ managed } <= 1, #{ P2P-device } <= 1, #{ P2P-client, P2P-G                                                                                                                                                             O } <= 1,
                   total <= 3, #channels <= 2
                 * #{ managed } <= 1, #{ AP } <= 1, #{ P2P-client } <= 1, #{ P2P                                                                                                                                                             -device } <= 1,
                   total <= 4, #channels <= 1

what do you think?

philipdaely · October 20, 2019, 4:41pm

Is it better if I add a wifi dongle for each RPi, and make the integrated wifi module act as transceiver for mesh network, and the wifi dongle as access point?

markbirss · October 21, 2019, 6:02am

If you followed the complete justingoetz guide, by adding the HNA Announcements for each network and the node that has internet connectivity, that will provide internet access to the other nodes

philipdaely · October 21, 2019, 7:46pm

I have finished setting up two RPis using justingoetz guide. When I tested them, neither of them detect each other's node.

I also tried one by one as follows:

HNA "0.0.0.0 | 0.0.0.0" on RPi 1 (connected to internet via ethernet), HNA empty on RPi 2.
HNA empty on RPi 1, HNA "0.0.0.0 | 0.0.0.0" on RPi 2
Both RPis' HNA " "0.0.0.0 | 0.0.0.0".
Both RPis' HNA empty

but nothing could make either RPi detect each other.

Does anybody know the problem and solution?

markbirss · October 21, 2019, 10:26pm

You have not done the HNA part correctly
The HNA section needs to be populated

Only the node with internet connection you set HNA to 0.0.0.0 and 0.0.0.0

It should have it's own unique network address subnet let's say 192.168.5.0/24 or whatever

The other nodes must have there own network address and HNA also populated

If your other node uses for example 192.168.4.1/24

HNA is 192.168.4.0 255.255.255.0

So each node network have it's own subnet and the subnets HNA information

Now each node should be able to ping each other and devices in that subnet or devices on the internet

philipdaely · October 22, 2019, 4:05pm

I have follow your suggestion @markbirss , but RPi 1 still doesnt detect RPi 2. What is wrong with my configuration?

I have put below the configurations of both RPi 1 and 2

RPi 1
/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd8f:d9ae:4317::/48'

config interface 'lan'
        option type 'bridge'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option proto 'dhcp'
        option ifname 'eth0'

config interface 'meshnetw'
        option proto 'static'
        option ipaddr '192.168.10.2'
        option netmask '255.255.255.0'

/etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'platform/soc/3f300000.mmc/mmc_host/mmc1/mmc1:0001/mmc1:0001
        option htmode 'HT20'
        option legacy_rates '1'
        option country 'KR'

config wifi-iface
        option device 'radio0'
        option mode 'adhoc'
        option ssid 'MeshBackhaul'
        option encryption 'psk2'
        option key 'meshnetw'
        option network 'mesh meshnetw lan'

/var/etc/olsrd.conf

DebugLevel 0
AllowNoInt yes
IpVersion 4
FIBMetric "flat"
LinkQualityLevel 2
LinkQualityAlgorithm "etx_ff"
OlsrPort 698
Willingness 3
NatThreshold 1.0

Hna4
{
                0.0.0.0 0.0.0.0
}

LoadPlugin "olsrd_arprefresh.so.0.1"
{
}

LoadPlugin "olsrd_dyn_gw.so.0.5"
{
}

LoadPlugin "olsrd_httpinfo.so.0.1"
{
        PlParam "Net" "0.0.0.0 0.0.0.0"
        PlParam "port" "1978"
}

LoadPlugin "olsrd_nameservice.so.0.4"
{
        PlParam "name" "OpenWrtRPi1"
}

LoadPlugin "olsrd_txtinfo.so.1.1"
{
        PlParam "accept" "0.0.0.0"
}

LoadPlugin "olsrd_jsoninfo.so.1.1"
{
}

LoadPlugin "olsrd_pgraph.so.1.1"
{
}

InterfaceDefaults
{
}

Interface "wlan0"
{
        Mode "mesh"
}

RPi 2
/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd29:812a:a8cd::/48'

config interface 'lan'
        option type 'bridge'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option proto 'dhcp'
        option ifname 'eth0'

config interface 'meshnetw'
        option proto 'static'
        option ipaddr '192.168.10.3'
        option netmask '255.255.255.0'

/etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'platform/soc/3f300000.mmc/mmc_host/mmc1/mmc1:0001/mmc1:0001
        option htmode 'HT20'
        option country 'KR'
        option legacy_rates '1'

config wifi-iface
        option device 'radio0'
        option mode 'adhoc'
        option ssid 'MeshBackhaul'
        option encryption 'psk2'
        option key 'meshnetw'
        option network 'mesh meshnetw lan'

/var/etc/olsrd.conf

DebugLevel 0
AllowNoInt yes

Hna4
{
                192.168.10.0 255.255.255.0
}

LoadPlugin "olsrd_arprefresh.so.0.1"
{
}

LoadPlugin "olsrd_dyn_gw.so.0.5"
{
}

LoadPlugin "olsrd_httpinfo.so.0.1"
{
        PlParam "Net" "0.0.0.0 0.0.0.0"
        PlParam "port" "1978"
}

LoadPlugin "olsrd_nameservice.so.0.4"
{
        PlParam "name" "OpenWrtRPi2"
}

LoadPlugin "olsrd_txtinfo.so.1.1"
{
        PlParam "accept" "0.0.0.0"
}

LoadPlugin "olsrd_jsoninfo.so.1.1"
{
}

LoadPlugin "olsrd_pgraph.so.1.1"
{
}

Interface "wlan0"
{
        Mode "mesh"
}

markbirss · October 22, 2019, 4:33pm

Just try first using different lan networks as both are on 192.168.1.1

Could be the plugins

I will get back to you when I checked my old project

philipdaely · October 22, 2019, 5:36pm

Thank you sir, I'll be waiting for it

markbirss · October 23, 2019, 9:06am

Ok, i confirmed how to get it working.

The onboard wifi of Raspberry Pi 3 does not support IBBS for Adhoc at all so OLSR mesh wont work (wont work 802.11s either)

Your need USB dongle with mesh support
I use these

These dongle will do OLSR and 802.11s (batman-adv) but without encryption

As stated by @mk24 Atheros Qaulcomm chpsets are known to work well with AP/Mesh even same time

So you are better off running the AP from the onboard wifi and mesh (olsr or 802.11s batman from the usb dongle)

Internet is from my 1 Pi to a 3G wifi router and the others allow connection to internet

markbirss · October 23, 2019, 9:07am

markbirss · October 23, 2019, 9:10am

@philipdaely also take care with the plugin version names as these have changed with new OpenWRT versions and OLSRd will fail if it cannot find the library

Remember also the HNA for each subnet and 0.0.0.0 for the Gateway node as i already stated

/etc/config/wireless


config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11g'
	option path 'platform/soc/3f300000.mmc/mmc_host/mmc1/mmc1:0001/mmc1:0001:1'
	option htmode 'HT20'
	option disabled '0'
	option country 'ZA'
	option legacy_rates '1'
	option channel '9'

config wifi-device 'radio1'
	option type 'mac80211'
	option channel '11'
	option hwmode '11g'
	option path 'platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2:1.0'
	option htmode 'HT20'
	option disabled '0'
	option country 'ZA'
	option legacy_rates '1'

config wifi-iface
	option device 'radio1'
	option network 'mesh'
	option ssid 'penguinfarm'
	option mode 'adhoc'
	option key '********'
	option encryption 'none'

config wifi-iface
	option network 'wwan'
	option ssid 'HUAWEI-D376'
	option encryption 'psk2'
	option device 'radio0'
	option mode 'sta'
	option bssid 'A0:57:E3:C7:D3:76'
	option key '********'




/etc/config/olsrd

config olsrd

config LoadPlugin
	option library 'olsrd_arprefresh.so.0.1'
	option ignore '0'

config LoadPlugin
	option library 'olsrd_dyn_gw.so.0.5'
	option ignore '0'

config LoadPlugin
	option library 'olsrd_httpinfo.so.0.1'
	option port '1978'
	list Net '0.0.0.0 0.0.0.0'
	option ignore '0'

config LoadPlugin
	option library 'olsrd_nameservice.so.0.4'
	option ignore '0'

config LoadPlugin
	option library 'olsrd_txtinfo.so.1.1'
	option accept '0.0.0.0'
	option ignore '0'

config Interface
	option ignore '0'
	option interface 'mesh'
	option Mode 'mesh'

config LoadPlugin
	option library 'olsrd_pgraph.so.1.1'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_jsoninfo.so.1.1'
	option ignore '0'

config LoadPlugin
	option library 'olsrd_p2pd.so.0.1.0'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_watchdog.so.0.1'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_secure.so.0.6'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_bmf.so.1.7.0'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_quagga.so.0.2.2'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_dyn_gw_plain.so.0.4'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_dot_draw.so.0.3'
	option ignore '1'

config LoadPlugin
	option library 'olsrd_mdns.so.1.0.1'
	option ignore '1'

config Hna4
	option netaddr '192.168.51.0'
	option netmask '255.255.255.0'

The HNA of my Gateway chnaged to 0.0.0.0 since it has access to internet via Client STA connection to 3g router

mk24 · October 23, 2019, 6:55pm

What was your experience with the quality of them? I bought several of what looks like the exact same item a few years ago but found many were DOA and others died after a few hours of use.

markbirss · October 23, 2019, 7:48pm

Yes, I have also had same, some failed within first couple hours using them. Quality control not always great and dongles are skew, USB connector at angle.

These are the cheapest dongles and best to buy 5 or 10 of them at a time, since some will fail ir not work

I have tried other same chipset dongles but they much more expensive.

There are not many usable mesh USB dongles to choose from anymore in my opion.

I use them mostly for firmware testing and development.

I recently found some very cheap Aruba AP-105 devices for $3 each that have got recent Openwrt support, but you will need SPI flash skills to hack them

philipdaely · October 23, 2019, 8:05pm

In the active HNA image, why you have 3 other network addr beside the 0.0.0.0?
Sorry, my background in computer network is not so good.

markbirss · October 24, 2019, 12:08am

In the picture there is 1 x Raspberry Pi 3, 2 x ZSun wifi devices on powerbank and 1 x Huawei 3G Router

The Raspberry Pi using 2 HNA network 0.0.0.0 and 192.168.51.0
The ZSun wifi are also nodes 192.168.41 and 192.168.61

markbirss · October 23, 2019, 11:32pm

Each node has it's own access point network configured (192.168.x.0/24 - where x is 41,51 and 61) for clients to connect to

Then each node has a olsr mesh ip 10.0.71.x

Both these have to be unique for each node and node network

You can't use 192.168.1.1 repeatedly

philipdaely · October 24, 2019, 7:46am

Do you bridge all 3 interfaces (LAN, MESH, and WWAN) together for every node? Or do you separate all of them?