Dawn: a decentralized wireless controller

Community Builds, Projects & Packages
47

An important bugfix was pushed to hostapd:

Please upgrade to latest snapshot. Probably hostapd can crash when using dawn.
More Information:

1 Like
48

As a consequence for you, maybe running dawn on stable 21.02 will cause hostapd crashes and so actually a worse situation for your wireless connections. I will have a look if we can backport everything to 21.02. Maybe until then switch to snapshot, if you can.

I need some time to answer to all of your other questions. I was more interested in removing all the crashes of dawn, and also hostapd. :slight_smile:

1 Like
49

@PolynomialDivision - I may be able to provide a hint regarding the "sticky behavior" some people are experiencing with DAWN.

Note: this was from a DAWN build (using latest github source) 1-2 months ago. DAWN was seemingly unstable on my hardware, and my family's patience for internet trouble dried up, so Im not currently running DAWN and cant test anything. Sorry.

That said, when I was using DAWN I wrote a script to group the hearing map output in different ways (available HERE - calling this with argument showHM_groupAP is particularly useful). I noticed that some of the items were wrong for non-local AP's. I forget all the specifics but remember that all non-local AP's were shown as not having HT or VHT support (i.e., in a 2 AP setup, AP1 always scores AP2 as if it had no HT/VHT support for any device, and AP2 always scores AP1 as if it had no HT/VHT support for any device). This always increased the score of the current access point, which I imagine tended to make clients sticky.

I hope this may be of use in debugginh things. Good luck!

1 Like
50

I see something similar actually on my end with 3 APs. It seems the hearing map on each AP shows False for HT Sup for other APs than itself. However the scores remain the same on all APs I believe because by default ht_support and vht_support are 0 in the config.

AP1

Client MAC AP MAC Frequency HT Sup VHT Sup Signal RCPI RSNI Channel Utilization Station connect to AP Score
CLIENT1 AP3 2.437 GHz Channel: 6 False False -53 -1 -1 58.43 % 0 134
AP1 2.437 GHz Channel: 6 True False -73 -1 -1 37.65 % 1 94
AP1 5.765 GHz Channel: 153 True False -76 -1 -1 6.67 % 3 113
AP2 2.437 GHz Channel: 6 False False -48 -1 -1 57.65 % 0 144

AP2

Client MAC AP MAC Frequency HT Sup VHT Sup Signal RCPI RSNI Channel Utilization Station connect to AP Score
CLIENT1 AP3 2.437 GHz Channel: 6 False False -53 -1 -1 59.22 % 0 134
AP1 2.437 GHz Channel: 6 False False -73 -1 -1 37.65 % 1 94
AP1 5.765 GHz Channel: 153 False False -76 -1 -1 6.67 % 3 113
AP2 2.437 GHz Channel: 6 True False -48 -1 -1 57.65 % 0 144

AP3

Client MAC AP MAC Frequency HT Sup VHT Sup Signal RCPI RSNI Channel Utilization Station connect to AP Score
CLIENT1 AP3 2.437 GHz Channel: 6 True False -53 -1 -1 59.22 % 0 134
AP1 2.437 GHz Channel: 6 False False -73 -1 -1 37.65 % 1 94
AP1 5.765 GHz Channel: 153 False False -76 -1 -1 6.67 % 3 113
AP2 2.437 GHz Channel: 6 False False -48 -1 -1 57.65 % 0 144
1 Like
51

However the scores remain the same on all APs I believe because by default ht_support and vht_support are 0 in the config.

True. I had forgotten that in my setup I had made these non-zero, which made the scores for a given STA on a given band differ from AP to AP. Actually, I think that investigating why the scores differed was actually what led me to discovering this DAWN error.

Interesting note: If I remember correctly, the no_ht_support and no_vht_support metrics still were correct for non-local AP's, so if instead of using

option ht_support '100'
option no_ht_support '0'
option initial score '0'

you used

option ht_support '0'
option no_ht_support '-100'
option initial score '100'

then the computed scores were the same as if the ht_support=100 case scores were being evaluated correctly for non-local AP's.

Note: the following is mostly wild speculation on my part.

That said, even if (with default config) this doesnt change the DAWN metric scores, I could see it still causing sticky behavior IF this (incorrect) info about the [V]HT support of neighboring AP's is getting passed on the the STA's somehow. Being that

  1. the "end goal" of roaming (from the STA's viewpoint at least) is to switch a STA to a new AP so that STA gets faster speeds,

  2. the STA ultimately decides which AP to connect to,

  3. non-[V]HT speeds are considerably lower than [V]HT speeds, and

  4. I would guess that most devices that are mobile and that we actually care about wifi speed on (phones, tablets, laptops, etc) probably support [V]HT speeds...now days most 802.11a/b/g-only devices probably dont move around much

I could see a device with a [V]HT connection to their current AP not wanting to switch to another AP that is advertised as not having [V]HT support.

Again, wild speculation, but I could even see this explaining the (deleted?) comment re: setting bss_transition=0 helped make some devices less sticky. Specifically, if those devices determine the capabilities of neighboring AP's by

  • bss_transition=1 --> rely on info sent from their current AP about neighboring AP's

  • bss_transition=0 --> determine capabilities of neighboring AP's independently

52

Per this comment, there is no need to set bss_transition for Dawn.

Also worth noting is that you don't even need to set 80211k and 80211v either.

(In my previous reply I mentioned a different reason for setting bss_transition to 0 - that I wasn't using Dawn anymore. I am now using it once again, but still bss_transition is set to zero regardless... or in my case I am simply not including a bss_transition setting in /etc/config/wireless)

53

Should be zero, per this comment by the author himself.

54

Thanks!

The only thing is that the comment is still open to interpretation I think no?

ie he say “no you don’t need it, because I set it.”

I think that lines up more with your comment 2 above: That I guess it doesn’t matter because Dawn overrides the value anyways

55

Hello @PolynomialDivision, I built a snapshot image to use the latest version of Dawn with latest snapshot of hostapd. Here is my report.

  1. I set "option kicking '1'". It seems to work for my iPhone, but I recall seeing my iPhone roam even on a previous installation of Dawn without this option enabled. (How can I be sure that Dawn is sending out those BSS transition requests to devices like my iPhone to get them to switch APs, assuming this is what 'kicking' does? Is there a logging option that I can enable in places like /etc/init.d/dawn or something like that to allow me to see when Dawn is actually issuing commands to kick devices from one AP to another?)

  2. I also changed "option kicking_threshold" from 20 to 15, assuming that it means the difference in scores based on which Dawn will decide to kick devices out of the current AP with a lower score to the AP with the highest score. It SEEMS to work for my iPhone (again without any log that shows when Dawn is actually kicking devices, I cannot say for certain).

  3. I understand in your own environment, you are setting up your Dawn routers as dumb APs with a wired backbone connecting these APs. But I believe many others use 802.11s and batman to wirelessly create a hub of APs that use Dawn. With batman, I encounter this issue where the neighbor AP list takes a LONG time to populate (as long as one hour or so). This is very similar to the issue reported here quite some time ago.

  4. Also I am seeing these errors in logread for each of my routers:

Mon Nov 22 22:08:08 2021 kern.warn kernel: [ 7175.548272] br-lan: received packet on bat0 with own address as source address (addr:xx:xx:xx:xx:xx:xx, vlan:0)
Mon Nov 22 22:08:18 2021 kern.warn kernel: [ 7185.788576] br-lan: received packet on bat0 with own address as source address (addr:xx:xx:xx:xx:xx:xx, vlan:0)
Mon Nov 22 22:08:28 2021 kern.warn kernel: [ 7196.028277] br-lan: received packet on bat0 with own address as source address (addr:xx:xx:xx:xx:xx:xx, vlan:0)

One error every 10 second, and it continues as long as Dawn is running. If you could point me to the details on what command Dawn passes on to umdns once every 10 second, I could try filing a bug report over at umdns github to see if the umdms devs will respond. (I think a vast majority of Dawn users will be using a wireless backbone like batman, so fixing this, plus the problem on neighbor list population would mean a lot to many of us)

  1. I have an Android phone that just doesn't play well with 802.11v. I set both eval_auth_req and eval_assoc_req to 1 to see if the AP will simply deny access to force my Android to switch to another AP, but nope. Even when the current AP has a score 30 points lower than the adjacent AP, no switching happened. Is it because the kicking option takes precedence (I have kicking set to 1 also) and Dawn continues to steer through BSS_tm_req messages?

  2. On a related note, may I make a request that should make Dawn even more useful: option to allow for a list of MAC addresses to use the 'eval_auth_req' and 'eval_assoc_req' methods and for the rest of the devices not in this MAC list to fall under 'kicking'. Because like you said somewhere else, 'eval_auth_req' and 'eval_assoc_req' are primarily used to cater for legacy devices. And in a network environment, typically there will be a mix of legacy (bad) devices that do not comply with 802.11v, and modern (good) devices that do.

  3. Sorry, this is something that I feel a strong urge to say... the documentation (or lack thereof) is probably driving a lot of people away from otherwise a very powerful and useful tool that to me has the potential to become a core part of Openwrt. This is probably the only place with a bit of official information: https://github.com/berlin-open-wireless-lab/DAWN, but even there the info is seriously outdated. For example, 'kicking' isn't even there :slight_smile: . Also I end up having to read a lot of other posts along with many guess work to give myself enough confidence of the meaning of the scoring system in general, and a lot of the options.

Edit: for item 4 above - turns out the problem was not related to umdns / dawn at all. It was the way I configured batman that caused the loops. Fixed it by adding in wifi-iface: option macaddr 'xx:xx:xx:xx:xx:xx' to change the mac address of the mesh interface.

2 Likes
56

I'm enabling the features here:

That is a very interesting finding. Actually bss_tm_req should kick it because we set some timer?

But I just realized sometimes I use del_client_interface. I need further investigation into it.

If you want to help me, feel free to open a wiki artikel about your findings and config options and I am happy to read the things and correct them if they are wrong?

Currently, I lack on a good testbed, and I have more important coding tasks in my limited free time to do. :confused: If you want to start coding for DAWN, I am happy to help. Or please write issues on GitHub, and then I can rewrite DAWN code and add more debug output that you need. I take it with me that I should write some log, that shows the steering decisions?

1 Like
57

Can you create a ring buffer that has the last ~50 decisions that can be read from /proc ?

58

Sure. Can you point to some example code, I can use as reference?

59

Are you in kernel space or userspace only? I think proc is kernel
Kernel space example

60

Userspace only.

61

To be honest I was gonna give up on Dawn - as it wasn't really working in my network environment, but....

... after I actually went back from snapshot to release build 21.02.1 without dawn, by accident I bumped into this old comment from you: How does rrm work? - #36 by PolynomialDivision

I then went ahead and try without any expectation but very much to my surprise, both my iPhone (old 6s+) and my Android (Xiaomi Mi11) could be steered at will to the AP of my choice ! All I did was

ubus call hostapd.wlan0-1 wnm_disassoc_imminent '{"addr":"xx:xx:xx:xx:xx:xx", "duration": 120, "neighbors":"$sam"}'

where $sam is

sam=$(ssh root@<ip of another AP> ubus call hostapd.wlan0-1 rrm_nr_get_own)

... works like magic. So now I have high hopes for Dawn again all of a sudden. But too bad last time I did any sort of programming was something like 20 years ago so it will be difficult for me to determine from your source code if kicking means 'wnm_disassoc_imminent' is issued (nor will I be able to start contributing to your github project even though I am willing to do it).

I will soon set up Dawn once again in snapshot Openwrt, and try to determine whether wnm_disassoc_imminent was somehow never triggered in Dawn, or something went wrong during wnm_disassoc_imminent causing the transition to have never taken place for my phones. At least now I know what to look for in logread: any appearance of BSS-TM-RESP xx:xx:xx:xx:xx:xx status_code=0 bss_termination_delay=0 target_bssid=yy:yy:yy:yy:yy:yy as I can see these logs all over my manual tests.

If Dawn ends up working well in my environment, I will be more than happy to contribute to your future wiki (or your README.md in github) :+1:

(so far for Dawn I can see a scoring system that seems to calculate the scores well, plus I know hostapd's wnm_disassoc_imminent is used to steer/kick devices around. What seems to be missing for me is a clear sign of linkage between these two aspects of Dawn. But I am happy there is progress in my testing)

62

With the latest snapshot build of OpenWrt + Dawn on all of my routers (just two in total to be exact), plus the knowledge I have accumulated over the past two weeks or so particularly hostapd's ubus wnm_disassoc_imminent method, I can now consistently reproduce an issue with Dawn. I thought of opening an issue on GitHub, but this is such a basic core feature of Dawn that really shouldn't be failing so I am writing here first for @PolymomialDivision to see if I have overlooked anything first before filing a bug report.

Here is the crux of the issue: I cannot get Dawn to send out any wnm_disassoc_imminent command whatsoever.

  1. I have option kicking set to '1' on both of my routers.

  2. I can see the other AP with ubus call hostapd.wlan0-1 rrm_nr_list. So the neighbor list has been appropriately populated. I can also see both routers on "Network Overview" on luci interface with devices connected to each.

  3. I make sure my device (will expose the MAC addresses for the sake of clarity, fingers crossed no actual damage will be done by exposing MAC addresses) 9C:BC:F0:A1:10:B7 responds to roaming instructions by manually constructing and sending wnm_disassoc_imminent requests to the device. Sure enough each time it responds by switching to the other AP, plus I see a line like hostapd: wlan0-1: BSS-TM-RESP 9c:bc:f0:a1:10:b7 status_code=0 bss_termination_delay=0 target_bssid=10:6f:3f:3d:67:64 in syslog.

  4. I then make sure the device 9C:BC:F0:A1:10:B7 is connected to router 10:6F:3F:3D:6E:8D, and move the device to a place where 10:6F:3F:3D:6E:8D's signal is weaker than 10:6F:3F:3D:67:64.

  5. Check LUCI's Dawn Hearing Map, and it shows (appropriately) that 10:6F:3F:3D:6E:8D has a score 20 plus points lower than that of 10:6F:3F:3D:67:64, like so

Dawn_possible_bug
Dawn_possible_bug1406×167 12.6 KB

  1. Waited until I ran out of patience. Device is still connected to 10:6F:3F:3D:6E:8D and no sign of moving to 10:6F:3F:3D:67:64.

  2. Checked syslog. I do not see any BSS-TM-RESP message other than the ones that I have triggered manually like in step 3 above. This to me means Dawn is not generating any wnm_disassoc_imminent like it should not only for 9C:BC:F0:A1:10:B7, but for any of my other devices that are connected to my network as well that are eligible for switching according to the Hearing Map.

Like I said, I feel this is such a trivial function of Dawn to be able to issue wnm_disassoc_imminent messages, I am really surprised to find out zero BSS-TM-RESP syslog messages are appearing in either of my two routers. I am hesitant to file a bug at this stage... could very well be something that I have overlooked?

(note: I see lots of other Dawn-related hostapd logs like BEACON-REQ-TX-STATUS and BEACON-RESP-RX. I stop Dawn and these logs stop getting generated. So I think Dawn is functional)

1 Like
63

Thanks for all your great documentation and bug reports. I will later answer on it in more detail. Just to be sure DAWN is sending out wnm_disassoc_imminent you can just use "ubus monitor" to see if DAWN actually sends requests to hostapd over the bus.

And we already have debug messages. If you enable stdout in dawn-procd file you should also see those messages:

1 Like
64

Btw reading the discussion here, I have also enabled "kicking" with DAWN and changed min RSSI to 70dB still some clients are hanging to original AP even when stronger AP exists, while they report between 75-79dB...

config network
        option broadcast_ip '10.0.0.255'
        option broadcast_port '1025'
        option tcp_port '1026'
        option network_option '2'
        option shared_key 'Niiiiiiiiiiiiick'
        option iv 'Niiiiiiiiiiiiick'
        option use_symm_enc '0'
        option collision_domain '-1'
        option bandwidth '-1'

config ordering
        option sort_order 'cbfs'

config hostapd
        option hostapd_dir '/var/run/hostapd'

config times
        option update_client '10'
        option denied_req_threshold '30'
        option remove_client '15'
        option remove_probe '30'
        option remove_ap '460'
        option update_hostapd '10'
        option update_tcp_con '10'
        option update_chan_util '5'
        option update_beacon_reports '600'

config metric 'global'
        option rssi_weight '0'
        option rssi_center '0'
        option initial_score '0'
        option kicking_threshold '20'
        option duration '600'
        option rrm_mode 'apt'
        option ap_weight '0'
        option ht_support '0'
        option vht_support '0'
        option no_ht_support '0'
        option no_vht_support '0'
        option rssi '0'
        option low_rssi '0'
        option freq '0'
        option chan_util '0'
        option max_chan_util '0'
        option rssi_val '-60'
        option low_rssi_val '-75'
        option chan_util_val '0'
        option max_chan_util_val '0'
        option min_probe_count '0'
        option bandwidth_threshold '6'
        option use_station_count '0'
        option max_station_diff '0'
        option eval_probe_req '1'
        option eval_auth_req '1'
        option eval_assoc_req '1'
        option deny_auth_reason '1'
        option deny_assoc_reason '17'
        option use_driver_recog '1'
        option chan_util_avg_period '3'
        option set_hostapd_nr '1'
        option kicking '1'

config metric '802_11a'
        option rssi_weight '2'
        option rssi_center '-70'
        option initial_score '125'

config metric '802_11g'
        option rssi_weight '2'
        option rssi_center '-70'
        option initial_score '100'

Any idea why it's not kicked out and roam to another AP ? When I disconnect it manually, it immediately connects to other AP (on hearing map I see both APs), with Debug enabled I don't see roaming happening however.
Shall I further decrease option low_rssi_val or adjust other parameter ? Thank you.

65

Yet another observation that I'd like to bring to your attention.

(Dawn has no direct control over this, but...)

I have discovered the wnm_disassoc_imminent actually makes my Android phone's wifi performance much worse after a transition induced by it (bandwidth reduced to less than 30% of usual). My Apple "burner phone" doesn't seem to be impacted however. Problem reproducible every time.

I then tried the other methods exposed by hostapd, namely del_client and even the newest bss_transition_request. Of the three, only del_client allows my phone to retain its normal bandwidth after connecting to another AP.

(bss_transition_request downright doesn't work. After several transitions it totally breaks down. Clients disconnect before reconnecting to another AP).

I realize using del_client would mean sacrificing fast transition 802.11r (observed from my testing). But at least my phone still doesn't lose connection altogether during roaming, and I don't have to deal with reduced bandwidth.

So for now I won't be experimenting with Dawn anymore unfortunately. Will definitely keep an eye on further developments plus possibly improvements on hostapd so it will be compatible with (my model of) Android phones. I truly think Dawn has huge potential, when it becomes 100% stable and functional.

(for those interested, this tool does make use of del_client: Wi-Fi roaming recipe)

Edit 7 days later - I have since discovered that my Android phone does not play well with Openwrt's 802.11r. So I disabled it, and wnm_disassoc_imminent no longer makes an impact on my phone's data throughput anymore. So the culprit seems to be 802.11r and not wnm_disassoc_imminent. Also, I collaborated with the author of wifi-disconnect-low-signal (i.e. the tool I mentioned that's in this link: Wi-Fi roaming receipe) so now it makes use of wnm_disassoc_imminent instead of del_client. With it my wifi roaming solution is finally complete at least for now. Of course it is primitive compared with Dawn, but until Dawn can actually "kick" clients to other APs, I will stick with my current solution.

1 Like
66

I have noticed that there is no Settings menu item in DAWN menu in Luci in the latest snapshot realease, is it normal ? Settings for dawn were present previousely in luci