WPA3/SAE authentication fails for macOS when using sae_password_file on mt76/filogic

Installing and Using OpenWrt Network and Wireless Configuration
1

Hello folks! I've tried once again to switch over to WPA3 (SAE) with per-vlan PSKs. This general configuration (per-vlan PSK) works well in my environment with WPA2. I'm just trying to step into the future (:smiling_face_with_sunglasses:) and get onto WPA3. I've tried this several times in the past and have failed, ultimately.

I saw some recent [accepted] PRs with awesome work @rany2 has been doing and thought I'd give this a go again with those updates. I've been working on this extensively with Claude's help and we've hit an impasse. The summary I'm going to post below the horizontal line was the summary Claude help me pull together based on hours and hours of testing. Looking for anyone who has thoughts or ideas after reading the summary. Thanks!

SAE authentication fails for macOS clients when using sae_password_file on mt76/filogic

Summary

WPA3-SAE authentication fails for macOS clients when hostapd is configured with sae_password_file or sae_password (multi-password mode), but succeeds when using a single wpa_passphrase. iPhone clients work in all configurations.

The failure occurs after successful SAE exchange - the AP reports "did not acknowledge authentication response" and the Mac reports error -3906.

Environment

Access Point

  • Device: GL.iNet MT6000 (Flint 2)
  • SoC/WLAN: MediaTek MT7986AV / MT7976 (mt798x-wmac)
  • OpenWrt: SNAPSHOT r32834-a495a09bd9
  • Target: mediatek/filogic (aarch64_cortex-a53)
  • Kernel: 6.12.59
  • hostapd: v2.12-devel
  • Driver: mt76 (mt798x-wmac 18000000.wifi)

Clients

  • Failing: MacBook Pro M3 Max (Mac15,9), macOS 26.1 (25B78), WiFi 6E (Broadcom BCM4388)
  • Working: iPhone 15 Pro Max, iOS 26.1

Minimal Configuration to Reproduce

config wifi-iface 'wpa3test'
    option device 'radio1'
    option mode 'ap'
    option ssid 'TestSSID'
    option encryption 'sae'
    option ieee80211w '2'
    option sae_pwe '1'
    option key 'testpassword123'
    option ppsk '1'
    option dynamic_vlan '2'
    option vlan_tagged_interface 'br-lan'

From my understanding: option ppsk '1' causes OpenWrt to generate sae_password_file=/var/run/hostapd-phyX-apY.sae in the hostapd config instead of using wpa_passphrase.

Corresponding sae_password_file content:

testpassword123|vlanid=123

Test Results

Password Configuration iPhone 15 Pro MacBook Pro M3
wpa_passphrase (single key, no VLAN) :white_check_mark: Works :white_check_mark: Works
sae_password_file (with vlanid) :white_check_mark: Works :cross_mark: Fails
sae_password_file (no vlanid) :white_check_mark: Works :cross_mark: Fails
sae_password inline via UCI :white_check_mark: Works :cross_mark: Fails

Failure Symptoms

AP side (logread -f):

daemon.info hostapd: wlan1: STA 60:3e:5f:xx:xx:xx IEEE 802.11: authentication OK (SAE)
daemon.info hostapd: wlan1: STA 60:3e:5f:xx:xx:xx IEEE 802.11: did not acknowledge authentication response

Note: SAE completes successfully ("authentication OK"), but the subsequent frame is not ACKed.

Mac side (log stream --predicate 'process == "airportd"'):

Info: <airport[503]> <en0> Failed to associate to Wi-Fi network, returned error code -3906, retrying
Info: <airport[503]> <en0> Failed to associate to Wi-Fi network, returned error code -3906, no more attempts
[corewifi] AUTO-JOIN: Join FAILED (duration=876ms, error=(-3906 '(null)'), network=(... security=wpa3-sae ...))

Error -3906 = APPLE80211_REASON_UNSPECIFIED_AUTH_FAILURE

Observations

Key finding: The only variable that changes behavior is whether hostapd uses a single wpa_passphrase vs. sae_password_file/sae_password. All other settings remain identical.

Possible causes (speculative):

  • Timing difference in password lookup code path
  • Different SAE state machine behavior when multiple passwords are configured
  • Frame sequencing difference that macOS WiFi stack doesn't handle well
  • mt76-specific interaction with hostapd's SAE implementation

Notable: The SAE exchange itself completes ("authentication OK" logged), but the subsequent authentication response frame is not acknowledged by the Mac.

Attempted Mitigations (none successful)

# SAE timing parameters
uci set wireless.wpa3test.sae_sync='5'
uci set wireless.wpa3test.sae_anti_clogging_threshold='0'
uci set wireless.wpa3test.sae_confirm_immediate='1'
uci set wireless.wpa3test.dot11RSNASAERetransPeriod='1000'

Also tried:

  • Password file in tmpfs (/var/run/) vs persistent storage
  • Single password in file without VLAN specifier
  • Inline sae_password via UCI (bypassing file entirely)

Steps to Reproduce

  1. Configure WPA3-SAE SSID with ppsk=1 and dynamic_vlan=2
  2. Populate sae_password_file with: yourpassword|vlanid=XX
  3. Attempt to connect from macOS client
  4. Observe failure in logs

Expected Behavior

SAE authentication should complete successfully regardless of whether password is configured via single wpa_passphrase or sae_password_file.

Workaround

Use wpa_passphrase instead of sae_password_file. This works but loses per-device VLAN assignment capability for WPA3-SAE networks.

Request

  1. Has anyone else seen this behavior with mt76 + macOS + sae_password_file?
  2. Is this a known issue in hostapd upstream or mt76?
  3. Any suggested debug steps (e.g., hostapd debug logging, packet captures)?

Happy to provide additional logs, test patches, or capture 802.11 frames if that would help diagnose.

Additional Information

This could be:

  • mt76 driver issue: SAE handling in mt76 for multi-password mode
  • hostapd issue: Different code path when using password file vs. single passphrase
  • macOS quirk: BCM4388 may have stricter timing requirements than iPhone's Qualcomm modem

If this needs to go upstream to hostapd (w1.fi), let me know and I can file there as well.

Possibly related to SAE offload capabilities or lack thereof on mt7976.

2

This is (usually) an interop problem