WiFi for OpenWrt vm

Hi there,

I'm currently working on an installation inside a virtual machine (hypervisor: virtualbox).
It works good that far, but unfortunately I have problems with the WiFi-cards. Installed are two cards, one bgn (AR9287) and one ac (QCA988x), both cards work fine on the host OS (Ubuntu 18.04.3 LTS), I want to hand both cards through to the openwrt guest system. But in openwrt only the bgn card shows up in luci, the other one not.
What I did that far:

  1. Updated firmware-files (from https://github.com/kvalo/ath10k-firmware/blob/master/QCA988X/hw2.0)

Information:
Virtual machine:

user@Server:/root$ VBoxManage showvminfo openwrt
Name:            openwrt
Groups:          /
Guest OS:        Other Linux (64-bit)
UUID:            410aaaf4-f239-436e-8125-bd8ebca200c0
Config file:     /mnt/vm/openwrt/openwrt.vbox
Snapshot folder: /mnt/vm/openwrt/Snapshots
Log folder:      /mnt/vm/openwrt/Logs
Hardware UUID:   410aaaf4-f239-436e-8125-bd8ebca200c0
Memory size:     8192MB
Page Fusion:     off
VRAM size:       16MB
CPU exec cap:    100%
HPET:            off
Chipset:         ich9
Firmware:        BIOS
Number of CPUs:  6
PAE:             on
Long Mode:       on
Triple Fault Reset: off
APIC:            on
X2APIC:          on
CPUID Portability Level: 0
CPUID overrides: None
Boot menu mode:  message and menu
Boot Device (1): Floppy
Boot Device (2): DVD
Boot Device (3): HardDisk
Boot Device (4): Not Assigned
ACPI:            on
IOAPIC:          on
BIOS APIC mode:  APIC
Time offset:     0ms
RTC:             UTC
Hardw. virt.ext: on
Nested Paging:   on
Large Pages:     off
VT-x VPID:       on
VT-x unr. exec.: on
Paravirt. Provider: Default
Effective Paravirt. Provider: KVM
State:           running (since 2019-09-13T18:51:17.713000000)
Monitor count:   1
3D Acceleration: off
2D Video Acceleration: off
Teleporter Enabled: off
Teleporter Port: 0
Teleporter Address:
Teleporter Password:
Tracing Enabled: off
Allow Tracing to Access VM: off
Tracing Configuration:
Autostart Enabled: off
Autostart Delay: 0
Default Frontend:
Storage Controller Name (0):            IDE
Storage Controller Type (0):            PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0):  2
Storage Controller Port Count (0):      2
Storage Controller Bootable (0):        on
IDE (0, 0): /mnt/vm/openwrt/Snapshots/{e0a5cb3b-afd7-466a-be21-9c5057f7d23a}.vdi (UUID: e0a5cb3b-afd7-466a-be21-9c5057f7d23a)
IDE (1, 0): Empty
NIC 1:           MAC: 08002735B8D0, Attachment: Bridged Interface 'enp11s0f0', Cable connected: on, Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: allow-vms, Bandwidth group: none
NIC 2:           MAC: 08002735B8D1, Attachment: Bridged Interface 'enp5s0', Cable connected: on, Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny, Bandwidth group: none
NIC 3:           MAC: 08002735B8D2, Attachment: Internal Network 'intnet', Cable connected: on, Trace: off (file: none), Type: virtio, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: allow-vms, Bandwidth group: none
NIC 4:           disabled
[...]
NIC 36:           disabled
Pointing Device: USB Tablet
Keyboard Device: PS/2 Keyboard
UART 1:          disabled
[...]
UART 4:          disabled
LPT 1:           disabled
LPT 2:           disabled
Audio:           disabled
Audio playback:  enabled
Audio capture:   disabled
Clipboard Mode:  disabled
Drag and drop Mode: disabled
Session name:    headless
Video mode:      720x400x0 at 0,0 enabled
VRDE:            disabled
USB:             disabled
EHCI:            disabled
XHCI:            enabled

USB Device Filters:

<none>

Available remote USB devices:

<none>

Currently Attached USB Devices:

<none>


Attached physical PCI devices:

   Host device host08:00.0 at 08:00.0 attached as 02:00.0
   Host device host09:00.0 at 09:00.0 attached as 02:02.0

Bandwidth groups:  <none>

Shared folders:  <none>

VRDE Connection:    not active
Clients so far:     0

Capturing:          not active
Capture audio:      not active
Capture screens:    0
Capture file:       /mnt/vm/openwrt/openwrt.webm
Capture dimensions: 1024x768
Capture rate:       512 kbps
Capture FPS:        25
Capture options:    ac_enabled=false

Guest:

Configured memory balloon size:      0 MB
OS type:                             Linux_64
Additions run level:                 0

Guest Facilities:

No active facilities.

root@Server_VM1_Router:~# lspci
00:02.0 VGA compatible controller: InnoTek Systemberatung GmbH VirtualBox Graphics Adapter
00:03.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:04.0 System peripheral: InnoTek Systemberatung GmbH VirtualBox Guest Service
00:07.0 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:08.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:09.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:0c.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller
00:18.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f2)
00:19.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f2)
00:1f.0 ISA bridge: Intel Corporation 82801GBM (ICH7-M) LPC Interface Bridge (rev 02)
00:1f.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
02:00.0 Network controller: Qualcomm Atheros QCA986x/988x 802.11ac Wireless Network Adapter
02:02.0 Network controller: Qualcomm Atheros AR9287 Wireless Network Adapter (PCI-Express) (rev 01)
root@Server_VM1_Router:~# ls -l /sys/class/net
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 br-gast -> ../../devices/virtual/net/br-gast
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 br-iot -> ../../devices/virtual/net/br-iot
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 br-kvm -> ../../devices/virtual/net/br-kvm
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 br-lan -> ../../devices/virtual/net/br-lan
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 eth0 -> ../../devices/pci0000:00/0000:00:03.0/virtio0/net/eth0
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 eth0.1 -> ../../devices/virtual/net/eth0.1
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 eth0.2 -> ../../devices/virtual/net/eth0.2
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 eth0.3 -> ../../devices/virtual/net/eth0.3
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 eth0.5 -> ../../devices/virtual/net/eth0.5
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 eth1 -> ../../devices/pci0000:00/0000:00:08.0/virtio1/net/eth1
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 eth2 -> ../../devices/pci0000:00/0000:00:09.0/virtio2/net/eth2
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 ifb0 -> ../../devices/virtual/net/ifb0
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 ifb1 -> ../../devices/virtual/net/ifb1
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 lo -> ../../devices/virtual/net/lo
lrwxrwxrwx    1 root     root             0 Sep 13 21:35 wg0 -> ../../devices/virtual/net/wg0
lrwxrwxrwx    1 root     root             0 Sep 13 20:51 wlan0 -> ../../devices/pci0000:00/0000:00:19.0/0000:02:02.0/net/wlan0

kernel log related to ath10k:

[    5.264830] ath10k_pci 0000:02:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[    5.413404] ath10k_pci 0000:02:00.0: failed to poke copy engine: -16
[    5.533475] ath10k_pci 0000:02:00.0: failed to poke copy engine: -16
[    5.683525] ath10k_pci 0000:02:00.0: failed to poke copy engine: -16
[    5.903320] ath10k_pci 0000:02:00.0: failed to poke copy engine: -16
[    6.053552] ath10k_pci 0000:02:00.0: failed to poke copy engine: -16
[    6.193233] ath10k_pci 0000:02:00.0: failed to poke copy engine: -16
[    6.292865] ath10k_pci 0000:02:00.0: Failed to get pcie state addr: -16
[    6.293149] ath10k_pci 0000:02:00.0: failed to setup init config: -16
[    6.293715] ath10k_pci 0000:02:00.0: could not start pci hif (-16)
[    6.293979] ath10k_pci 0000:02:00.0: could not probe fw (-16)
[    6.394797] ath: EEPROM regdomain: 0x0
[    6.394798] ath: EEPROM indicates default country code should be used
[    6.394798] ath: doing EEPROM country->regdmn map search
[    6.394799] ath: country maps to regdmn code: 0x3a
[    6.394800] ath: Country alpha2 being used: US
[    6.394800] ath: Regpair used: 0x3a
[    6.414647] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'
[    6.415039] ieee80211 phy1: Atheros AR9287 Rev:2 mem=0xffffc90000210000, irq=19

installed packages (ath9k and ath10k):

Unfortunately I don't find anything related on the web - does someone has a clue what might be the problem and maybe also how to solve it? :slightly_smiling_face:

Best,
ssdnvv

To be sure the issue is not related to virtualization, you can test PCI passthrough with libvirt+KVM/QEMU.

3 Likes

https://www.google.com.au/search?q="failed+to+poke+copy+engine"&oq="failed+to+poke+copy+engine"

opkg list-installed | grep -E '(qca|ath)'
ubus call system board
2 Likes

Yes, you are right. I only searched the web for "could not probe fw (-16)", and was not creative enough to do the other researches. Anyhow - a -16 as error code didn't show up in related threads, only in the ath10k sourcecode. So this combination is somewhat strange to me. I've really read a lot of those threads by now, but found nothing related to my problems and no real suggested solution (other than replace the firmware, wich I already did by myself).
I also tried another card, a WLE1216V5-23, which ended in a firmware crash (error code -110) - no matter what firmware I tried. A WLE1216V5-20 has not been recognized by the host OS at all (maybe a problem introduced by the mPCIe to PCIe-adapter).

So I reinstalled the WLE900VX (QCA9880) and got the very same error messages as in the op.
Here's the output of the queries you suggested:

opkg list-installed | grep -E '(qca|ath)'
ath10k-firmware-qca988x - 2018-05-12-952afa49-1
ath10k-firmware-qca9984 - 2018-05-12-952afa49-1
kmod-ath - 4.14.131+2017-11-01-10
kmod-ath10k - 4.14.131+2017-11-01-10
kmod-ath9k - 4.14.131+2017-11-01-10
kmod-ath9k-common - 4.14.131+2017-11-01-10
ubus call system board
{
        "kernel": "4.14.131",
        "hostname": "Server_VM1_Router",
        "system": "AMD Ryzen 5 3600 6-Core Processor",
        "model": "innotek GmbH VirtualBox",
        "board_name": "innotek-gmbh-virtualbox",
        "release": {
                "distribution": "OpenWrt",
                "version": "18.06.4",
                "revision": "r7808-ef686b7292",
                "target": "x86\/64",
                "description": "OpenWrt 18.06.4 r7808-ef686b7292"
        }
}

Thanks, I will try that and report the results.

In the meantime it seems I've got no luck with the AR9287, too. When trying to setup an ESSID, I get the following error several times for each restart of the wifi interface:

ath: phy1: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42100020 DMADBG_7=0x00008040

This seems to be a well known error, which should have been reverted with an older kernel patch and never happens when using the card directly in the host OS or in my Turris Omnia (running a forked OpenWRT version). It is also stated that this is only a debug message - unfortunately I cannot connect to that ESSID with whatever client I try (iPhone 7, SGS7, Dell XPS 13@Intel 9260).
When doing a wifi down && wifi up I get the following syslog:

Wed Sep 25 22:32:17 2019 daemon.notice hostapd: lan0: interface state ENABLED->DISABLED
Wed Sep 25 22:32:17 2019 daemon.notice hostapd: lan0: AP-DISABLED
Wed Sep 25 22:32:17 2019 daemon.notice hostapd: lan0: CTRL-EVENT-TERMINATING
Wed Sep 25 22:32:17 2019 daemon.notice hostapd: nl80211: deinit ifname=lan0 disabled_11b_rates=0
Wed Sep 25 22:32:17 2019 kern.info kernel: [ 2989.093573] device lan0 left promiscuous mode
Wed Sep 25 22:32:17 2019 kern.info kernel: [ 2989.093606] br-lan: port 3(lan0) entered disabled state
Wed Sep 25 22:32:17 2019 kern.err kernel: [ 2989.132296] ath: phy1: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42100020 DMADBG_7=0x00006040
Wed Sep 25 22:32:17 2019 daemon.notice netifd: Network device 'lan0' link is down
Wed Sep 25 22:32:17 2019 kern.err kernel: [ 2989.167349] ath: phy1: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42100020 DMADBG_7=0x00008040
Wed Sep 25 22:32:19 2019 kern.info kernel: [ 2990.610561] IPv6: ADDRCONF(NETDEV_UP): tmp.radio0: link is not ready
Wed Sep 25 22:32:19 2019 kern.info kernel: [ 2990.774794] IPv6: ADDRCONF(NETDEV_UP): tmp.radio0: link is not ready
Wed Sep 25 22:32:19 2019 kern.info kernel: [ 2990.954975] IPv6: ADDRCONF(NETDEV_UP): tmp.radio0: link is not ready
Wed Sep 25 22:32:20 2019 user.notice mac80211: Failed command: iw phy phy1 set antenna all all
Wed Sep 25 22:32:20 2019 daemon.err hostapd: Configuration file: /var/run/hostapd-phy1.conf
Wed Sep 25 22:32:20 2019 kern.err kernel: [ 2991.961170] ath: phy1: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42100020 DMADBG_7=0x00008040
Wed Sep 25 22:32:20 2019 daemon.notice hostapd: lan0: interface state UNINITIALIZED->COUNTRY_UPDATE
Wed Sep 25 22:32:20 2019 daemon.err hostapd: Using interface lan0 with hwaddr 04:f0:21:23:22:07 and ssid "Server"
Wed Sep 25 22:32:20 2019 kern.info kernel: [ 2991.973697] br-lan: port 3(lan0) entered blocking state
Wed Sep 25 22:32:20 2019 kern.info kernel: [ 2991.973698] br-lan: port 3(lan0) entered disabled state
Wed Sep 25 22:32:20 2019 kern.info kernel: [ 2991.973739] device lan0 entered promiscuous mode
Wed Sep 25 22:32:20 2019 kern.err kernel: [ 2992.006869] ath: phy1: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42100020 DMADBG_7=0x00006040
Wed Sep 25 22:32:20 2019 kern.err kernel: [ 2992.040194] ath: phy1: DMA failed to stop in 10 ms AR_CR=0x00000024 AR_DIAG_SW=0x42100020 DMADBG_7=0x00008040
Wed Sep 25 22:32:20 2019 daemon.notice netifd: Network device 'lan0' link is up
Wed Sep 25 22:32:20 2019 daemon.notice hostapd: lan0: interface state COUNTRY_UPDATE->ENABLED
Wed Sep 25 22:32:20 2019 daemon.notice hostapd: lan0: AP-ENABLED
Wed Sep 25 22:32:20 2019 kern.info kernel: [ 2992.052442] br-lan: port 3(lan0) entered blocking state
Wed Sep 25 22:32:20 2019 kern.info kernel: [ 2992.052443] br-lan: port 3(lan0) entered forwarding state
2 Likes

Well... i suspect your in pioneer territory....

All i can suggest is trying with master / alternate host ( intel ) hardware / hypervisor ... to see what happens...

1 Like

Unfortunately it seems I have to abandon this project for a while.
Power consumption measurements I did yesterday evening showed ca. 7 watts per miniPCIe card in idle mode. If I'd ever manage to get the cards working this might even increase. As the WRT32X I hoped to get rid of when finally powering up my X86-64 openwrt vm consumes 10,8 watts average the usecase for running WiFi from within the vm has gone :neutral_face:
I'll follow up on this when my usecase changes in the future...

3 Likes