Moving to a Self-Compiled Image: Kernel Modules & Compilation

For Developers
1

Context

Hi all,
Firstly, the reason I want (need) to switch to a self-compiled OpenWrt is because I seemingly need to make some kernel option changes.

I am using a DL-WRX36 and would like to get BLE working. The device doesn't have BLE, so I am using an nRF52840 dongle with appropriate firmware. I ended up getting stuck being unable to enable the dongle through hciconfig:

root@OpenWrt:~# hciconfig
hci0:	Type: Primary  Bus: USB
	BD Address: 00:00:00:00:00:00  ACL MTU: 27:7  SCO MTU: 0:0
	DOWN 
	RX bytes:254 acl:0 sco:0 events:22 errors:0
	TX bytes:99 acl:0 sco:0 commands:22 errors:0

# Got this bit wrong, see first replies
root@OpenWrt:~# hciconfig hci 0 up
Warning: unknown command - "0"
Can't init device hci0: Address not available (99)

There was a surprising amount of similar issues online. But, after exhausting the fixes to most of them, I was left with only the most annoying fixes to try. This post, and a couple others, reference some options that should be enabled in the Linux kernel to get BLE mesh working. These options being:

CONFIG_CRYPTO_USER
CONFIG_CRYPTO_USER_API
CONFIG_CRYPTO_USER_API_AEAD
CONFIG_CRYPTO_USER_API_HASH

CONFIG_CRYPTO_AES
CONFIG_CRYPTO_CCM
CONFIG_CRYPTO_AEAD
CONFIG_CRYPTO_CMAC

If someone knows how to correct this issue without touching the kernel, that would be perfect.

Not so easy

Naively, I quickly went back to my machine, cloned the openwrt repository, made the changes to the kernel following these steps, compiled, and flashed to a DL-WRX36 router (I thankfully have 2 to play around with).

I quickly realised I couldn't install kernel modules using opkg like I used to. I am now having issues getting these 'baked' into the image I compile.

Setup

Being new to OpenWrt, making changes to the image and compiling it is not the smooth process I would like it to be. I am having a few issues finding what I need to do to get my desired image.

Here are the changes I have made to a freshly cloned repository:

  • Move to tag v23.5.02
  • make menuconfig to select target, subtarget, etc.
  • ./scripts/feeds update
  • Edit target/linux/ipq807/config-5.15 to include listed kernel options

Here is where I first tried to compile the image, only to later find out about kmod requirements. Coming back, I realise I need to now select all of my required kmods in make menuconfig - but they are not there!

Here is a list of kmod packages from my old system that I would like to try include with my new one (some are probably redundant from testing):

kmod-bluetooth         kmod-nf-ipt6
kmod-crypto-ecb        kmod-nf-nat6
kmod-crypto-ecdh       kmod-nft-compat
kmod-crypto-kpp        kmod-nft-compat
kmod-fs-vfat           kmod-nls-cp437
kmod-hid               kmod-nls-iso8859-1
kmod-input-core        kmod-nls-utf8
kmod-input-evdev       kmod-regmap-core
kmod-ip6tables-extra   kmod-scsi-core
kmod-ip6tables         kmod-tun
kmod-ipt-conntrack     kmod-usb-acm
kmod-ipt-core          kmod-usb-ehci 
kmod-ipt-extra         kmod-usb-ohci
kmod-ipt-ipset         kmod-usb-storage
kmod-ipt-nat           kmod-usb-uhci
kmod-ipt-nat6          kmod-usb2
kmod-nf-ipt

So, unless I am mistaken, my problem is with my feeds. If the kmod packages are not appearing, then I simply need to include the feed(s) that contain them. But where are they?
It didn't help looking at /etc/opkg/distfeeds.conf in the old system, since these are all compiled .ipks, when I need an appropriate git repository.

I played around a bit with branches and commits, but ultimately my feeds.conf does not differ from feeds.conf.default. Is it expected behavior for them to not to be included in the default feeds?

# feeds.conf.default
# I am guessing the commit hashes are from when I checked out this tag?
src-git packages https://git.openwrt.org/feed/packages.git^8e3a1824645f5e73ec44c897ac0755c53fb4a1f8
src-git luci https://git.openwrt.org/project/luci.git^7739e9f5b03b830f51d53c384be4baef95054cb3
src-git routing https://git.openwrt.org/feed/routing.git^83ef3784a9092cfd0a900cc28e2ed4e13671d667
src-git telephony https://git.openwrt.org/feed/telephony.git^9746ae8f964e18f04b64fbe1956366954ff223f8

Oh, and I found some mentions of Image Builder, can this help me with my problem? This looks like a whole other system, and I would rather stick with the openwrt repository I am familiar with - so long as my problem can be solved without too much difficultly that is.

On a similar note, how do I bake LuCi and it's kmod dependencies into my image? Trying to install LuCi is where I found my issue with kmods.

Cheers
: )

2

Your hci up command does nothing due to extra space and the command becomes 0 up

3

Haha,
Sorry, I wrote and grabbed the log without even looking at it.
I saw the "Can't init device hci0: Address not available (99)" and assume it was right.

Looks like it defaulted to hci0 anyway since the same error is given with the correct command. Here is what it should be:

root@OpenWrt:~# hciconfig
hci0:	Type: Primary  Bus: USB
	BD Address: 00:00:00:00:00:00  ACL MTU: 27:7  SCO MTU: 0:0
	DOWN 
	RX bytes:254 acl:0 sco:0 events:22 errors:0
	TX bytes:99 acl:0 sco:0 commands:22 errors:0

root@OpenWrt:~# hciconfig hci0 up
Can't init device hci0: Address not available (99)

: p

4

Can you at least successfully scan the environment (hcitool scan)?

5

nope, afraid not.

root@OpenWrt:~# hcitool scan
Device is not available: No such device

The inactive dongle is all I have, nothing in the hardware of the device.

root@OpenWrt:~# hciconfig -a
hci0:	Type: Primary  Bus: USB
	BD Address: 00:00:00:00:00:00  ACL MTU: 27:7  SCO MTU: 0:0
	DOWN 
	RX bytes:254 acl:0 sco:0 events:22 errors:0
	TX bytes:99 acl:0 sco:0 commands:22 errors:0
	Features: 0x00 0x00 0x00 0x00 0x60 0x00 0x00 0x00
	Packet type: DM1 DH1 HV1 
	Link policy: 
	Link mode: PERIPHERAL ACCEPT
6

That looks as if something of the more basic module (firmware!) loading went pear shaped, check dmesg and logread for hints.

7

Hcitool is outdated since a few years. It is advised to not use it with newer kernels. Kernel 5.6+ dropped some support for it. You have to use bluez-utils respectivley "btmgmt/btattach". So sth. like:

btmgmt --index 0
add-device -t x 01:23:45:.... (I don't know the type of your device for "x". Just try 1,2,3)

For hci tool like:
bdaddr -i hci0 01:23:45:.... (hci0 is the index of the device here, could be different if you have more)

to set the device' mac address. I hope its printed on the device itself and not on the packaging like often for cheap dongles. I don't know if the kernel has the MAC anywhere available.

8

You don't need any feeds for those kernel modules (but its a good idea to install them anyway usually for other stuff like the web interface) they should be all selectable from 'make menuconfig'
the web interface is called luci and there's a luci section in there too along with the kernel modules section

the feeds usually install easily with 'scripts/feeds update -a ; scripts/feeds install -a', usually just need working internet connection assuming you have all the needed tools installed on your host system and thats easy enough after reading here https://openwrt.org/docs/guide-developer/toolchain/install-buildsystem

9

I see.
I have looked at dmesg before but the logs were unhelpful. In dmesg it simply logs that a usb device has been disconnected / connected.

I did not know about logread, and I have found some more helpful info here:

# Plug in usb BLE dongle
root@OpenWrt:~# logread
...
kern.info kernel: [  188.670054] usb 1-1.3: new full-speed USB device number 5 using xhci-hcd
daemon.err bluetoothd[2441]: src/adapter.c:get_static_addr() Unable to load key file from /var/lib/bluetooth/addresses: (No such file or directory)
daemon.err bluetoothd[2441]: src/adapter.c:get_static_addr() Failed to open crypto
daemon.err bluetoothd[2441]: No Bluetooth address for index 0

The first error looks traceable. /var/lib/ contains a sym-link called bluetooth to the location /etc/bluetooth/keys/ which is empty. After a quick Google, I have found this section of an OpenWrt wiki page that looks relevant. I will have a closer read after looking into the other replies.

The mention of "crypto" here makes me think it is mentioning the crypto functionality that my listed kernel options would enable. Hopefully I can still avoid this though.

10

Thanks for the info.

My build is running kernel version 5.15.134, so I will try switch to using these commands.

Firstly, doesn't look like my first attempt at running these commands were successful. But right now I am willing to bet it is an error on my part. Here is my attempt:

root@OpenWrt:~# btmgmt --index 0
[hci0]# add-device -t 1 01:23:45:67:89:00
Adding device with 01:23:45:67:89:00 (LE Public)
hci0 device_flags_changed: 01:23:45:67:89:00 (LE Public)
     supp: 0x00000001  curr: 0x00000000
[hci0]# exit
root@OpenWrt:~# bdaddr -i hci0 01:23:45:67:89:00
Invalid device: Network is down

I would like to understand a bit more about Bluetooth and these commands.

What does it mean to "add" a Bluetooth device here? I couldn't find a more detailed explanation than "add-device = Add Device" online.

What is the address "01:23:45:67:89:00"? I expanded the example you gave and assumed it is some new address value, so the value used doesn't matter.

I have come across bdaddr before while trying to get this working. My conclusion is that it wouldn't work here because the dongle has self-compiled firmware. The manufacturer value here is "Linux ..." something or other - which isn't supported.
I also concluded that it isn't necessary (could be completely wrong) since we don't need a public address (which I think this is, can be wrong again) and it should work completely fine with just a randomized private address or similar. I used this page to reach this conclusion.

11

Hiya
I have been spending a little time using the search function in make menuconfig and I can find all of the kernel modules I randomly select from the list I made before.

I am honestly confused what I was thinking last week. Maybe there was a specific package that I couldn't find and haven't tried right now? I am a little bamboozled.

That said, I still can't find LuCI anywhere. I search for "luci" using / but the only thing it returns is the feed called luci, no modules or anything. Is there a specific directory structure I should follow in the config? I have searched everywhere and cannot find anything. Perhaps my feeds need correcting, they are still the same as feeds.conf.default in the original post.

I am also hopeful that this can be resolved without a re-compile.

12

I don't really have any clue about. I've tried to get integrated bt running on my router. But I've found out that they've used a device faking its ID to a very common device from the early years bt came up. To make it work I needed to load the firmware file and a MAC file in a certain format. This would have worked with older kernels and tools. But for newer kernels the firmware file would have needed a rework. --> dead end. I don't have any clue about despite the fact that its not really a firmware file. It's more like a code file for the kernel.

But as I saw your thread with empty MAC I've remembered some things about and gave it just as a hint.

13

I have one router with integrated bluetooth (ASUS MAP-AC2200 'lyra'), but as the bluetooth feature serves no real purpose for me there, I never tested beyond hcitool scan (which works) - which appeased my mind well enough to defer anything beyond that until a use case arises (unlikely, the router has no USB, so nothing sensible to add which could make use of bluetooth connectivity).

My suggestion would be to test this bluetooth stick on x86_64 and a general purpose distribution (arch, debian, gentoo, Mageia/ Mandriva, OpenSuSE, Ubuntu, pick your poison), to check out how things work there. This allows you to compares installed packages, (to some extent) kernel versions, and configurations. Running a distribution in live mode, booted from a USB stick should be sufficient for this. Yes, this side-steps the issue a bit, but seeing how it works elsewhere (on a full-featured linux, where it should work) makes it easier to debug the issue for OpenWrt.

14

@pwned
That is fine, thanks for replying to this nonetheless.

I'll keep your issue in mind, although the solution sounds like a handful. If you remember any online resources you found to come to your solution, that would be great.

You have reminded me of something I have stupidly forgot, which is I don't have to use this dongle. I happened to have a spare nRF52840 dongle lying around, an was happy I could use it. I seem to have forgotten that buying a BT dongle online is still an option for me. I'll keep it as a back-up plan (although I fear the problem is more so with my router).

@slh
You have also reminded me of something. I did try using the dongle on my Ubuntu laptop at the beginning - and it worked! Probably should have remembered that.

By worked I mean it got past my current problem of getting my dongle from DOWN to RUNNING. I tried again now and I can still get it RUNNING.

I couldn't get hcitool scan to work:

user@ubuntu:~$ sudo hcitool scan
Scanning ...
Inquiry failed: Operation not supported

Quick googling, stumbled upon hcitool lescan before a solution. After trying it, hcitool lescan works straight away:

user@ubuntu:~$ sudo hcitool lescan
LE Scan ...
# Loooong list of scan results

So that is good news. The dongle should work, at least with BLE which I need it for, and it narrows the problem down to the router I am running it on.

Hopefully this means once the dongle can be enabled, it all works. I will dig deeper into the logs from logread, although I feel it will continue to point me down the path of recompiling w/ new kernel options.

1 Like
15

Hi, @haydn, what is the benefts using your openwrt-USB to USB-BLE-dougle?

Btw, we use openwrt-serial to uart-BLE-dougle many years.

16

Hiya,

My ultimate goal is Matter commissioning.
I have been trying to fuse Matter with OpenWrt, you can see my initial efforts in my first post here.

I do not think that commissioning with BLE communication between smart device and router is overly beneficial, or going to be a trend in the future. But getting this working has some personal advantages for my setup.

I am hoping that once my dongle is in an operational state, like able to perform hcitool lescan, the development tools for Matter that I am using can pick that up and run with it.

I am using USB because I am more familiar and comfortable with it, I haven't ever touched UART. I do belive the DL-WRX36 can support it, though. Any pointers for looking into UART BLE?

17

I'm sure your data path is no need very hi-speed likes USB connection, and uart 115200bps is enough.

We are using openwrt serial port (/dev/ttyS0) wire to a BLE device with uart, put the BLE dongle into the corner in the router box. The BLE dongle is scanning other BLE devices and get the data, then pass data to openwrt via uart port, openwrt could handle the data locally or just upload the data to a server by internet connection. Many open source code on github for this function.

1 Like
18

Success

Ok, I have managed to get my USB dongle active and can successfully run hcitool lescan which I believe means I have BLE operational ready for my next steps.

After some replies from here I was hopeful that the issue could be resolved without touching the Linux kernel. Unfortunately, my options ran dry and I was once again left with recompiling.

Compiling

As I said in a reply, all the kernel modules (at least the ones I needed) were present in the make menuconfig menu. I have no clue how I missed them when I made the original post.

After selecting the kernel modules I think I need, and changing the Linux kernel options in /target as described in the post, I had some new firmware to try.

Initially I was unsure if the kernel changes had applied. I didn't completely understand what the r12345-abcdef1234 characters meant in the line seen on ssh connection:

OpenWrt SNAPSHOT, r12345-abcdef1234

They remained the same, which I initially thought meant the changes had not been made. I then learned they reflect the status of git, and they hadn't changed because I hadn't committed locally.

Fixing the issue

Firstly, even if the r12345-abcdef1234 characters were not supposed to change, I found it difficult to know for sure if the changes were made. I was still unsuccessful in enabling the dongle. By looking at logread, I got a similar error message(s) when trying to initiate bluetoothd:

daemon.info bluetoothd[4149]: Bluetooth daemon 5.66
daemon.info bluetoothd[4149]: Starting SDP server
daemon.warn bluetoothd[4149]: profiles/audio/vcp.c:vcp_init() D-Bus experimental not enabled
daemon.err bluetoothd[4149]: src/plugin.c:plugin_init() Failed to init vcp plugin
daemon.warn bluetoothd[4149]: profiles/audio/mcp.c:mcp_init() D-Bus experimental not enabled
daemon.err bluetoothd[4149]: src/plugin.c:plugin_init() Failed to init mcp plugin
daemon.warn bluetoothd[4149]: profiles/audio/bap.c:bap_init() D-Bus experimental not enabled
daemon.err bluetoothd[4149]: src/plugin.c:plugin_init() Failed to init bap plugin
daemon.err bluetoothd[4149]: src/rfkill.c:rfkill_init() Failed to open RFKILL control device
daemon.info bluetoothd[4149]: Bluetooth management interface 1.21 initialized
daemon.err bluetoothd[4149]: src/adapter.c:get_static_addr() Unable to load key file from /var/lib/bluetooth/addresses: (No such file or directory)
daemon.err bluetoothd[4149]: src/adapter.c:get_static_addr() Failed to open crypto
daemon.err bluetoothd[4149]: No Bluetooth address for index 0

Clearly my changes (if applied) hadn't fixed the issue - and the errors still mentioned crypto. To confirm my changes had been compiled, I made the addition of kmod-ikconfig so that kernel options could be viewed with:

zcat /proc/config.gz

I could see the options I changed were there, so I was quite disappointed that BLE still wouldn't work.

I went back to the online post referencing these kernel options that supposedly should fix the problem. It linked to a page I had already skimmed through which stated these kernel dependencies. It was here I noticed an additional kernel option that had been omitted in the post. My guess would be this option is already enabled for that hardware (a Pi) and wouldn't be required to change manually.

Here is the completed list of Linux kernel dependencies:

CONFIG_CRYPTO_USER
CONFIG_CRYPTO_USER_API
CONFIG_CRYPTO_USER_API_AEAD
CONFIG_CRYPTO_USER_API_HASH
CONFIG_CRYPTO_USER_API_SKCIPHER    # this was missing!

CONFIG_CRYPTO_AES
CONFIG_CRYPTO_CCM
CONFIG_CRYPTO_AEAD
CONFIG_CRYPTO_CMAC

Compiled, installed BT libraries, and the dongle worked straight away.

Whether or not BLE is in a state to work with the software I intend to use will probably be outside the scope of OpenWrt.

Thanks everyone for the help.
: )

1 Like
19

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.