[Solved] Debugging ModemManager 1.16.6 segfault on OpenWrt 21.02

Hello,

I'm trying to gather information to aid in figuring out the cause of the segfault in ModemManager 1.16.6 on OpenWrt 21.02. Following link is the issue describing the problem in detail, in summary I have a T77W968 DW5821e modem and ModemManager is exiting with segfault when initializing.

I'm currently stuck with the question:

What's the value of CONFIG_MODEMMANAGER_WITH_MBIM, CONFIG_MODEMMANAGER_WITH_QMI and CONFIG_LIBQMI_WITH_MBIM_QMUX in your .config?

The fact is that I installed ModemManager from the OpenWrt package repo/feeds and so I'm wondering if there were any build logs for ModemManager on OpenWrt that could give some answers.

Feel free to ask or comment with regards to issue if there's something I missed.

Thanks.

Two of those are set in the default config at https://git.openwrt.org/?p=feed/packages.git;a=blob_plain;f=net/modemmanager/Config.in

Thanks and yes I saw the default config though I wasn't sure of the process of building packages that could lead to completely different understanding. Eventually I have gotten into building the packages for testing purposes, that way I can know exactly what the config is for building and since its using the same package feed published it should give me an accurate representation of the config for the published packages.

1 Like

Once you are building and deploying the binary yourself it is worth learning how to connect a remote debugging session. It will quickly tell you where the crash is and that can remind the developers of something they thought they had fixed, etc.

1 Like

I'm grateful for your presence and advise as I go through the early stages of doing this, thanks a lot!

With regards to installing compiled binary and also then debugging, I just want to ask if my thoughts/process is correct as I am reading the docs/topics and still worried that I may have missed something.

I'm using the Docker SDK container to perform the build for the packages (for modemmanager particularly), also I decided to run the SDK container on an instance of OpenWrt x86_64 21.02 now that it supports docker (which is awesome!) thinking that it would help in transferring the compiled packages between the SDK container and OpenWrt with just a volume binding.

So I got the bin folder ready and it has all the compiled *.ipk package files ready. The question I'm figuring out is how to configure opkg/feeds on the OpenWrt instance to use the bin folder as the primary source of installing packages (it begs to ask if I should I be doing that?). Do I just add the following at the top of the feeds.conf file? Will I then be able to run opkg install modemmanager and it will pickup the packages (if available) from the bin folder?

src-link mypackages /root/docker-data/sdk/bin

You mentioned debugging, I'm familiar with gdb, the idea behind -g flag to have built in debugging support and debugging in general, though I'm not sure (for now) how I would do it with the OpenWrt instance. Is the general idea close or far from it?

Thanks again!

On the first bit, if you follow the instructions here for using a local copy of code it will bring any tree you are working with into the OpenWRT build work area: https://openwrt.org/docs/guide-developer/packages#working_on_local_application_source.

And this should help with getting GDB running: https://openwrt.org/docs/guide-developer/gdb.

1 Like

That reply was a bit rushed, so a couple of other pointers...

You already have MM available as binary from feeds, and that simplifies a lot of things. Without the source tree override it installs the pre-built package, and setting the override is a super simple way to be able to work on your own code. Note that you have to commit the code into git locally for it to be pulled into the OpenWRT build, so it can get the sources and build the binary for the right platform.

Add the package to menuconfig and it'll be incorporated into the initial image that you'll deploy to the platform.

Once you build your first bootable target environment you can of course shortcut having to do full builds to deploy updates, or even repackaging the ipk. I just scp the binary from where it has appeared in my OpenWRT build environment onto a relevant place of the device. To avoid flash wear I put it in /tmp on routers, and adjust the init.d script to run that if it exists (and the /bin one if not).

Once you have all that working it's a pretty quick job to try new code.

Final tip would be that I have previously got the OpenWRT build chain working within an x86 OpenWRT instance. You can then build, copy, run, gdb, etc all within a single environment.

1 Like

Thanks a lot for the pointers and your time for sharing them. I'm happy to tell you that the root cause was found, though I did not get to the part of trying out gdb as the problem was a bit straightforward, I'm happy that the 4G modem is now working without any issues :smiley: .

More details can be found from the links below:

Thanks again!

One question, how do I setup the key-build for signing the package index? It seems when using the docker SDK container the file doesn't get created.

I haven't ever done anything like that, so can't help on it.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.