Troubleshooting QMI connection to a LTE modem, does not get IP address

Installing and Using OpenWrt Network and Wireless Configuration
21

For those interested, this is the patch I used, it's exactly the same thing iuncuim suggested above but now it is working with latest uqmi sources used in OpenWrt. From what I understand, it is modifying the return message from the modem to always be "raw-ip", which is the only thing these modems support.
It hacks around the issue that uqmi can't read return messages from this modem, and since it does not see the modem is "raw-ip" mode it does not trigger the functions that control modems with only raw-ip mode in the qmi.sh shell script so the interface does not start up.

Drop this text in a text file called "code_fix.patch" in openwrt/package/network/utils/uqmi/patches/ folder, (create the "patches" folder if it does not exist), then make clean ; make to recompile the firmware

--- a/commands-wda.c
+++ b/commands-wda.c
@@ -61,7 +61,7 @@ cmd_wda_get_data_format_cb(struct qmi_de

 	qmi_parse_wda_get_data_format_response(msg, &res);
 	for (i = 0; i < ARRAY_SIZE(link_modes); i++) {
-		if (link_modes[i].val != res.data.link_layer_protocol)
+		if (link_modes[i].val != (res.data.link_layer_protocol&0x03))
 			continue;

 		name = link_modes[i].name;
1 Like
22

I use this patch with a master branch and CC-branch, it works in both. It correctly defines both modes: raw-ip and 802.3. I checked it too. If I wanted to hardcode only raw-ip, I would edit qmi.sh.

What your response to --get-pin-status command?

23

Sorry, I have no clue why this happens. I suggest you submit the patch to @malaakso and @nbd explaining the problem. They are the ones knowing the uqmi code. I looked at it the last time this came up, but couldn't see anything obvious. IIRC, the QMI packet was correct.The corruption is somewhere in the uqmi parser.

Anyway, the workaround looks harmless except for papering over the real bug, so I guess it should be applied if we can't figure out the root cause

24 
root@OpenWrt:~# uqmi -d /dev/cdc-wdm0   --get-pin-status
"Not supported"

I have disabled the SIM's PIN check from a smartphone, so I think it's normal.

25


Because ec25 don't support pin status command, qmi.sh will be looped here. Remove or fix that, and your modem will work correct

26

My modem works correctly when I use your duct-tape-patch.

Why you say it will loop there? the condition is always 1, the code in the "while... done" will never be executed and next line of the script will be executed.

Test script:

#!/bin/sh
while echo '"Not supported"' | grep '"UIM uninitialized"' > /dev/null ; do
echo aaa
done
echo bbb

executing it:

alby@linux-7mjq:~> ./test 
bbb
27

Yes you are right.

28

@bobafetthotmail:

I am also using EC25 with trunk Openwrt. Tried QMI mode, and when there is good coverage, it works correctly, but in the moment the coverage drops even for a few minutes, the modem will either not recover, or go unstable after recovered. This is with the latest firmware from Quectel.

In modem mode (ppp) this error is not present, but the speed is limited to 15/15Mbits.

But, there more to this. I think this is not a modem error, but somehow the QMI logical connection got severed between the modem and Openwrt.

Someone also complained in some topics, that uqmi --synch is not working. Well, I might have the answer to that:

Use AT command AT+QCFG=? and take a look at the parameters. There is one called "qmisync" in the list, which on our modems are disabled by default. If someone want to try it, it can be enabled. with AT+QCFG="qmisync",1

The short summary is: the EC25 is not stable in QMI mode for sure, I dont think it is the modem itself as even when the connection is not working on openwrt, I can ping the modem's PDN IP from the core network. So likely the QMI implementation fails.

29

I don't see these issues as my modem is in a static position (with good coverage), but I see that when I disconnect the modem manually (by just unplugging it from the USB hub I use to power it) then when I connect it again the router does not establish a connection again until I issue a "stop connection" with the modem connected to the USB port, and then I issue a "connect" afterwards.

Issuing a "stop connection" while the modem is unplugged does not work.

I was suspecting the "autoconnect" feature of the modem.

What is uqmi --sync supposed to do?

P.S. where you got the latest firmware? I assume you are working for a company so you have a Quectel support contact?

EDIT: I'm probably going to try setting an Internet connection with a miniPC and qmicli commandline tool from a normal linux distro, to see if this is indeed an issue of uqmi or of the modem.

30

That was my first thought as well. This feature usually does a great job keeping the modem connected, but not so great keeping the host informed on the current state. Which easily can lead to situationes where the host doesn't notice that it needs to reconfigure the IP settings because the modem decided to reconnect.

It sends a CTL_SYNC QMI request, which will (or is supposed to) clear absolutely all modem state. Both modem and host will obviously keep a lot of state related to active connections etc. This request is useful if the states become unsyncronized for some reason. E.g. as an effect of the autoconnect feature, as mentioned above. But clearing all state is also obviously rather destructive. You will have to set up all sessions from scratch. So it is generally something to avoid.

31

@bmork

That was my first thought as well. This feature usually does a great job keeping the modem connected, but not so great keeping the host informed on the current state. Which easily can lead to situationes where the host doesn’t notice that it needs to reconfigure the IP settings because the modem decided to reconnect.

Sounds exactly like my issue, that after a coverage drop the modem comes and goes online and offline even if the coverage returned normal. The question is if this is the modem's issue, or the QMI implementation in OWRT? Cause in AT mode, the same modem is recovering completely fine. If you want me to do any tests, please let me know.

32

Hm, but the current qmi control script https://github.com/openwrt/openwrt/blob/master/package/network/utils/uqmi/files/lib/netifd/proto/qmi.sh is using it (line 119) as part of the modem connect sequence, first deletes configuration and then configures other things.

I assume that this was supposed to deal with the use case of my modem, where I unplug it from the USB to power it off. This means the modem will be in random states when I connect it again. While if I run the disconnect "sequence" when the modem is connected then it clears its state (it is using direct commands, that I assume are working) and I can connect again.

Will try to enable the qmisync option from AT commands, but I'm somewhat afraid of learning the hard way why it was disabled by default.

33

Didn't do much tests on this, but I got the modem to drop connection or lock up consistently by updating Windows (to latest 1803 release) on 2 PCs while watching youtube.

Reset the thing and it locked up a hour later, reset again and it locked up again in a little while.

By looking at dmesg, it seems that the modem itself is resetting, as I'm getting spammed with these that are hardware disconnect and reconnect events. I have the thing connected to a 3A power supply (over a powered hub that just routes that to its downstream ports with no power limits enforced, so it can actually take up to 3A) so I don't think it is just hitting USB power limitations by going in some "high power mode".

[  399.782385] usb 1-1.4.4: USB disconnect, device number 4
[  399.788211] option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
[  399.796391] option 1-1.4.4:1.0: device disconnected
[  399.801782] option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
[  399.809947] option 1-1.4.4:1.1: device disconnected
[  399.815337] option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
[  399.823475] option 1-1.4.4:1.2: device disconnected
[  399.828895] option1 ttyUSB3: GSM modem (1-port) converter now disconnected from ttyUSB3
[  399.837061] option 1-1.4.4:1.3: device disconnected
[  399.889292] qmi_wwan 1-1.4.4:1.4 wwan0: unregister 'qmi_wwan' usb-f1050000.ehci-1.4.4, WWAN/QMI device
[ 1338.322020] usb 1-1.4.4: new high-speed USB device number 5 using orion-ehci
[ 1338.485019] option 1-1.4.4:1.0: GSM modem (1-port) converter detected
[ 1338.491741] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB0
[ 1338.499587] option 1-1.4.4:1.1: GSM modem (1-port) converter detected
[ 1338.506305] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB1
[ 1338.514132] option 1-1.4.4:1.2: GSM modem (1-port) converter detected
[ 1338.520824] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB2
[ 1338.528651] option 1-1.4.4:1.3: GSM modem (1-port) converter detected
[ 1338.535358] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB3
[ 1338.594576] qmi_wwan 1-1.4.4:1.4: cdc-wdm0: USB WDM device
[ 1338.601722] qmi_wwan 1-1.4.4:1.4 wwan0: register 'qmi_wwan' at usb-f1050000.ehci-1.4.4, WWAN/QMI device, c2:6c:2a:f3:e2:fe
[ 1588.902510] usb 1-1.4.4: USB disconnect, device number 5
[ 1588.908310] option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
[ 1588.916457] option 1-1.4.4:1.0: device disconnected
[ 1588.921855] option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
[ 1588.930018] option 1-1.4.4:1.1: device disconnected
[ 1588.935361] option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
[ 1588.943511] option 1-1.4.4:1.2: device disconnected
[ 1588.948873] option1 ttyUSB3: GSM modem (1-port) converter now disconnected from ttyUSB3
[ 1588.956994] option 1-1.4.4:1.3: device disconnected
[ 1589.013264] qmi_wwan 1-1.4.4:1.4 wwan0: unregister 'qmi_wwan' usb-f1050000.ehci-1.4.4, WWAN/QMI device
[ 1601.997031] usb 1-1.4.4: new high-speed USB device number 6 using orion-ehci
[ 1602.159084] option 1-1.4.4:1.0: GSM modem (1-port) converter detected
[ 1602.165814] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB0
[ 1602.173662] option 1-1.4.4:1.1: GSM modem (1-port) converter detected
[ 1602.180380] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB1
[ 1602.188397] option 1-1.4.4:1.2: GSM modem (1-port) converter detected
[ 1602.195100] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB2
[ 1602.202929] option 1-1.4.4:1.3: GSM modem (1-port) converter detected
[ 1602.209633] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB3
[ 1602.347389] qmi_wwan 1-1.4.4:1.4: cdc-wdm0: USB WDM device
[ 1602.354344] qmi_wwan 1-1.4.4:1.4 wwan0: register 'qmi_wwan' at usb-f1050000.ehci-1.4.4, WWAN/QMI device, c2:6c:2a:f3:e2:fe
[ 2648.742384] usb 1-1.4.4: USB disconnect, device number 6
[ 2648.748215] option1 ttyUSB0: GSM modem (1-port) converter now disconnected from ttyUSB0
[ 2648.756434] option 1-1.4.4:1.0: device disconnected
[ 2648.761825] option1 ttyUSB1: GSM modem (1-port) converter now disconnected from ttyUSB1
[ 2648.769998] option 1-1.4.4:1.1: device disconnected
[ 2648.775389] option1 ttyUSB2: GSM modem (1-port) converter now disconnected from ttyUSB2
[ 2648.783550] option 1-1.4.4:1.2: device disconnected
[ 2648.788907] option1 ttyUSB3: GSM modem (1-port) converter now disconnected from ttyUSB3
[ 2648.797057] option 1-1.4.4:1.3: device disconnected
[ 2648.802249] qmi_wwan 1-1.4.4:1.4 wwan0: unregister 'qmi_wwan' usb-f1050000.ehci-1.4.4, WWAN/QMI device
[61531.849769] usb 1-1.4.4: new high-speed USB device number 7 using orion-ehci
[61532.011761] option 1-1.4.4:1.0: GSM modem (1-port) converter detected
[61532.018493] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB0
[61532.026333] option 1-1.4.4:1.1: GSM modem (1-port) converter detected
[61532.033059] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB1
[61532.040890] option 1-1.4.4:1.2: GSM modem (1-port) converter detected
[61532.047582] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB2
[61532.055409] option 1-1.4.4:1.3: GSM modem (1-port) converter detected
[61532.062117] usb 1-1.4.4: GSM modem (1-port) converter now attached to ttyUSB3
[61532.122297] qmi_wwan 1-1.4.4:1.4: cdc-wdm0: USB WDM device
[61532.129359] qmi_wwan 1-1.4.4:1.4 wwan0: register 'qmi_wwan' at usb-f1050000.ehci-1.4.4, WWAN/QMI device, c2:6c:2a:f3:e2:fe

All this is not really detected and handled by uqmi daemon in the slightest (just as reported above, uqmi protocol does not seem to have any idea that the modem was disconnected until I click on "disconnect" in LuCi). in logread I'm not seeing much about this activity, only the times I've disabled or enabled the modem manually.

Moved the sim card in my smartphone and it's running perfectly fine, USB connection (virtual ethernet, RNDIS driver) is detected fine and it notices if it goes down or up.

I'm going to go with B plan above, (buying a separated 4G modem with ethernet port) as I really don't think I can do much without investing a whole lot of time (that I don't have) into this.

I'll still keep the Quectel modem for a while to make more tests with a linux distro and qmicli or Network Manager's modem subsystem, and see if I can get it to work reliably at all.

34

I've found that most of the scripts I looked at (using qmicli) poll multiple QMI-based ways of trying to determine if the modem is actually connected or not. You would think it would be something clear that would be reported, but apparently not. I've seen where there is no antenna on the unit and it is reporting as connected, registered, and packet service available (and I know the signal strength is far too low here for that to be true).

http://www.embeddedpi.com/documentation/3g-4g-modems/raspberry-pi-sierra-wireless-mc7304-modem-qmi-interface-setup has been pretty helpful

35

Hm, apparently one of the lead libqmi (qmicli) developers set up a custom source feed with ModemManager and qmicli https://sigquit.wordpress.com/tag/openwrt/
with procd (OpenWrt's init system) support, UCI support, hotplug scripts support, all bells and whistles.

Looks up-to-date, last change 04-04-2018 https://bitbucket.org/aleksander0m/modemmanager-openwrt

For those that don't know what I'm talking about (ModemManager is what most PC linux distros use by default so Linux users might know about it already), here is the description from the article

OpenWRT already has some support for a lot of the devices that ModemManager is able to manage (e.g. through the uqmi, umbim or wwan packages), but unlike the current solutions, ModemManager doesn’t require protocol-specific configurations or setups for the different devices; i.e. the configuration for a modem running in MBIM mode may be the same one as the configuration for a modem requiring AT commands and a PPP session.

Currently the OpenWRT package prepared is based on ModemManager git master, and therefore it supports: QMI modems (including the new MC74XX series which are raw-ip only and don’t support DMS UIM operations), MBIM modems, devices requiring QMI over MBIM operations (e.g. FCC auth), and of course generic AT+PPP based modems, Cinterion, Huawei (both AT+PPP and AT+NDISDUP), Icera, Haier, Linktop, Longcheer, Ericsson MBM, Motorola, Nokia, Novatel, Option (AT+PPP and HSO), Pantech, Samsung, Sierra Wireless (AT+PPP and DirectIP), Simtech, Telit, u-blox, Wavecom, ZTE… and even Iridium and Thuraya satellite modems. All with the same configuration.

Has anyone tried it? I'm probably going try this out soon.
(yes I know, this is the second time I'm putting on hold the B Plan, but I think it's worth it)

36

Tried with ModemManager on both OpenWrt and my PC running OpenSUSE, it only goes in ppp mode (it creates a ppp0 interface and uses that).

Seems like the ModemManager on OpenWrt has issues dealing with sudden removal of the device, but that's unrelated.

I asked a friend for the newer firmware, I got it and tried again with the latest firmware. No change, qmi still needs the "duct-tape patch", and it still requires babysitting if the device is disconnected or powered off.

Enabling the sync does not change anything either.

Meanwhile, the huawei 310s-22 LTE modem/router I bought to replace this thing works fine, and since it is self-contained and over ethernet I'm not using qmi or modem manager to get to the internet.

An interesting thing is that it can actually switch between cone NAT and symmetric NAT (an option I've seen very rarely), which would have been nice if the ISP itself wasn't running on IPv4 and doing a symmetric NAT on their side I guess.

37

I can confirm this.

I talked to Quectel, and they would like to have a QMI trace about what happens. They have a trace tool called Qlog, but that does not running on OWRT.

Question is: is there a way we can meaningfully do QMI trace under OWRT? I am willing to do this if someone gives a bit of instruction.

38

Does it at least run on x86 Linux? Is there a download link for it? All Qlog downloads I found (on secondary sites) are for Windows.

There is no point in troubleshooting on Windows as the driver isn't the same.

39

Yes, it runs on x86 linux. If you are willing to do some trial runs, I can send a copy to you, so we can send the traces to Quectel. Please let me know.

40

I'm willing to do the tests, but I'll be able to do that after 14th of july.