Issue with Sierra Wireless MC7710 LTE module

Lebedev · May 16, 2022, 2:06pm

Hi Everyone,

i am successfully running the Sierra Wireless MC7710 LTE module with ModemManager and mmcli, but i have one problem. I want to have the LTE connection always-on, but after some hours, the connection is lost. I guess, it has something todo with power saving or idle features...

Latest QMI Firmware (Version 03.05.29.03) is flashed onto the module.
Here's the output of cat /sys/kernel/debug/usb/devices:

root@OpenWrt:~# cat /sys/kernel/debug/usb/devices

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  2, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e101000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=480  MxCh= 0
D:  Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
P:  Vendor=1199 ProdID=68a2 Rev= 0.06
S:  Manufacturer=Sierra Wireless, Incorporated
S:  Product=MC7710
S:  SerialNumber=358178040153472
C:* #Ifs= 4 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=01(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 2 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=82(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 3 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=83(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
E:  Ad=84(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=03(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 8 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=ff Driver=qmi_wwan
E:  Ad=85(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
E:  Ad=86(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=04(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms

T:  Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e106000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

First step of troubeshooting is study logs, i think. But i don't know, which logfiles to look for. Can someone help?

Lebedev · May 16, 2022, 5:55pm

If found this:

But i have two problems:

/usr/sbin/NetworkManager doesn't exist
The problems often happens not before 10-20 hours. I'd have to keep the ssh session alive, so that ModemManager --debug keeps running?

Lebedev · May 16, 2022, 9:22pm

Hey @bmork

I found this Thread from you, and the symptomes are 100% identical to my problem. I read the whole thread and the outcome was, that the XHCI controller was the reason in your case? In my case, it's not XHCI. I use a Fritzbox 3370 with USB 2.0, so it should be EHCI.

Do you have any idea?

bmork · May 17, 2022, 7:00am

No idea, really.

But Aleksander has made major improvements to the ModemManager disconnect handling and other stuff r in OpenWrt recently. See for example:

github.com/openwrt/packages

modemmanager: report network initiated disconnections to netifd

committed 04:39AM - 15 May 22 UTC

aleksander0m

+38 -0

The new connection dispatcher scripts support integrated in ModemManager 1.18.8 …allows us to provide a openwrt-specific dispatcher script used to report netifd that the underlying network connection is down. See also https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/merge_requests/775 Fixes https://github.com/openwrt/openwrt/issues/8368 Fixes https://github.com/openwrt/packages/issues/14096 Signed-off-by: Aleksander Morgado <aleksander@aleksander.es>

So you might want to test a current master snapshot to see if any of this fixes the issues. Or wait until it ends up in a release if you feel uncomfortable running bleeding edge images...

Wrt the root cause of the disconnects, I suspect they are network initiated. Many operators to this to avoid a large number of stale sessions or something. If possible, you could test that theory temporarily switching to a SIM from another operator. Depending on what options you have in your area of course.

Can't really think of anything else causing such issues. The MC7710 is getting old. but it was very stable in my experience. The only time I can remember having problems with unwanted disconnects was with a long and thin USB extension cable. This resulted in brownouts as soon as the modem tried to register in the LTE network. Not likely the cause of your issues of the connection is stable for hours before disconnecting.

I don't believe power saving problems is likely either with such long stable periods. The drivers and firmware supports USB autosuspend just fine, even while being connected.(using the Remote Wakeup USB feature). And if active, that's usually set up with a timeout of a few seconds. Not hours.

FWIW, the MC7710 was one of the modems I used to test and verify autosuspend with the qmi_wwan and cdc_mbim drivers.

Lebedev · May 17, 2022, 7:52am

Hm, so i see three options to continue testing:

try another router with a different USB Host controller. If i understood you correctly, that was the cause in your case. Maybe the "DWC OTG Controller" in my Fritzbox 3370 or it's driver causes this problems.

try the latest version of ModemManager you linked. But this would be a workaround, actually. The real problem (modem disconnect) would still be existent. If i am informed right, ModemManager would only handle the problems in a different way.

try 'QMI Cellular' protocol instead of ModemManager, and see what happens.

Since you wrote the MC7710 is a well tested device, i think i'll try a different router first.
Also: i don't think the problem is related to the mobilenetwork or operator. I used the same SIM Card with other routers/modems before without any disconnect problems.

mrhaav · May 17, 2022, 1:40pm

Hi
I agree with @bmork that it is quite common that the operator disconnect your session time to time.
If you use 'QMI Cellular' you can verify the status of your connection with:

uqmi -d /dev/cdc-wdm0 --get-current-settings
ubus call network.interface.'name of your interface' status

If your modem is not connected then you have been disconnected , if it´s connected but the IP-address in modem is different from what you have on the interface, then you have been dis-connected and the modem has re-connect.
QMI does not handle this in a good way. I don´t know how the ModemManager handle this scenario.

Lebedev · May 17, 2022, 5:32pm

i'm sorry, i did not explain the issue detailed. The connection is lost, yes, but the modem is lost as well:

root@OpenWrt:~# mmcli -L
No modems were found

It's like bmork explained it here:
https://lists.freedesktop.org/archives/modemmanager-devel/2020-April/007769.html

Lebedev · May 21, 2022, 10:25am

Hey @bmork everytime i search on the internet to troubleshoot things regarding this topic, i find your name on various forums or mailinglists. You seem to be a real expert on this! I appreciate your help

Thanks to your explanation here and here, i now know how to switch between QMI and MBIM modes. But i have two questions left, if you allow:

AT Command Reference guide states:

The modem has 2 MB of non-volatile memory that is used to store:
• Factory calibration data
• Settings made in a host application such as Watcher

I bought the MC7710 used from ebay. Is it possible, that there are old config parameters left on the device? If yes, is there a way to perform a factory reset? Maybe using a special AT command?

I want to use the MBIM mode. I understood, which USB comp value i have to pick, but i don't know which Product-ID i have to choose. I have the latest firmware installed (03.05.29.03). These are the available IDs:

AT!UDPID=?
68A3 (Direct IP mode?)
68AA
68A2 (QMI, i think?)
68B1
68A9

OK

BR,
Simon

bmork · May 21, 2022, 12:14pm

There are commands to backup and restore NVRAM mentioned in section 6 "Memory Management Commands" of the Extended AT Command Reference (doc id 2130616). I have never tried them and won't repeat them here due to the risk involved. Better read that document including the big fat warning in there...

Personally I would not try to mess with the NVRAM unless I had specific problems. And DO NOT try to erase/reset all of it. The NVRAM includes factory calibration data and other data specific to your module. This cannot be restored unless you have a backup, and the modem will probably not work at all without it.

MBIM shares the 68A2 device ID with QMI. MBIM was added in one of the last firmware releases for this modem and is therefore often not mentioned in the docs.

You're correct that 68A3 is DirecIP. I don't know anything about the other device IDs. But I see that 68AA is included in the sierra_net driver so that's another DirectIP mode. And 68A9 is included in qmi_wwan do that's another QMI mode.

Just use 68A2 unless you have reasons to do something else.

Lebedev · May 21, 2022, 2:48pm

Switching to MBIM worked well, i think. After entering AT!UDUSBCOMP=6 i did not get a "OK" response... I waited for some time, but the Modem did not respond to any other AT Commands. I powercyled the router. And everything seems to work well, but do you know, why there are 3 MBIM devices now? Is that normal?

root@OpenWrt:~# cat /sys/kernel/debug/usb/devices

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  1, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e101000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=480  MxCh= 0
D:  Ver= 2.00 Cls=ef(misc ) Sub=02 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1199 ProdID=68a2 Rev= 0.06
S:  Manufacturer=Sierra Wireless, Incorporated
S:  Product=MC7710
S:  SerialNumber=358178040153472
C:* #Ifs= 5 Cfg#= 1 Atr=e0 MxPwr=  0mA
A:  FirstIf#=12 IfCount= 2 Cls=02(comm.) Sub=0e Prot=00
I:* If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=qcserial
E:  Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=01(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 2 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=qcserial
E:  Ad=82(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 3 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=ff Driver=qcserial
E:  Ad=83(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
E:  Ad=84(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=03(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#=12 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=0e Prot=00 Driver=cdc_mbim
E:  Ad=85(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
I:  If#=13 Alt= 0 #EPs= 0 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
I:* If#=13 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
E:  Ad=86(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=04(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms

T:  Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e106000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

bmork · May 21, 2022, 5:49pm

Lebedev:

do you know, why there are 3 MBIM devices now? Is that normal?

I:* If#=12 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=0e Prot=00 Driver=cdc_mbim
E:  Ad=85(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
I:  If#=13 Alt= 0 #EPs= 0 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
I:* If#=13 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
E:  Ad=86(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=04(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms

I assume you mean those 3 "Driver=cdc_mbim" lines? If so, then yes, that's normal.

It's not 3 devices. It's one MBIM function using 2 USB interfaces, where one of those interfaces have 2 alt settings. The driver has attached to all of them, which is why Linux shows the 3 lines.

This is how MBIM works and similar to many other USB CDC variants (ECM, NCM, etc). There is a control interface with a single interrupt endpoint and a data interface with 2 bulk endpoints (in/out). The data interface also has a default alt setting with no endpoints. This setting is inactive after the driver has loaded. But it is still under control by the driver.

This is just a way to organize the endpoints. If you look at a QMI function, you'll see that it use a similar set of endpoints (one interrupt and bulk in/out), but they are all associated with a single USB interface.

Lebedev · June 25, 2022, 3:34pm

Old topic, but i finally found the reason. The power supply was insufficient! I bought a new USB adapter from Aliexpress, and since then, it works flawless. I wasted a lot of time for this. Hopefully it will help someone.

old (green): Mini PCI-E to USB Adapter (With SIM Card) Version 2.0
new (blue): Mini PCI-E to USB Adapter (With SIM Card) Version 5.0

I also noticed, that there are big differences regarding the routers. Some routers have big capacitors and coils which provide electric power for the USB ports, whilst other routers have a minimalistic component design. Perhaps this also has an effect

system · July 5, 2022, 3:34pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.