Issue with Sierra Wireless MC7710 LTE module

Hi Everyone,

i am successfully running the Sierra Wireless MC7710 LTE module with ModemManager and mmcli, but i have one problem. I want to have the LTE connection always-on, but after some hours, the connection is lost. I guess, it has something todo with power saving or idle features...

Latest QMI Firmware (Version 03.05.29.03) is flashed onto the module.
Here's the output of cat /sys/kernel/debug/usb/devices:

root@OpenWrt:~# cat /sys/kernel/debug/usb/devices

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  2, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e101000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=480  MxCh= 0
D:  Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs=  1
P:  Vendor=1199 ProdID=68a2 Rev= 0.06
S:  Manufacturer=Sierra Wireless, Incorporated
S:  Product=MC7710
S:  SerialNumber=358178040153472
C:* #Ifs= 4 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=01(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 2 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=82(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 3 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=ff Driver=(none)
E:  Ad=83(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
E:  Ad=84(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=03(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 8 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=ff Driver=qmi_wwan
E:  Ad=85(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
E:  Ad=86(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=04(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms

T:  Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e106000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

First step of troubeshooting is study logs, i think. But i don't know, which logfiles to look for. Can someone help?

If found this:

But i have two problems:

  1. /usr/sbin/NetworkManager doesn't exist
  2. The problems often happens not before 10-20 hours. I'd have to keep the ssh session alive, so that ModemManager --debug keeps running?

Hey @bmork

I found this Thread from you, and the symptomes are 100% identical to my problem. I read the whole thread and the outcome was, that the XHCI controller was the reason in your case? In my case, it's not XHCI. I use a Fritzbox 3370 with USB 2.0, so it should be EHCI.

Do you have any idea?

No idea, really.

But Aleksander has made major improvements to the ModemManager disconnect handling and other stuff r in OpenWrt recently. See for example:

So you might want to test a current master snapshot to see if any of this fixes the issues. Or wait until it ends up in a release if you feel uncomfortable running bleeding edge images...

Wrt the root cause of the disconnects, I suspect they are network initiated. Many operators to this to avoid a large number of stale sessions or something. If possible, you could test that theory temporarily switching to a SIM from another operator. Depending on what options you have in your area of course.

Can't really think of anything else causing such issues. The MC7710 is getting old. but it was very stable in my experience. The only time I can remember having problems with unwanted disconnects was with a long and thin USB extension cable. This resulted in brownouts as soon as the modem tried to register in the LTE network. Not likely the cause of your issues of the connection is stable for hours before disconnecting.

I don't believe power saving problems is likely either with such long stable periods. The drivers and firmware supports USB autosuspend just fine, even while being connected.(using the Remote Wakeup USB feature). And if active, that's usually set up with a timeout of a few seconds. Not hours.

FWIW, the MC7710 was one of the modems I used to test and verify autosuspend with the qmi_wwan and cdc_mbim drivers.

Hm, so i see three options to continue testing:

try another router with a different USB Host controller. If i understood you correctly, that was the cause in your case. Maybe the "DWC OTG Controller" in my Fritzbox 3370 or it's driver causes this problems.

try the latest version of ModemManager you linked. But this would be a workaround, actually. The real problem (modem disconnect) would still be existent. If i am informed right, ModemManager would only handle the problems in a different way.

try 'QMI Cellular' protocol instead of ModemManager, and see what happens.

Since you wrote the MC7710 is a well tested device, i think i'll try a different router first.
Also: i don't think the problem is related to the mobilenetwork or operator. I used the same SIM Card with other routers/modems before without any disconnect problems.

Hi
I agree with @bmork that it is quite common that the operator disconnect your session time to time.
If you use 'QMI Cellular' you can verify the status of your connection with:

uqmi -d /dev/cdc-wdm0 --get-current-settings
ubus call network.interface.'name of your interface' status

If your modem is not connected then you have been disconnected :slight_smile:, if it´s connected but the IP-address in modem is different from what you have on the interface, then you have been dis-connected and the modem has re-connect.
QMI does not handle this in a good way. I don´t know how the ModemManager handle this scenario.

i'm sorry, i did not explain the issue detailed. The connection is lost, yes, but the modem is lost as well:

root@OpenWrt:~# mmcli -L
No modems were found

It's like bmork explained it here:
https://lists.freedesktop.org/archives/modemmanager-devel/2020-April/007769.html

Hey @bmork everytime i search on the internet to troubleshoot things regarding this topic, i find your name on various forums or mailinglists. You seem to be a real expert on this! I appreciate your help :slight_smile:

Thanks to your explanation here and here, i now know how to switch between QMI and MBIM modes. But i have two questions left, if you allow:

AT Command Reference guide states:

The modem has 2 MB of non-volatile memory that is used to store:
• Factory calibration data
• Settings made in a host application such as Watcher

I bought the MC7710 used from ebay. Is it possible, that there are old config parameters left on the device? If yes, is there a way to perform a factory reset? Maybe using a special AT command?

I want to use the MBIM mode. I understood, which USB comp value i have to pick, but i don't know which Product-ID i have to choose. I have the latest firmware installed (03.05.29.03). These are the available IDs:

AT!UDPID=?
68A3 (Direct IP mode?)
68AA
68A2 (QMI, i think?)
68B1
68A9

OK

BR,
Simon

There are commands to backup and restore NVRAM mentioned in section 6 "Memory Management Commands" of the Extended AT Command Reference (doc id 2130616). I have never tried them and won't repeat them here due to the risk involved. Better read that document including the big fat warning in there...

Personally I would not try to mess with the NVRAM unless I had specific problems. And DO NOT try to erase/reset all of it. The NVRAM includes factory calibration data and other data specific to your module. This cannot be restored unless you have a backup, and the modem will probably not work at all without it.

MBIM shares the 68A2 device ID with QMI. MBIM was added in one of the last firmware releases for this modem and is therefore often not mentioned in the docs.

You're correct that 68A3 is DirecIP. I don't know anything about the other device IDs. But I see that 68AA is included in the sierra_net driver so that's another DirectIP mode. And 68A9 is included in qmi_wwan do that's another QMI mode.

Just use 68A2 unless you have reasons to do something else.

Switching to MBIM worked well, i think. After entering AT!UDUSBCOMP=6 i did not get a "OK" response... I waited for some time, but the Modem did not respond to any other AT Commands. I powercyled the router. And everything seems to work well, but do you know, why there are 3 MBIM devices now? Is that normal?

root@OpenWrt:~# cat /sys/kernel/debug/usb/devices

T:  Bus=01 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  1, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e101000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

T:  Bus=01 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#=  2 Spd=480  MxCh= 0
D:  Ver= 2.00 Cls=ef(misc ) Sub=02 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1199 ProdID=68a2 Rev= 0.06
S:  Manufacturer=Sierra Wireless, Incorporated
S:  Product=MC7710
S:  SerialNumber=358178040153472
C:* #Ifs= 5 Cfg#= 1 Atr=e0 MxPwr=  0mA
A:  FirstIf#=12 IfCount= 2 Cls=02(comm.) Sub=0e Prot=00
I:* If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=qcserial
E:  Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=01(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 2 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=qcserial
E:  Ad=82(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#= 3 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=ff Driver=qcserial
E:  Ad=83(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
E:  Ad=84(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=03(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms
I:* If#=12 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=0e Prot=00 Driver=cdc_mbim
E:  Ad=85(I) Atr=03(Int.) MxPS=  64 Ivl=2ms
I:  If#=13 Alt= 0 #EPs= 0 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
I:* If#=13 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=02 Driver=cdc_mbim
E:  Ad=86(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E:  Ad=04(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms

T:  Bus=02 Lev=00 Prnt=00 Port=00 Cnt=00 Dev#=  1 Spd=480  MxCh= 1
B:  Alloc=  0/800 us ( 0%), #Int=  0, #Iso=  0
D:  Ver= 2.00 Cls=09(hub  ) Sub=00 Prot=01 MxPS=64 #Cfgs=  1
P:  Vendor=1d6b ProdID=0002 Rev= 5.04
S:  Manufacturer=Linux 5.4.188 dwc2_hsotg
S:  Product=DWC OTG Controller
S:  SerialNumber=1e106000.usb
C:* #Ifs= 1 Cfg#= 1 Atr=e0 MxPwr=  0mA
I:* If#= 0 Alt= 0 #EPs= 1 Cls=09(hub  ) Sub=00 Prot=00 Driver=hub
E:  Ad=81(I) Atr=03(Int.) MxPS=   4 Ivl=256ms

I assume you mean those 3 "Driver=cdc_mbim" lines? If so, then yes, that's normal.

It's not 3 devices. It's one MBIM function using 2 USB interfaces, where one of those interfaces have 2 alt settings. The driver has attached to all of them, which is why Linux shows the 3 lines.

This is how MBIM works and similar to many other USB CDC variants (ECM, NCM, etc). There is a control interface with a single interrupt endpoint and a data interface with 2 bulk endpoints (in/out). The data interface also has a default alt setting with no endpoints. This setting is inactive after the driver has loaded. But it is still under control by the driver.

This is just a way to organize the endpoints. If you look at a QMI function, you'll see that it use a similar set of endpoints (one interrupt and bulk in/out), but they are all associated with a single USB interface.

Old topic, but i finally found the reason. The power supply was insufficient! I bought a new USB adapter from Aliexpress, and since then, it works flawless. I wasted a lot of time for this. Hopefully it will help someone.

old (green): Mini PCI-E to USB Adapter (With SIM Card) Version 2.0
new (blue): Mini PCI-E to USB Adapter (With SIM Card) Version 5.0

I also noticed, that there are big differences regarding the routers. Some routers have big capacitors and coils which provide electric power for the USB ports, whilst other routers have a minimalistic component design. Perhaps this also has an effect