This was way too optimistic. Shows the holes in my memory... We got proper mainline support for MT7621 as late as v5.18. And my first attempt on booting that failed with
Starting kernel ...
[ 0.000000] Linux version 5.18.19 (bjorn@canardo) (mipsel-linux-gnu-gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #660 SMP Tue Jun 25 18:30:30 CEST 2024
[ 0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
[ 0.000000] printk: bootconsole [early0] enabled
[ 0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
[ 0.000000] MIPS: machine is ZyXEL NR7101
[ 0.000000] Kernel sections are not in the memory maps
[ 0.000000] Wasting 36 bytes for tracking 1 unused pages
[ 0.000000] Initrd not found or empty - disabling initrd
[ 0.000000] Kernel panic - not syncing: early_init_dt_alloc_memory_arch: Failed to allocate 32768 bytes align=0x40
[ 0.000000] Rebooting in 1 seconds..
[ 0.000000] Reboot failed -- System halted
didn't bother looking for the problem. I take it this kernel isn't old enough in any case.
I don't really feel like testing older OpenWrt versions with all sorts of out-of-tree drivers. Even if we do find that one of those work better than current version, that doesn't tell us anything new. Finding a bug by comparing different drivers to each other is hard/impossible. And we already know the OEM drivers work better so we don't really need more examples. We need something which would help pinpont hte issue in the current mainline drivers.
(no, forklifting some old out-of-tree driver into mainline is not going to fly either)
Does anyone know of a hardware platform which has had mainline support for a while where this problem can be reproduced? I assume it's not present on x86?
Continue to main subject , I bought new modem with PCIe mode for this modem 5G
I did some test (PCIe ) between MCUZONE and OpenWrt
MCUZONE PCIe
i'm getting 800mb/s to 900mb/s like USB Speeds (before)
Where The PCIe for OpenWrt
i'm getting 550 mb/s
So I started searching to know why ? I found CPU utilization ( when Its running for all CPUs 0,1,2 & 3) IRQ .......only CPU 0 is running and reaching up to 75% utilization where other CPUs 1,2 and 3 are almost sleeping (low utilization)
So I tried to let all CPUs run at same time on PCIe mode but no luck ... still system is assigning load to CPU 0 also when I activated IRQ package to disrepute load but no luck
Also with this
echo f > /sys/class/net/usb0/queues/rx-0/rps_cpus
only CPU 0 is running
I did one thing made a big different on speed and utilization
CPU 1 and 2 are giving a higher speeds
CPU 0 and 4 are giving a lower speeds
I appreciate answering on these questions
How I can let the OpenWrt use all CPUs ? only CPU 0 will work .... echo f > /proc/irq/116/smp_affinity
root@OpenWrt:~# echo 3f > /sys/class/net/usb0/queues/rx-0/rps_cpus
ash: write error: Value too large for data type
root@OpenWrt:~# echo f > /sys/class/net/usb0/queues/rx-0/rps_cpus <<<---- (This only worked , where only CPU 0 will run )
How I can let OpenWrt use CPU 1 and 2 same time ?
echo 6 > /proc/irq/116/smp_affinity
I tried this echo and didn't work for dual cores (1 & 2) only core 1 was running and core 2 idle
When I tried with ChatGPT 4 ,each step with sharing results with GPT ... he/it felt there is something wrong with Kernel or Network Drivers
Dear @jdwl1o1
let me correct something with what you said it . Where already many of peoples agreed the issue >> either with Drivers or Kernel due to no assigning properly of CPUs utilization ...
Where already mentioned above by me and and other members ....( read previous post )
ChatGpt all his information based on your inputs so if you gave him a proper information with full details ,, will give you a good output ...
Where all his points share with me to follow commands and other things ,,, no benefit...... so he/it understand there limitation on driver or kernel .... Which is i believe that and we mentioned that before asking ChatGPT..
So let focus on our subject where the gap on this system to solve this issue
Not running behind what ChatGPT is saying... is right or wrong
I appreciate anyone if he/she have the knowledge to add something to solve this gap or fix this bug
Any one have idea about Package Tasksethttps://openwrt.org/packages/pkgdata/taskset
... I approched to linux community so they advised me to utilize Taskset to force CPUs to work as per tasks
I appreciate any can guide me with steps so I can assign functions to CPUs
Maybe the selected PID was not related to network of modem ...
Note: echo 2 > /proc/irq/116/smp_affinity this command added to startup of system to avoid utilization of CPU 0 and focusing on CPU 1 to avoid PCIe speed limitation
@bmork@IceG
I would like to share with you latest tries, step by step Video on YouTube - 4 minutes.. Private video ( Where I installed latest OpenWrt firmware with Kernel 6.6.36)
Testing with :
Packet Steering
offloading
IRQ-Balance
echo 6 > /proc/irq/116/smp_affinity OR echo 2 > /proc/irq/116/smp_affinity( you can see CPUs utilization with this command) and speeds at Minute 3:30
I have read the whole discussion and I'm still confused due to conflicting opinions.
I will buy a module soon (not sure if Quectel) and I'm studying so many papers to ensure I will get the best performance.
Please correct me if what I have understood is not right.
This is an old OpenWrt bug since 2023 and is not going to be solved
This only applies to Quectel modules (all the models)
This only applies to USB connection for which, with or without bug, I have been told that it is always going to be slower. These modules seems to be available in different variants: "PCIe only", "PCIe+USB" and "USB only". I have understood that the "PCIe only" or "PCIe+USB" (when in PCIe mode) ones always get full speed when they use ethernet as output (with or without such bug).
Processor speed is another important factor that could affect speed in a negative way. I'm surprised to see that an expensive Banana BPI-R4 can't handle even speeds below 1Gbps. There are other alternatives with faster processors (ex Raspberry PI 5 with a quad-core Arm Cortex A76 processor @ 2.4GHz), but I suspect that even if they could work well now, they are not future proof and they will fail when 5G network will be updated to faster speeds (right now, I'm not even sure it can handle an mmWave speed). I'm wondering if I'm better off with a Mini ITX. Not only is much cheaper, but it is easier customizable and expandable. There are some with PCIe slot where, at least in theory, I should be able to to connect a cheap 10Gbit network card. With a 10Gbit capable board I suppose I will be free from maintenance costs for many and many years (apart the modules upgrade, of course).
This issue cause by OpenWrt System .....and its to all modems Quectel , Fibocom.... etc ...
Where if you tried other board (Driver Free) you will not face this issue
USB and PCIe modes
I found the reason for lower speeds of Network specially on PCie..... (Because the OpenWrt system CPU assigning )......I will add it later on this post
Reason for speed limitation PCIe 5G modems :
The system OpenWrt by default is assigning the main important roots to only Core 0 (CPU 0) even if you try echo f > /proc/irq/XX/smp_affinity
or echo 0f > /proc/irq/XX/smp_affinity
will not solve the problem ,still will be assigned to Core 0 (CPU 0 ) ....
Solution :
you need to do it manually from htop package or Taskset package(by using PID) to let system assign 4 Cores on network root to increase the network performance
:
root : napi/mtk_eth-5
Mostly will be their other roots from same list to be assigned to all CPUs @brada4 Thank you for guiding my before a month how to use (htop) @Thanks for Linux team to guide me how to use Taskset to load CPU
These are very bad news! All brands and all variants! If I should get 800Mbps and I only get 350Mbps, then I consider this as something that "does not work!".
I would like to know what have to say developers, but one of them has been pretty clear:
If I have understood well, you're trying to move traffic over USB and it does not matter if port 3.0 is capable of 4.8 Gbps. You lose speed anyway just for the complex stuff that has to be done to route traffic to a protocol not designed for network (USB in this specific case). This is what I have read on multiple sources, it is not just @981213 personal opinion. You should do M2>2.5G Ethernet
I continue to see reports of reduced speed due to processors. I'm not sure at all that such boards are good enough for such stuff.
I never had chance to play with OpenWrt, so I'm extraneous to "Driver free" concept. You demonstrated that Mcuzone board provides full speed, but from what I see they still use FriendlyWrt, a customized OpenWrt suitable for applications in IoT, NAS and etc. Did you tried it?
I think that users who visit this topic will get confused as happened to me. They probably already expect lower speeds on USB mode, but absolutely not with a direct connection to a 2.5GBit port and the reason is not clear.
Thanks for sharing the solution. This is is with an USB connection right? I am curious to see a M2>RJ45 speed test (without any kind of USB overhead) to see how much speed is lost.
This is with a Quectel RM551E-GL and a Banana BPI-R4 if I'm not wrong. Could you also share bands configurations?
This is probably the most expensive module in the market, around $500-$600 (featuring an x75 Snapdragon, if I'm not wrong).
I'm surprised to see that you're getting the same speed you would get with a cheap $50 Mediatek T700 based module (which I also saw running at more than 1Gbps).
I didn't try FrindlyWrt .... I tried Immortalwrt ... ROOter and OpenWrt
where all of two Immortalwrt and ROOter are depending on OpenWrt Data
So if OpenWrt said ****** your self ,Other Wrts will copy and paste on their sources .....
Where I like ROOter ( they are doing a great job on their system ) ,this system is fully made for 5G and 4G modems10 of 10
Driver Free better than Openwrt system on board ( As network speeds ) because all of the network and inputs will be depended on 5G/4G modems .....
If you are asking me which one you are preferring ... i'm preferring OpenWrt system than others free driver boards
My Opinion :
The major issue here Developers on OpenWrt are ignoring such as issue from long time 2022 and 2023 ... So I wish to add this solution to next firmware of openwrt
Note :
I tried Fibocom modem FM190W ( X75 Snapdragon ) and I tried Quectel modem RM551E (X75 Snapdragon) & RM521F (X65) RM520 (X62) all of them have same issue on openwrt where the issue form system not modems
My tower speed limited to 800 or 900 mb/s where I got it on OpenWrt recently with doing that solution . I think with this solution I shared it previously ,their will be no speed limitation on PCIe mode for 5G/4G modems ( I hope that )
Many thanks for your contribution and detailed explanation. This topic should be pinned because it will be the main issue people will have to go through when using these modules with OpenWrt (at least people that know what speed to expect).
If MCUZone is Driver Free and they use FriendlyWrt, I'm wrong to deduce that even OpenWrt could be driver free capable? I also prefer OpenWrt because, I suppose, it allows a much deeper control. However, if OpenWrt causes a reduced speed, than it's another story. We are all trying to do whatever we can to achieve maximum performance and the last thing we want to see is precious bandwidth lost, expecially when the cause is purely software.
Is there an estimate of the speed reduction when used in conjunction with OpenWrt (and related forks) ??
Yes, I really can't believe such bug is so old and still unsolved, but I don't think it is developers fault. It's probably something that should be handled from the community by submitting the needed pull requests.
@KFO May I take advantage of your deep knowledge? I'm stuck from very long time because I don't know what modem and board I should buy.
My target tower has N.5 LTE bands + N.2 NR bands (N3, N78).
MODEM:
I initially thought to Fibocomm FM350-GL. It's just $40-50 and there is a good topic here about it.
It should come with (a very old and very entry level) Mediatek T700, the same chipset that comes with Zyxel NR5103 (in my country reached a speed of around 1Gbps when placed outdoor and connected to a tower similar to mine). It has also 4x4 MiMo and an high modulation (256QAM for download and 64QAM for upload), at least this is what stated on manual (firmware 1.0.6).
However, if I have interpreted well the user manual, it only support "3LTE+2NR" or "5LTE+1NR". Whatever combo I choose I'm going to lose two LTE bands or one NR band.
Given the cheap cost of this module, I would prefer to invest more money and take something better (as long as the gains are proportional to the extra money needed).
I have checked RM520N-GL (X62) which still have an acceptable price, but seems to perform even worse: "3LTE+1NR" or "2LTE+2NR".
Even the much more expensive RG520F-EU (X65, $200 and more) still performs worse ("3LTE+2NR") except for RM521F-GL ($250 and more): "4LTE+2NR".
So, with 200$ more for RM521F-GL, I am only able to aggregate one additional LTE band for a total of 4 bands of the 5 available. This is not all proportial to the extra price.
Is an X75 able to aggregate all bands available in my tower? I don't know, but even if it could, it would be hundred dollars more respect to an already expensive X65 and just for one additional LTE band.
I think Qualcomm/Quectel are way overpriced especially because they seems to perform equally (in some cases even worse) than an old Mediatek T700.
There are several brands in the market and I was hoping to find a better module than FM350-GL at a reasonable price. What you would recommend?
BOARD:
I initially thought to an MCUZone like the one you posted. The board itself is not expensive, but once you add the needed stuff (case, heatsink, fan) it is. In addition to that, it is likely that I will end up to put the modem outdoor (on the roof) if I'll see better performance. The board with PoE support is $40 more. It is going to cost lot of money and what you get is not certainly what you paid for.
There are Raspberries, Bananas (like the one you posted at beginning) or similar, but they still become very expensive once added the needed stuff and I think mostly because such boards comes with lot of unneeded stuff, at least in my case (I will end up to connect the modem to a router, so I only need the connection from the ethernet port). I saw lot of people not recommending these popular boards anymore because now the market offer much better and cheaper alternatives, but I have yet to explore such possibility.
As of now, what to me seems to be really the best solution is a Mini ITX board. It can be found for just 20$ and some also comes with PCIe slot where, in theory, I could connect a very cheap 10Gb network card and get a future proof board that should last for many and many years. The cost of the missing parts (RAM, CPU, enclosure etc) are pretty cheap and at worse I think it will always be much cheaper than the above solutions (and I suppose an x86 is always going to be more performant). And it's still a very compact solution (Mini ITX boards are just 17cm x 17cm in size).
What you think?
As I know they don't use any system on their boards ... they depend only on modem drivers ... each modem has own drivers installed on it by manufacture
OpenWrt is not driver free
All of them same speeds ... because as I mentioned they are doing copy & paste with some editing based on their requirements and targets
I advise you to buy Fibocom FM190W (X75) 5G/4G ---> long use ...price approximately 280$
if you can't pay this amount then go with Quectel RM520 (X62) 5G/4G ---- around 100$
Note: RM521-F abandoned on place as per Quectel .. they mentioned on their website ( no more support for it )
Board :
if you are looking for high speeds . better to go with Driver free board
like this one ... Aliexpress (Best Board)
if you are looking to have apps and a good add-on on home network better to go with openwrt with Banana BPI-R4
Finally : OpenWrt (Team / Developers) need to adjust system root & network to let system use all core functionally
OpenWrt with solution
Driver Free ( Download Speed is similar to Openwork ---- upload is better on Driver free board .... i think on OpenWrt there is a root for upload need to be assigned to all CPUs)
At end : i'm Mechanical Engineer I don't have experience with programing or other things but I like to find solution for issues ... i'm sure there are a better people .they can do a better solution than me with full details with roots to main source
@KFOYou deserve some special reward from the whole community! I'm stuck from months on this and with just a couple of your messages I have a much clear idea on what to do.
You're right! They use FriendlyWrt on other boards, but not on this one. I must have checked the wrong page when I posted this info.
There's not much difference with a driver free board. Upload running on a single CPU? Well, I bet there is a way to assign upload to all CPUS and in this case performance will become very near to driver free board. I think there is also another way to improve speed. Depending from the board, there could be several checks on IP tables that could slow down speed. It is possible to disable the checks through an option called "Fasttrack" and get a speed boost. Here is where I have discovered this trick (you can watch the whole video or skip to minute 16:15 where is described the trick). Spoken language is Italian, but you can enable subtitles (translation is pretty accurate). The guy talked about an RBM board in USB mode, but I suspect such checks may occur in any case. It may worth to give it a try!
Oh yes, I know GSMANTENNA TECH Store. I had with him some messaging. He is currently developing a new version (which should have been available by the end of June) and just yesterday I asked him the status. This is the picture of the work in progress he sent me:
The board looks different from the one you linked and I deduce it is out of stock because it is going to be replaced by this one. To me seems a better option than MCUZone. If I have understood well, for $60 you get everything (case, cooling system etc), it only needs antennas. I also see that port is PoE which is good for outdoor (MCUZone requires $40 more for PoE).
I can't see the CPU, but from your screenshot I see 1700Mbps in download and 160Mbps in upload (and in USB mode). I would say that CPU is great for high speeds. Have you been able to reach 1700Mbps on Banana?
My only concern is that I don't think the case can easily be adapted for outdoor. Although I can make it waterproof with silicone, there are still exposed parts very hard to seal (power plug, usb etc). In theory I only need the ethernet port (for data transfer and power supply), so may be that the guy can provide a case with all the other holes closed?
There's something that I don't understand: why 2 SIM slots?
I also checked Mikrotik boards, but I only see 1Gbit ethernet ports and very low power CPU processors (2 cores, 800Mhz). Mikrotik boards does not worth at all, right?
Very hard choice between a driver free board like the above or this especially because speed difference seems to be nearly unnoticeable.
Let me understand one thing: did you purchased a full featured Banana BPI-R4 because you want to make it a full featured router powered by OpenWrt?
I had the same idea at the beginning, but OpenWrt developers and gurus strongly discouraged me from such approach.
They recommended me to keep the modem functionality separated through a cheap board and connect it to a router (which could be an x86 powered by OpenWrt or PfSense). Same for a file/media server and wifi. They should be kept separated and the whole network joined through switches. You can see more info on the discussion here (look for @psherman messages specifically).
Yes, I have intention to enhance the connection with apps, but through separated and dedicated devices as recommended from OpenWrt gurus. From the modem I will only get the connectivity, all the rest will be done from the dedicated devices.
In such configuration I'm probably better off with a driver free board. However, I still think that a cheap developer board running OpenWrt offers more advantages like the possibility to extend with something I may need in the future and, I believe, a full control of the modem. Perhaps some cheap variant of a Nano Pi or similars (one 2.5Gbit PoE port + PCIe slot for the module is probably all I need). Do you have something to recommend? May be two 2.5Gbit ports are even better if they can be aggregated into one 5Gbit port (for a more future proof board).
If there is anything like that I will fallback to the board you linked (the new model).
I'm still in doubt if an x86 mini board could be a better option. For example this board on sale for $36 has one PCI-E 16x slot where I could connect a cheap 10Gbit network card (only $20). There are also two M.2 slots for the modem module. If I search better I can probably go even lower with the total cost. A 10Gbit capable modem will last for many and many years. I feel like it is not easy as I think, but I have no idea on what could go wrong. Do you see anything wrong with such approach?
This helped me a lot! Many thanks! I have no problem to invest $100 for an RM520 (X62) (GLOBAL or EUROPEAN version I still don't know), even more if I can see real benetifs.
My tower details: TDD sub-6GHz AAU, FDD Passive, DSS N3 N78 B1 B3 B7 B20 B38
On a Zyxel MC889 (x62) I saw in a review all the five LTE bands I have in my tower successfully aggregated with N78:
I suppose the same applies to RM520 module, right?
However, no N3 aggregation and since this is an NR band, I suppose I'm going to lose a good amount of bandwidth. Is there a way to force "N3 + N78" aggregation on RM520? I downloaded the user manual, but is very hard to understand the combos for someone that never had chance to play with a 5G modem. What you can recommend as an alternative module capable to aggregate the N3 I'm loosing with RM520? I can spend more than $100 as long as the extra money is proportional to the value I will get.
I was looking to Fibocom FM350-GL user manual and to me seems capable to aggregate 5 LTE bands + 1 NR band, just like RM520:
What makes RM520 better than FM350-GL? I am not an expert and I'm sure I'm missing several obvious things.
I have not been able to find any info about Fibocom FM190W supported aggregations. Is this modem capable to aggregate all the bands from my tower included N3+N78 ? $280 just to have an additional band is too excessive. Hope there is something with a more reasonable price.
low utilization) 
