Previously I used their image and it doesn't even boot....maybe better now, but with official OpenWrt snapshot I don't see any point to go back.
That's a bit shocking. The whole purpose of vendor firmware is for their hardware to 'work' out of the box. And provides a baseline for HW evaluation and performance comparison. Perhaps Banana Pi could do a better job for R5...
Strange, it booted for me just fine. Did that first, to see if the hw worked at all 
Hello there, did you manage to solve this?
I tried both patches and enabled hardware offloading, it's not dropping connections anymore, but when I doing speedtest, I'm still seeing cpu usage increasing? How would we verify if the hardware offloading properly working?
Solve what specifically?
Is my understanding that the SFP+ optical module connecting your Mikrotik router to the R4 not working, correct?
IIUC there is an issue with SFP/SFP+ modules that are 2.5Gb/s (realtek chipset), other modules that are based on aquantia or marvell (10Gb/s) are working well.
Also IIUC, the bandwidth negotiation does not work (on the realtek chipsets), so if you have 2.5G module, set to 2.5Gb and turn off negotiation.
Please someone correct me if I'm wrong.
I will soon be using one of the 10Gbit variety with specs as posted by buz earlier which made me curious of what the problem was, hence my question.
Most 2.5G modules expose a built-in PHY which can be accessed via I2C (some using Marvell and some using RollBall protocol, sometimes quirky). We've added detection-quriks to the kernel for most of them, and if the PHY (RealTek RTL8211B in most cases) is detected in that way they work fine with all OpenWrt boards, incl. the R3 and R4.
Some other 2.5G modules (like TL-SM410U) which do not expose the PHY (or at least we haven't found out yet how to access it) also work fine but only with 2.5G link partners.
Yet other PHYs without exposed PHYs still work fine even with 10M/100M/1000M link partners as they perform rate adaptation using pause frames -- all you need to do is enable flow-control.
However, it is (sadly) true that there are also many 2.5G copper/RJ-45 modules which just don't work. The main reason for this is that there is no standard covering how 2.5G PHYs should be attached and especially also how they should deal with link partners with lower speed. So every vendor has their own interpretation on how things should be done, and they differ a lot.
For 1G the story is pretty clear: It's SGMII for copper SFPs and 1000Base-X (fixed speed) for fiber. Very rarely you see copper SFPs which still work with 1000Base-X and some of those (poorly) provide rate-adaption for 10M/100M and/or half-duplex link partners. But that's rare.
For 10G it's also again very clear: USXGMII is used is most cases and in some rare cases 10GBase-R with pause-frame-based rate-adaptation. Practically all of them allow accessing the PHY via I2C using the RollBall protocol, so even changing PHY parameters in ethtool works as expected.
And that's the situation with Ethernet copper SFPs. Fiber Ethernet PHYs generally all work fine, haven't found or heard of any exceptions.
xPON SFP modules are again a whole different and much more messy story, most of them can be configured internally to the desired host interface, and many need additional kernel quirks added.
6 Likes
No, there you misunderstood me. The SFP+ in the RB5009 sits really tight and the fiber patch cable going into it is flimsy. So not to risk my uplink, I ordered a second FS SFP+ (specifically, the generic one, https://www.fs.com/products/74681.html) and two more patch cables to try with the R4. Will report when I have them.
There is, BTW, also 10G optics from sinovoip (e.g., https://www.aliexpress.com/item/1005006634780419.html?algo_exp_id=b4f151bd-66f2-4068-9694-89cbf9d0a42e-2&utparam-url=scene%3Asearch|query_from%3A) but
a) they are 20km (which would probably be ok, but I'd want to check with my ISP first, no good frying their optics)
b) I could not yet figure out how you get the proper RX/TX combo (or if indeed they send you a matching pair?)
As this will be running at native 10gbit, I envision no rate-related issues. FWIW, this is with init7.net.
If this works, next step will probably be to try a 10gbit DAC to connect the Mikrotik to the R4 which would then be acting as border router (or possibly get a simple 10gbit switch and use that).
1 Like
The PPE works, it reduces the CPU usage by a significant amount, but not as much as you'd expect. It might be misconfigured. You can see its bindings in the debugfs as well. I'll look into it when I have time
My R4 LAN MAC still changed after reboot
@daniel
Will the above mentioned patches be included in future OpenWrt snapshots?
I wonder for vanilla OpenWrt if I want to use emmc do I still need to follow bpi r4 documentation on the bpi site?, or are the methods changed into easier ones?
who can point me to a good direction 
for now I only managed to get serial access and having OpenWrt on a sd card,but if I need to follow bpi documentation I understand that I have to flash from sd to nand and then use a usb stick(?) to flash from nand to emmc, that is alot of steps! 
will a sysupgrade once being emmc be easier or do I also need to go through the same process each time?
Under kernel 6.6 openwrt NCM865 card is actually unknown to get more details about the card (no support for ath12k).
only lspci:
0001:01:00.0 Network controller: Qualcomm Technologies, Inc WCN785x Wi-Fi 7(802.11be) 320MHz 2x2 [FastConnect 7800] (rev 01)
Confirmed you are right! I was only able to hit 4.3G with iperf3 test but now it hits full 10G!!!
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.36 GBytes 2.02 Gbits/sec 13 sender
[ 5] 0.00-10.00 sec 2.35 GBytes 2.02 Gbits/sec receiver
[ 7] 0.00-10.00 sec 2.69 GBytes 2.31 Gbits/sec 10 sender
[ 7] 0.00-10.00 sec 2.69 GBytes 2.31 Gbits/sec receiver
[ 9] 0.00-10.00 sec 3.28 GBytes 2.82 Gbits/sec 9 sender
[ 9] 0.00-10.00 sec 3.28 GBytes 2.82 Gbits/sec receiver
[ 11] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec 10 sender
[ 11] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec receiver
[SUM] 0.00-10.00 sec 11.0 GBytes 9.42 Gbits/sec 42 sender
[SUM] 0.00-10.00 sec 11.0 GBytes 9.41 Gbits/sec receiver
2 Likes
device select Jumper Setting SD 1 1
7. Install bootloader, recovery and production to NAND.
device select Jumper Setting NAND 0 1
9. Install bootloader, recovery and production to eMMC.
optional resize eMMC
- Run default boot command.
cat /proc/partitions
opkg update
opkg install parted
parted /dev/mmcblk0
(parted) print
(parted) resizepart 5
device select Jumper Setting eMMC 1 0
2 Likes
generic FS optic is detected:
[ 8.972144] sfp sfp1: module FS SFP-10G-BX rev A sn F2320040965 dc 231228
Could not yet hook it up to ISP to see what happens.
Fiber works perfectly:
root@OpenWrt:~# iperf3 -c speedtest.init7.net
Connecting to host speedtest.init7.net, port 5201
[ 5] local 212.51.131.2 port 35048 connected to 77.109.175.63 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.09 GBytes 9.38 Gbits/sec 0 2.90 MBytes
[ 5] 1.00-2.00 sec 1.09 GBytes 9.41 Gbits/sec 0 2.90 MBytes
[ 5] 2.00-3.00 sec 1.09 GBytes 9.40 Gbits/sec 0 3.05 MBytes
[ 5] 3.00-4.00 sec 1.10 GBytes 9.42 Gbits/sec 0 3.05 MBytes
[ 5] 4.00-5.00 sec 1.09 GBytes 9.41 Gbits/sec 0 3.05 MBytes
[ 5] 5.00-6.00 sec 1.10 GBytes 9.42 Gbits/sec 0 3.05 MBytes
[ 5] 6.00-7.00 sec 1.10 GBytes 9.42 Gbits/sec 0 3.05 MBytes
[ 5] 7.00-8.00 sec 1.09 GBytes 9.41 Gbits/sec 0 3.05 MBytes
[ 5] 8.00-9.00 sec 1.10 GBytes 9.41 Gbits/sec 0 3.05 MBytes
[ 5] 9.00-10.00 sec 1.10 GBytes 9.42 Gbits/sec 0 3.05 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 11.0 GBytes 9.41 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 11.0 GBytes 9.41 Gbits/sec receiver
To be clear, this is running on the router, not through the router.
3 Likes