Unfortunately not, didn't really ever have the tools to even attempt something like that.
Just sniffing would probably be the "easy" part, then you gotta decode that
Yep, that's why I had the idea to just hook it up to some GMAC or something which would do that for me. But I am not sure whether the receive chain is independent and whether or not one can load two receivers to such diff pairs. Let's see...
BTW: I found a shop in Switzerland that supposedly has the RB5009UG on stock and ordered one. So I might receive it early next week. Let's see...
I don't think you can add just add another GMAC, impedances will be completely out of spec and you will have 2 MAC-s trying to signal the same PHY which will at best completely break the in-band.
Great, its nice to see that RB5009 is popular.
If your needing a vendor, Baltic Networks still has some in stock and the shipping was pretty fast. Got mine a few days ago. Not a EE but I've been following this thread for a while and really hoping this can get wrt support. Hopefully I can be a guinea pig or add ideas into this thread (if I can.)
3 Likes
That HW is at a good price, if it will be supported by Openwrt I could also give a try for my ISP. (I have IPv4aaS and it is cpu intensive normal hw fails on full uplink throughput with v4 traffic) Right now I'm using a x86 machine. Openwrt is the only known CPE that support the MAP-T protocol with my ISP.
I'm currently on an EdgeRouter 4 but seems Cavium is a bit of a dead end on Linux - and I found it just locks up under load. I'm eyeing this one as a replacement.
Oh, you are one of the guys chasing the 5.10 memory leak?
Yeah, except my EdgeRouter 4 locked up under load on 21.02 while moving a few hundred GB. So haven't even tested 5.10 on that one yet (daily driver and homeworking and whatnot), just on my EdgeRouter Lite. I don't have the skills to chase it down and go digging through the code myself either, unfortunately. But people more familiar with the Octeon code said it's a mess and I believe them.
Would you mind sharing the shop you found it?
In Switzerland, I usually just check prices and availability here:
This time I picked wifx.net due to availability. And indeed, a couple hours later I already got a tracking number...
I needed to take a break from IPQ807x so went looking for the SFP signals, basically, when I insert the module in U-boot the following ones change:
CP GPIO0:
2 goes LOW
6 goes LOW
11 goes LOW
So these are probably MOD_ABS(Presence) and LOS signals as the link are down, also could be TX disable as well.
I have a friend making me a breakout to plug in the SFP slot with LED-s so you just toggle the GPIOS and see which one is which signal but I am impatient
2 Likes
Ok, found the 2 blue LED-s labeled HDR1 and 2, both are active LOW.
HDR1 is on CP0 GPIO1 pin 4
HDR2 is on CP0 GPIO2 pin 19
I was looking for the QCA8081 reset, but going through all of the GPIOS-s got me nowhere.
I mean it could not even be connected to a GPIO but rather just pulled up so it doesn't float and randomly reset.
1 Like
Then we found out the same 
never heard of that shop so I was hesitant, unfortunately neither brack nor one of the ususal suspects has it stocked...
1 Like
baltic networks in the US has them stocked now, I waited on my pre-order from them for 3 months but now they are shipping same day 
3 Likes
There is an easy way to find out which GPIOs are occupied by the kernel (even without debugfs). here it is:
/ # for a in $(seq 0 19); do echo $a > /sys/class/gpio/export || echo "gpio$a is BUSY"; done
/ # for a in $(seq 32 63); do echo $a > /sys/class/gpio/export || echo "gpio$a is BUSY"; done
sh: write error: Device or resource busy
gpio32 is BUSY
sh: write error: Device or resource busy
gpio33 is BUSY
sh: write error: Device or resource busy
gpio34 is BUSY
sh: write error: Device or resource busy
gpio35 is BUSY
sh: write error: Device or resource busy
gpio37 is BUSY
sh: write error: Device or resource busy
gpio38 is BUSY
sh: write error: Device or resource busy
gpio43 is BUSY
sh: write error: Device or resource busy
gpio60 is BUSY
/ # for a in $(seq 64 94); do echo $a > /sys/class/gpio/export || echo "gpio$a is BUSY"; done
sh: write error: Unknown error 524
gpio71 is BUSY
sh: write error: Unknown error 524
gpio72 is BUSY
sh: write error: Unknown error 524
gpio73 is BUSY
sh: write error: Unknown error 524
gpio74 is BUSY
sh: write error: Unknown error 524
gpio75 is BUSY
sh: write error: Device or resource busy
gpio87 is BUSY
sh: write error: Device or resource busy
gpio89 is BUSY
sh: write error: Device or resource busy
gpio90 is BUSY
sh: write error: Device or resource busy
gpio91 is BUSY
So the RouterOS kernel uses the following GPIO:
32 - i2c-gpio:SDA
33 - i2c-gpio:SCL
34, 35, 37, 38, 43 - ?
60 - button
71, 72, 73, 74, 75 - ?
87 - usb-power-off
89 - sfp-led
90 - user-led
91 - all-leds
So maybe gpio-i2c is the QCA8081 PHY control ?
[ 39.898966] i2c-gpio i2c-gpio: GPIO lookup for consumer sda
[ 39.898979] i2c-gpio i2c-gpio: using lookup tables for GPIO lookup
[ 39.898998] i2c-gpio i2c-gpio: GPIO lookup for consumer scl
[ 39.899005] i2c-gpio i2c-gpio: using lookup tables for GPIO lookup
[ 39.899060] i2c-gpio i2c-gpio: using lines 32 (SDA) and 33 (SCL)
Although no, this is most likely I2C for SFP!
No, I2C GPIO is for the SFP, already confirmed that 0x50 and 0x51 pop up on the bus when SFP is inserted.
I would say that we have most of the pins, I have a basic OpenWrt DTS just to test things, will share it.
LED-s and button are all known, LED and USB power as well as the bitbanged I2C ones.
Need to spin up a 5.16 image to test the switch driver and see whether it works without the PHY being controlled.
It would be awesome to static compile the gpiod utils, then it would be easier to see what registered a GPIO and whether its input or output.
As promised, here is the WIP DTS:
// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
/dts-v1/;
#include "armada-7040.dtsi"
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
/ {
model = "MikroTik RB5009";
compatible = "mikrotik,rb5009", "marvell,armada7040",
"marvell,armada-ap806-quad", "marvell,armada-ap806";
chosen {
stdout-path = "serial0:115200n8";
};
memory@0 {
device_type = "memory";
reg = <0x0 0x0 0x0 0x40000000>;
};
aliases {
led-boot = &led_user;
led-failsafe = &led_user;
led-running = &led_user;
led-upgrade = &led_user;
ethernet0 = &cp0_eth0;
};
sfp_i2c: sfp-i2c {
compatible = "i2c-gpio";
sda-gpios = <&cp0_gpio1 0 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>;
scl-gpios = <&cp0_gpio1 1 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>;
};
keys {
compatible = "gpio-keys";
reset {
label = "reset";
gpios = <&cp0_gpio1 28 GPIO_ACTIVE_LOW>;
linux,code = <KEY_RESTART>;
};
};
leds {
compatible = "gpio-leds";
led_user: user {
label = "green:user";
gpios = <&cp0_gpio2 26 GPIO_ACTIVE_LOW>;
};
sfp {
label = "green:sfp";
gpios = <&cp0_gpio2 25 GPIO_ACTIVE_LOW>;
};
hdr1 {
label = "blue:hdr1";
gpios = <&cp0_gpio1 4 GPIO_ACTIVE_LOW>;
};
hdr2 {
label = "blue:hdr2";
gpios = <&cp0_gpio2 19 GPIO_ACTIVE_LOW>;
};
};
sfp: sfp {
compatible = "sff,sfp";
i2c-bus = <&sfp_i2c>;
};
};
/* microUSB UART console */
&uart0 {
status = "okay";
pinctrl-0 = <&uart0_pins>;
pinctrl-names = "default";
};
&spi0 {
status = "okay";
spi-flash@0 {
#address-cells = <1>;
#size-cells = <1>;
compatible = "jedec,spi-nor";
reg = <0x0>;
spi-max-frequency = <20000000>;
partitions {
compatible = "fixed-partitions";
#address-cells = <1>;
#size-cells = <1>;
partition@0 {
compatible = "mikrotik,routerboot-partitions";
#address-cells = <1>;
#size-cells = <1>;
label = "RouterBoot";
reg = <0x0 0x1000000>;
read-only;
hard_config {
read-only;
size = <0x2000>;
};
soft_config {
};
dtb_config {
read-only;
};
};
};
};
};
&cp0_nand_controller {
status = "okay";
nand@0 {
reg = <0>;
nand-rb = <0>;
nand-ecc-mode = "hw";
nand-ecc-strength = <4>;
nand-ecc-step-size = <512>;
partitions {
compatible = "fixed-partitions";
#address-cells = <1>;
#size-cells = <1>;
partition@0 {
label = "RouterBoard NAND Boot";
reg = <0x0 0x800000>;
};
partition@800000 {
label = "RouterBoard NAND Main";
reg = <0x800000 0x3f800000>;
};
};
};
};
&cp0_gpio2 {
enable-usb-power {
gpio-hog;
gpios = <23 GPIO_ACTIVE_HIGH>;
output-high;
line-name = "enable USB power";
};
enable-leds-power {
gpio-hog;
gpios = <27 GPIO_ACTIVE_HIGH>;
output-high;
line-name = "enable LED-s power";
};
};
&cp0_usb3_1 {
status = "okay";
};
&cp0_i2c0 {
status = "okay";
clock-frequency = <100000>;
};
&cp0_mdio {
status = "okay";
};
&cp0_ethernet {
status = "okay";
};
&cp0_eth0 {
/* This port is connected to 88E6393X switch */
status = "okay";
phy-mode = "10gbase-r";
managed = "in-band-status";
};
&cp0_mdio {
status = "okay";
switch@0 {
/* Actual device is MV88E6393X */
compatible = "marvell,mv88e6190";
#address-cells = <1>;
#size-cells = <0>;
reg = <0>;
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
label = "cpu";
ethernet = <&cp0_eth0>;
};
port@1 {
reg = <1>;
label = "p8";
phy-handle = <&switch0phy1>;
};
port@2 {
reg = <2>;
label = "p7";
phy-handle = <&switch0phy2>;
};
port@3 {
reg = <3>;
label = "p6";
phy-handle = <&switch0phy3>;
};
port@4 {
reg = <4>;
label = "p5";
phy-handle = <&switch0phy4>;
};
port@5 {
reg = <5>;
label = "p4";
phy-handle = <&switch0phy5>;
};
port@6 {
reg = <6>;
label = "p3";
phy-handle = <&switch0phy6>;
};
port@7 {
reg = <7>;
label = "p2";
phy-handle = <&switch0phy7>;
};
port@9 {
reg = <9>;
label = "p1";
phy-mode = "sgmii";
managed = "in-band-status";
};
port@a {
reg = <10>;
label = "sfp";
phy-mode = "10gbase-r";
managed = "in-band-status";
sfp = <&sfp>;
};
};
mdio {
#address-cells = <1>;
#size-cells = <0>;
switch0phy1: switch0phy1@1 {
reg = <0x1>;
};
switch0phy2: switch0phy2@2 {
reg = <0x2>;
};
switch0phy3: switch0phy3@3 {
reg = <0x3>;
};
switch0phy4: switch0phy4@4 {
reg = <0x4>;
};
switch0phy5: switch0phy5@5 {
reg = <0x5>;
};
switch0phy6: switch0phy6@6 {
reg = <0x6>;
};
switch0phy7: switch0phy7@7 {
reg = <0x7>;
};
};
};
};
@adron So shockingly it appears that the switch works under 5.16-rc8 with a gigabit on the 2.5G port.
I don't have a 2.5G switch port to test with, unfortunately, but it looks good enough to just work.
Yeah, you cant to 2500Base-X without pointing to the PHY as it needs to be reconfigured from the SGMII mode.
There also must be a GPIO bit-banged MDIO as MikroTik uses the Single-chip mode for the switch, meaning that it has to be the only thing on the MDIO bus as it will reply to any address on the bus.
This is according to the GPL dump in the MVPP2:
port->sw.mii_phy.phy_id = 0; /* single chip mode */
Also, from your pin list, I know that 34 (B2), 38 (B4) and 43 (B11) have something to do with SFP as they all changed to 0 once SFP was plugged in.
So they are most likely the presence and LOS pins but no idea which is which.
BTW, it appears that SFP works as well (10G DAC), but obviously no actual control over it
5 Likes
Did I miss anything?
BTW: Concerning the DSA switch driver, there are also still further fixes pending:
https://lore.kernel.org/netdev/20211130170151.7741-1-kabel@kernel.org/T/
Nope, that is just copy/paste from another 7040 board I did.
Yeah, those are already in 5.16-rc8
2 Likes
Here is the 5.15 based OpenWrt WIP tree
6 Likes