SFP support for MikroTik RB922UAGS-5HPacD

Update: SFP is supported since commit 4387fe00cb7536d9d341f6d27e465ff0f0b29b1b

Hi,

I recently got an SFP fiber module and I wanted it to work with the MikroTik RouterBoard 922UAGS-5HPacD we got supported in ath79 some time ago. I've made some progress, until the point that the SFP module is detected, but now I need some help on the .dts file.

The changes are at this branch https://github.com/rogerpueyo/openwrt/tree/ath79-mikrotik-rb922-sfp, and (here is the DTS file) to make the whole thing work. Current status is:

I²C bus + GPIOs: SFP device detected:

root@OpenWrt:/# [  789.802075] sfp sfp: module removed
[  794.716593] sfp sfp: module CISCO-OEM        GLC-LH-SMD       rev A    sn S1712160038      dc 171216  
[  795.667040] sfp sfp: module removed
[  799.004592] sfp sfp: module OEM              GLC-T            rev      sn 202005150065     dc 200515

DTS file: not able to talk to the SFP module

I've tried all combinations like:

&mdio1 {
status = "okay";

sfphy: ethernet-phy@0 {
	reg = <0>;
	sfp = <&sfp1>;
};

};

&eth1 {
status = "okay";

phy-handle = <&sfphy>;
phy-mode = "sgmii";

};

to try to bring the SFP up on eth1, but I can't make it send/receive anything. What can I try now?

Thanks!

@xback:

Some time ago, in Mikrotik RB922/RB921 QCA9557 & QCA9882 support?, you mentioned:

I got someone on chat who is also interested and may provide the code for making the SFP slot work :slight_smile:

Do you remember, by chance, who was that person? I got the slot to detect and talk to SFP modules, but I couldn't bring it up as a "regular" interface.

Cheers

You might have more success with a recent version of the mainline ag71xx driver, since that one has phylink support. Then SFPs are just Plug and Play, literally.

I don't know shit about these things, but I don't think you need those mdio and phy nodes. There's a MAC and an SFP slot. The sfp driver handles the slot, loads a phy driver if required (e.g for a copper SFP), and provides MII access to it for the MAC

So you'd have something like

&eth1 {
  status = "okay";
  phy-mode = "sgmii";
  managed = "in-band-status";
  sfp = <&sfp1>;
};

and no &mdio1 node. phy-mode is already set to sgmii, so you don't really need that either.

The only issue is getting that mac driver with phylink support.

I've send you a PM with the name.

Checking this stuff here in the lab ..

@bmork

Link to patch

Hi,

@bmork, thanks for your tips. Following your suggestion, I tried this configuration:

&eth1 {
 status = "okay";
 phy-mode = "sgmii";
//  managed = "in-band-status";
 sfp = <&sfp1>;
};

with and without the in-band-status option.

Enabling the in-band-status option makes the eth1 interface appear, but it causes an early kernel error, when loading the driver and later, more crashes at a rate of once per second:

Click here to see the early kernel error message
[    4.532468] libphy: Fixed MDIO Bus: probed
[    4.537263] ag71xx 19000000.eth: invalid MAC address, using random address
[    4.859530] libphy: ag71xx_mdio: probed
[    4.884707] ag71xx 19000000.eth: connected to PHY at mdio.0:04 [uid=004dd074, driver=Atheros 8031 ethernet]
[    4.895171] eth0: Atheros AG71xx at 0xb9000000, irq 4, mode: rgmii
[    4.901793] ag71xx 1a000000.eth: invalid MAC address, using random address
[    5.222837] ------------[ cut here ]------------
[    5.227573] WARNING: CPU: 0 PID: 1 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[    5.236044] Modules linked in:
[    5.239154] CPU: 0 PID: 1 Comm: swapper Not tainted 5.4.87 #0
[    5.244980] Stack : 806f0000 80697768 00000000 00000000 80696938 87c2da8c 87c28300 806cfd03
[    5.253463]         8062a86c 00000001 80b132d8 00000000 ffffffff 00000001 87c2da40 75a21740
[    5.261946]         00000000 00000000 80b40000 0000005d 61696e74 00000000 2e342e38 37202330
[    5.270429]         0000005d f81d5b68 00000000 0003a632 00000000 00000009 00000000 803d0aa0
[    5.278912]         00000009 00000000 ffffffff 80b10000 00000000 8035499c 00000000 80b10000
[    5.287395]         ...
[    5.289878] Call Trace:
[    5.292374] [<80069944>] show_stack+0x30/0x100
[    5.296896] [<80082624>] __warn+0xc0/0x10c
[    5.301061] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[    5.306107] [<803d0aa0>] swphy_read_reg+0x48/0x170
[    5.310987] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[    5.316032] [<803cfd74>] mdiobus_read+0x54/0x80
[    5.320634] [<803cec8c>] get_phy_device+0x6c/0x26c
[    5.325501] [<803db8f8>] __fixed_phy_register.part.10+0x98/0x27c
[    5.331602] [<803f0100>] of_phy_register_fixed_link+0x11c/0x1a4
[    5.337623] [<803dfb08>] ag71xx_phy_connect+0x38/0x130
[    5.342846] [<803de478>] ag71xx_probe+0x854/0xb80
[    5.347642] [<80368354>] platform_drv_probe+0x40/0x94
[    5.352772] [<803664a8>] really_probe+0x104/0x358
[    5.357550] [<80366bf0>] device_driver_attach+0x70/0x98
[    5.362857] [<80366c78>] __driver_attach+0x60/0x100
[    5.367811] [<803644a0>] bus_for_each_dev+0x68/0xa4
[    5.372764] [<8036580c>] bus_add_driver+0x1f0/0x200
[    5.377719] [<803672e4>] driver_register+0x84/0x148
[    5.382677] [<8070bea4>] ag71xx_module_init+0x2c/0x50
[    5.387813] [<80060a1c>] do_one_initcall+0x7c/0x1dc
[    5.392786] [<806f6e18>] kernel_init_freeable+0x158/0x23c
[    5.398268] [<80593e98>] kernel_init+0x10/0xf0
[    5.402782] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[    5.408271] ---[ end trace d531a7b043413d2e ]---
[    5.412959] ------------[ cut here ]------------
[    5.417662] WARNING: CPU: 0 PID: 1 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[    5.426130] Modules linked in:
[    5.429226] CPU: 0 PID: 1 Comm: swapper Tainted: G        W         5.4.87 #0
[    5.436459] Stack : 806f0000 80697768 00000000 00000000 80696938 87c2da8c 87c28300 806cfd03
[    5.444942]         8062a86c 00000001 80b132d8 00000000 ffffffff 00000001 87c2da40 75a21740
[    5.453425]         00000000 00000000 80b40000 00000080 20202020 00000000 2e342e38 37202330
[    5.461908]         00000080 978aab38 00000000 00068caa 00000000 00000009 00000000 803d0aa0
[    5.470391]         00000009 00000000 ffffffff 80b10000 00000000 8035499c 00000000 80b10000
[    5.478874]         ...
[    5.481350] Call Trace:
[    5.483843] [<80069944>] show_stack+0x30/0x100
[    5.488362] [<80082624>] __warn+0xc0/0x10c
[    5.492523] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[    5.497568] [<803d0aa0>] swphy_read_reg+0x48/0x170
[    5.502442] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[    5.507484] [<803cfd74>] mdiobus_read+0x54/0x80
[    5.512087] [<803ceca4>] get_phy_device+0x84/0x26c
[    5.516954] [<803db8f8>] __fixed_phy_register.part.10+0x98/0x27c
[    5.523053] [<803f0100>] of_phy_register_fixed_link+0x11c/0x1a4
[    5.529066] [<803dfb08>] ag71xx_phy_connect+0x38/0x130
[    5.534290] [<803de478>] ag71xx_probe+0x854/0xb80
[    5.539075] [<80368354>] platform_drv_probe+0x40/0x94
[    5.544207] [<803664a8>] really_probe+0x104/0x358
[    5.548985] [<80366bf0>] device_driver_attach+0x70/0x98
[    5.554292] [<80366c78>] __driver_attach+0x60/0x100
[    5.559245] [<803644a0>] bus_for_each_dev+0x68/0xa4
[    5.564199] [<8036580c>] bus_add_driver+0x1f0/0x200
[    5.569155] [<803672e4>] driver_register+0x84/0x148
[    5.574111] [<8070bea4>] ag71xx_module_init+0x2c/0x50
[    5.579239] [<80060a1c>] do_one_initcall+0x7c/0x1dc
[    5.584207] [<806f6e18>] kernel_init_freeable+0x158/0x23c
[    5.589694] [<80593e98>] kernel_init+0x10/0xf0
[    5.594209] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[    5.599694] ---[ end trace d531a7b043413d2f ]---
[    5.624110] ------------[ cut here ]------------
[    5.628868] WARNING: CPU: 0 PID: 1 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[    5.637342] Modules linked in:
[    5.640453] CPU: 0 PID: 1 Comm: swapper Tainted: G        W         5.4.87 #0
[    5.647688] Stack : 806f0000 80697768 00000000 00000000 80696938 87c2dad4 87c28300 806cfd03
[    5.656170]         8062a86c 00000001 80b132d8 00000000 87de8000 00000001 87c2da88 75a21740
[    5.664653]         00000000 00000000 80b40000 000000a3 20202020 00000000 2e342e38 37202330
[    5.673136]         000000a3 51370bf6 00000000 0009c5c5 00000000 00000009 00000000 803d0aa0
[    5.681619]         00000009 00000000 87de8000 80b10000 00000000 8035499c 00000000 80b10000
[    5.690102]         ...
[    5.692577] Call Trace:
[    5.695080] [<80069944>] show_stack+0x30/0x100
[    5.699600] [<80082624>] __warn+0xc0/0x10c
[    5.703758] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[    5.708804] [<803d0aa0>] swphy_read_reg+0x48/0x170
[    5.713683] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[    5.718729] [<803cfd74>] mdiobus_read+0x54/0x80
[    5.723331] [<803cf11c>] genphy_read_abilities+0x90/0x1c0
[    5.728812] [<803cf418>] phy_probe+0x1cc/0x260
[    5.733326] [<803ce378>] phy_attach_direct+0x254/0x304
[    5.738546] [<803ce4c8>] phy_connect_direct+0x28/0x8c
[    5.743678] [<803efef4>] of_phy_connect+0x4c/0xa0
[    5.748460] [<803dfb38>] ag71xx_phy_connect+0x68/0x130
[    5.753682] [<803de478>] ag71xx_probe+0x854/0xb80
[    5.758479] [<80368354>] platform_drv_probe+0x40/0x94
[    5.763609] [<803664a8>] really_probe+0x104/0x358
[    5.768387] [<80366bf0>] device_driver_attach+0x70/0x98
[    5.773692] [<80366c78>] __driver_attach+0x60/0x100
[    5.778648] [<803644a0>] bus_for_each_dev+0x68/0xa4
[    5.783601] [<8036580c>] bus_add_driver+0x1f0/0x200
[    5.788556] [<803672e4>] driver_register+0x84/0x148
[    5.793514] [<8070bea4>] ag71xx_module_init+0x2c/0x50
[    5.798640] [<80060a1c>] do_one_initcall+0x7c/0x1dc
[    5.803613] [<806f6e18>] kernel_init_freeable+0x158/0x23c
[    5.809095] [<80593e98>] kernel_init+0x10/0xf0
[    5.813610] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[    5.819094] ---[ end trace d531a7b043413d30 ]---
[    5.823868] ------------[ cut here ]------------
[    5.828588] WARNING: CPU: 0 PID: 1 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[    5.837056] Modules linked in:
[    5.840169] CPU: 0 PID: 1 Comm: swapper Tainted: G        W         5.4.87 #0
[    5.847403] Stack : 806f0000 80697768 00000000 00000000 80696938 87c2daac 87c28300 806cfd03
[    5.855885]         8062a86c 00000001 80b132d8 00000000 87de8000 00000001 87c2da60 75a21740
[    5.864368]         00000000 00000000 80b40000 000000c8 20202020 00000000 2e342e38 37202330
[    5.872851]         000000c8 06fc242e 00000000 000cd1e9 00000000 00000009 00000000 803d0aa0
[    5.881334]         00000009 00000000 87de8000 80b10000 00000000 8035499c 00000000 80b10000
[    5.889818]         ...
[    5.892293] Call Trace:
[    5.894790] [<80069944>] show_stack+0x30/0x100
[    5.899307] [<80082624>] __warn+0xc0/0x10c
[    5.903466] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[    5.908512] [<803d0aa0>] swphy_read_reg+0x48/0x170
[    5.913385] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[    5.918427] [<803cbd48>] __phy_modify_changed+0x38/0xa8
[    5.923732] [<803cbf50>] phy_modify+0x48/0x80
[    5.928159] [<803cdd80>] __phy_resume+0x48/0x90
[    5.932762] [<803cddf0>] phy_resume+0x28/0x4c
[    5.937187] [<803ce2d0>] phy_attach_direct+0x1ac/0x304
[    5.942406] [<803ce4c8>] phy_connect_direct+0x28/0x8c
[    5.947538] [<803efef4>] of_phy_connect+0x4c/0xa0
[    5.952326] [<803dfb38>] ag71xx_phy_connect+0x68/0x130
[    5.957544] [<803de478>] ag71xx_probe+0x854/0xb80
[    5.962337] [<80368354>] platform_drv_probe+0x40/0x94
[    5.967469] [<803664a8>] really_probe+0x104/0x358
[    5.972247] [<80366bf0>] device_driver_attach+0x70/0x98
[    5.977554] [<80366c78>] __driver_attach+0x60/0x100
[    5.982508] [<803644a0>] bus_for_each_dev+0x68/0xa4
[    5.987462] [<8036580c>] bus_add_driver+0x1f0/0x200
[    5.992418] [<803672e4>] driver_register+0x84/0x148
[    5.997375] [<8070bea4>] ag71xx_module_init+0x2c/0x50
[    6.002510] [<80060a1c>] do_one_initcall+0x7c/0x1dc
[    6.007483] [<806f6e18>] kernel_init_freeable+0x158/0x23c
[    6.012967] [<80593e98>] kernel_init+0x10/0xf0
[    6.017479] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[    6.022963] ---[ end trace d531a7b043413d31 ]---
[    6.027658] ag71xx 1a000000.eth: connected to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
[    6.037262] eth1: Atheros AG71xx at 0xba000000, irq 5, mode: sgmii

Click here to see the subsequent kernel error message:
[  264.809973] ------------[ cut here ]------------
[  264.814707] WARNING: CPU: 0 PID: 109 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[  264.823361] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack_rtcache nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc sfp nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 mdio_i2c iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat i2c_gpio i2c_algo_bit i2c_dev i2c_core nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 fsl_mph_dr_of ehci_platform ehci_fsl ehci_hcd gpio_button_hotplug usbcore nls_base usb_common
[  264.890929] CPU: 0 PID: 109 Comm: kworker/0:1 Tainted: G        W         5.4.87 #0
[  264.898706] Workqueue: events_power_efficient phy_state_machine
[  264.904719] Stack : 8062a86c 803d0aa0 00000009 00000000 806d0000 800b9808 87dba520 00000000
[  264.913208]         00000017 87ce01bc 8061bc7c 87d3bca4 806d0000 00000001 87d3bc78 ef551e6c
[  264.921690]         00000000 00000000 806df520 0000208f 00000065 00000000 74655f6d 61636869
[  264.930173]         0000208f 80b30000 00000000 000db692 00000000 00000009 00000000 803d0aa0
[  264.938656]         00000009 00000000 806d0000 806d0000 00000002 8035499c 00000000 80b10000
[  264.947139]         ...
[  264.949615] Call Trace:
[  264.952117] [<80069944>] show_stack+0x30/0x100
[  264.956639] [<80082624>] __warn+0xc0/0x10c
[  264.960806] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[  264.965851] [<803d0aa0>] swphy_read_reg+0x48/0x170
[  264.970740] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[  264.975786] [<803cfd74>] mdiobus_read+0x54/0x80
[  264.980395] [<803cd5b0>] genphy_update_link+0x20/0x100
[  264.985614] [<803cf714>] genphy_read_status+0x24/0x134
[  264.990831] [<803ca064>] phy_check_link_status+0x110/0x118
[  264.996401] [<803cac5c>] phy_state_machine+0xb8/0x1e4
[  265.001551] [<80099a9c>] process_one_work+0x220/0x480
[  265.006683] [<80099e64>] worker_thread+0x168/0x5c8
[  265.011553] [<8009ebf8>] kthread+0x138/0x140
[  265.015895] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[  265.021378] ---[ end trace e945e5ab7cf2c7ec ]---
[  265.026072] ------------[ cut here ]------------
[  265.030770] WARNING: CPU: 0 PID: 109 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[  265.039418] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack_rtcache nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc sfp nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 mdio_i2c iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat i2c_gpio i2c_algo_bit i2c_dev i2c_core nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 fsl_mph_dr_of ehci_platform ehci_fsl ehci_hcd gpio_button_hotplug usbcore nls_base usb_common
[  265.106974] CPU: 0 PID: 109 Comm: kworker/0:1 Tainted: G        W         5.4.87 #0
[  265.114747] Workqueue: events_power_efficient phy_state_machine
[  265.120757] Stack : 8062a86c 803d0aa0 00000009 00000000 806d0000 800b9808 87dba520 00000000
[  265.129238]         00000017 87ce01bc 8061bc7c 87d3bca4 806d0000 00000001 87d3bc78 ef551e6c
[  265.137721]         00000000 00000000 806df520 000020aa 00000065 00000000 74655f6d 61636869
[  265.146205]         000020aa 80b30000 00000000 0001c03b 00000000 00000009 00000000 803d0aa0
[  265.154687]         00000009 00000000 806d0000 806d0000 00000002 8035499c 00000000 80b10000
[  265.163170]         ...
[  265.165646] Call Trace:
[  265.168140] [<80069944>] show_stack+0x30/0x100
[  265.172659] [<80082624>] __warn+0xc0/0x10c
[  265.176819] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[  265.181865] [<803d0aa0>] swphy_read_reg+0x48/0x170
[  265.186747] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[  265.191790] [<803cfd74>] mdiobus_read+0x54/0x80
[  265.196392] [<803cd5fc>] genphy_update_link+0x6c/0x100
[  265.201609] [<803cf714>] genphy_read_status+0x24/0x134
[  265.206827] [<803ca064>] phy_check_link_status+0x110/0x118
[  265.212396] [<803cac5c>] phy_state_machine+0xb8/0x1e4
[  265.217536] [<80099a9c>] process_one_work+0x220/0x480
[  265.222669] [<80099e64>] worker_thread+0x168/0x5c8
[  265.227551] [<8009ebf8>] kthread+0x138/0x140
[  265.231895] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[  265.237383] ---[ end trace e945e5ab7cf2c7ed ]---
[  265.242068] ------------[ cut here ]------------
[  265.246767] WARNING: CPU: 0 PID: 109 at drivers/net/phy/swphy.c:127 swphy_read_reg+0x48/0x170
[  265.255414] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat ath9k_hw ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_table nf_conntrack_rtcache nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc sfp nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 mdio_i2c iptable_mangle iptable_filter ip_tables hwmon crc_ccitt compat i2c_gpio i2c_algo_bit i2c_dev i2c_core nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 fsl_mph_dr_of ehci_platform ehci_fsl ehci_hcd gpio_button_hotplug usbcore nls_base usb_common
[  265.322971] CPU: 0 PID: 109 Comm: kworker/0:1 Tainted: G        W         5.4.87 #0
[  265.330740] Workqueue: events_power_efficient phy_state_machine
[  265.336745] Stack : 8062a86c 803d0aa0 00000009 00000000 806d0000 800b9808 87dba520 00000000
[  265.345225]         00000017 87ce01bc 8061bc7c 87d3bcbc 806d0000 00000001 87d3bc90 ef551e6c
[  265.353709]         00000000 00000000 806df520 000020c5 00000065 00000000 74655f6d 61636869
[  265.362192]         000020c5 80b30000 00000000 00050bf4 00000000 00000009 00000000 803d0aa0
[  265.370675]         00000009 00000000 806d0000 806d0000 00000002 8035499c 00000000 80b10000
[  265.379158]         ...
[  265.381633] Call Trace:
[  265.384128] [<80069944>] show_stack+0x30/0x100
[  265.388647] [<80082624>] __warn+0xc0/0x10c
[  265.392814] [<800826cc>] warn_slowpath_fmt+0x5c/0xac
[  265.397863] [<803d0aa0>] swphy_read_reg+0x48/0x170
[  265.402741] [<803dbcc8>] fixed_mdio_read+0x124/0x148
[  265.407786] [<803cfd74>] mdiobus_read+0x54/0x80
[  265.412388] [<803cf7d0>] genphy_read_status+0xe0/0x134
[  265.417604] [<803ca064>] phy_check_link_status+0x110/0x118
[  265.423174] [<803cac5c>] phy_state_machine+0xb8/0x1e4
[  265.428317] [<80099a9c>] process_one_work+0x220/0x480
[  265.433446] [<80099e64>] worker_thread+0x168/0x5c8
[  265.438318] [<8009ebf8>] kthread+0x138/0x140
[  265.442660] [<80064d98>] ret_from_kernel_thread+0x14/0x1c
[  265.448142] ---[ end trace e945e5ab7cf2c7ee ]---

Disabling the in-band-status option avoids the crashes, but neither the eth1 interface appears.

Therefore, not working so far.

@xback, did you discover anything new? I tried reverting the patch you linked, but ended up with both eth0 and eth1not being loaded. It looked like if the ag71xx driver wasn't even loaded.

Some time ago on RB962 / hap ac (similar hardware) I tried to get SFP working:

I think that I worked out I needed to:

  • Avoid using the SFP tx-disable
  • use a fixed link in DTS
  • restart (power down other end, or unplug & replug) the optical link after OpenWrt was running.

Then I got a connection.

I decided I would wait for a newer kernel and ag71xx with phylink, as bmork suggested.

helped me to build kernel with CONFIG_KERNEL_DYNAMIC_DEBUG=y, and use
echo -n 'file sfp.c +p' > /sys/kernel/debug/dynamic_debug/control
so that sfp phy driver shows the hardware events.
cat /sys/kernel/debug/gpio | grep -E 'los|def0|tx' also helpful

Dear @johnth,

Thanks for the info! I couldn't make it work, but I think I advanced a little bit. I made some changes to the DTS file to make it look like yours:

	sfp1: sfp {
		compatible = "sff,sfp";

		i2c-bus = <&i2c>;
		maximum-power-milliwatt = <1000>;
		los-gpios = <&gpio 21 GPIO_ACTIVE_HIGH>;
		mod-def0-gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
		// tx-disable-gpios = <&gpio 16 GPIO_ACTIVE_LOW>;
		};

[...]

&mdio1 {
	status = "okay";

	phy_sfp: ethernet-phy@0 {
		reg = <0>;
		phy-mode = "sgmii";
		sfp = <&sfp1>;
	};
};

&eth1 {
	status = "okay";

	phy-mode = "sgmii";
	phy-handle = <&phy_sfp>;
	//pll-data = <0x03000000 0x00000101 0x00001616>;

	fixed-link {
		speed = <1000>;
		full-duplex;
	};
};

On the other end, running RouterOS, I see there is a correct signal reception level, but no link is established:

image.

However, unplugging/plugging the fiber cable or the whole SFP module on any of the ends makes no difference. Link is still down, and no traffic is received:

eth1      Link encap:Ethernet  HWaddr E4:8D:8C:AA:BB:CC
          inet6 addr: fe80::e68d:8cff:feaa:bbcc64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:8066 (7.8 KiB)
          Interrupt:5 

Also, I don't see any difference when setting/unsetting the pll-data values (I also tried other values for 1000 Mbps, to no avail).

Last thing, I tried compiling your image (stripping down all the nodes the rb922 doesn't have/need), and booted it. Unfortunately, I got the same exact results.

I guess we'll have to wait for a newer kernel/ag71xx driver. Anyway, thanks for your help! :smiley:

Sorry, I forgot dmesg, in case it provides any relevant information:

Click here for the whole dmesg blob
[    4.564284] libphy: Fixed MDIO Bus: probed
[    4.569069] ag71xx 19000000.eth: invalid MAC address, using random address
[    4.893515] libphy: ag71xx_mdio: probed
[    4.918778] ag71xx 19000000.eth: connected to PHY at mdio.0:04 [uid=004dd074, driver=Atheros 8031 ethernet]
[    4.929236] eth0: Atheros AG71xx at 0xb9000000, irq 4, mode: rgmii
[    4.935863] ag71xx 1a000000.eth: invalid MAC address, using random address
[    5.257522] libphy: ag71xx_mdio: probed
[    5.301643] ag71xx 1a000000.eth: connected to PHY at fixed-0:00 [uid=00000000, driver=Generic PHY]
[    5.311322] eth1: Atheros AG71xx at 0xba000000, irq 5, mode: sgmii
[...]
[    6.532287] at803x_config_aneg: enter
[    8.589127] eth0: link up (100Mbps/Full duplex)
[    8.593772] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   10.661812] eth0: link down
[   11.906245] i2c-gpio i2c: using lines 18 (SDA) and 19 (SCL)
[   11.921488] Loading modules backported from Linux version v5.8.18-0-gab435ce49bd1
[   11.929121] Backport generated by backports.git v5.8.18-1-0-ga630fd46
[   11.975882] libphy: SFP I2C Bus: probed
[   11.979932] sfp sfp: Host maximum power 1.0W
[   11.999801] sfp sfp: No tx_disable pin: SFP modules will always be emitting.
[...]
[   12.311349] sfp sfp: module CISCO-OEM        GLC-LH-SMD       rev A    sn S1712160530      dc 171216  
[...]
[   25.565547] at803x_config_aneg: enter
[   25.570079] br-lan: port 1(eth0) entered blocking state
[   25.575437] br-lan: port 1(eth0) entered disabled state
[   25.581017] device eth0 entered promiscuous mode
[   25.640903] eth1: link up (1000Mbps/Full duplex)
[   25.673432] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[   29.645122] eth0: link up (100Mbps/Full duplex)
[   29.649928] br-lan: port 1(eth0) entered blocking state
[   29.655260] br-lan: port 1(eth0) entered forwarding state
[   29.662708] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
// Unplug fiber
[   60.422237] sfp sfp: los 0 -> 1
[   60.422252] sfp sfp: SM: enter waitdev:detached:down event los_high
[   60.422260] sfp sfp: SM: exit waitdev:detached:down
// Plug fiber
[   62.839197] sfp sfp: los 1 -> 0
[   62.839212] sfp sfp: SM: enter waitdev:detached:down event los_low
[   62.839221] sfp sfp: SM: exit waitdev:detached:down
// Unplug fiber
[   86.805477] sfp sfp: los 0 -> 1
[   86.805491] sfp sfp: SM: enter waitdev:detached:down event los_high
[   86.805500] sfp sfp: SM: exit waitdev:detached:down
// Plug fiber
[   89.910788] sfp sfp: los 1 -> 0
[   89.910803] sfp sfp: SM: enter waitdev:detached:down event los_low
[   89.910812] sfp sfp: SM: exit waitdev:detached:down

Short update, I got the SFP working with:

&mdio1 {
	status = "okay";

	phy_sfp: ethernet-phy@2 {
		reg = <2>;
		phy-mode = "sgmii";
		eee-broken-100tx;
		eee-broken-1000t;
		at803x-override-sgmii-link-check;
		sfp = <&sfp1>;
	};
};

&eth1 {
	status = "okay";

	phy-mode = "sgmii";
	phy-handle = <&phy_sfp>;
	pll-data = <0x03000000 0x00000101 0x00001616>;
	qca955x-sgmii-fixup;

	fixed-link {
		speed = <1000>;
		full-duplex;
	};

	gmac-config {
		device = <&gmac>;
		ge0-sgmii = <0>;
	};
};

Most likely not all these settings are needed, so now I'll try to cleanup the dts and see what is actually necessary.

Cheers!

Hi, if you do not mind, a silly question - I have a couple of the RB922 boards and I am having no luck getting them to boot openwrt. Which procedure did you use to get them to boot? I believe I tried everything (bootp, tftp, soldering serial port on and figuring out it is disabled, the works), and it does not seem to work at all. While I can manage to get it to download an image, it simply refuses to boot it. Given total lack of any debug options I am at a loss about how to proceed now. Any help is highly appreciated!

Hi @alex.pyattaev,

I am using the procedure described here: https://openwrt.org/toh/mikrotik/common.

Long story short, you need:

  • The initramfs image for the device
  • A DHCP/BOOTP/TFTP server running on your computer (I use dnsmasq as explained in the link above)

To make the device download the image via TFTP, you can configure RouterOS (System=>RouterBOARD=>Settings=>Boot Device=>try-ethernet-once-then-nand) so that next boot will try to fetch an image via TFTP instead of booting regularly from NAND. Otherwise, just press the reset button, and keep it pressed, while you power up the device. After 10~20 seconds the device will try to fetch the boot image via TFTP.

Unfortunately, the serial port is disabled for RouterBOOT and RouterOS, but once you start booting OpenWrt's kernel, it will start showing you the bootlog:

[    0.000000] Linux version 5.4.113 (roger@builder) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r16523-448e4279e5)) #0 Sun Apr 18 14:14:52 2021
[    0.000000] printk: bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 00019750 (MIPS 74Kc)
[    0.000000] MIPS: machine is MikroTik RouterBOARD 922UAGS-5HPacD
root@OpenWrt:/# dmesg | head -n10
[    0.000000] Linux version 5.4.113 (roger@7018) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r16523-448e4279e5)) #0 Sun Apr 18 14:14:52 2021
[    0.000000] printk: bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 00019750 (MIPS 74Kc)
[    0.000000] MIPS: machine is MikroTik RouterBOARD 922UAGS-5HPacD
[    0.000000] SoC: Qualcomm Atheros QCA9558 ver 1 rev 0
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
[    0.000000] Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x0000000007ffffff]

Thanks! Got it to work! The trick with it was that network boot only works with TFTP, as per mikrotik instructions. Also booting via reset button is substantially more reliable.

Ok, small update on my experiments. While the release version appears to boot just fine, trying to build and boot snapshot does not (no signal on serialport, so it seems openwrt kernel does not even boot properly at all). The image gets fetched by the bootloader, but no signal on serial port is produced, so probably it does not even recoginze its format or kernel just panics out and reboots (reason may be that image that is produced by buildroot is .bin, while image in release 19.07 that actually does boot is .elf, but I have no idea how to make the buildroot produce .elf image).

Also there is no prebuilt binary for this board in snapshots directory (https://downloads.openwrt.org/snapshots/targets/ath79/mikrotik/), which is in itself sort of alarming. Thus, question becomes is there a way to build a snapshot of openwrt for that board that will at least boot?

This is weird, I built it this morning and it was working normally. I uploaded both sysupgrade and initramfs images to https://filetransfer.io/data-package/OHRKOIuE if you want to try them.

You can enable the buzzer (speaker/beeper, however is it called) so that it beeps when a valid image is booted. Sadly, it's the only indicator you'll have.

The .bin initramfs should be OK, it's actually an ELF binary:

binwalk openwrt-ath79-mikrotik-mikrotik_routerboard-922uags-5hpacd-initramfs-kernel.bin 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             ELF, 32-bit MSB MIPS64 executable, MIPS, version 1 (SYSV)
76960         0x12CA0         xz compressed data
77088         0x12D20         xz compressed data
5578608       0x551F70        device tree image (dtb)

This is intended. MikroTik devices with NAND flash also have a small SPI NOR flash where the bootloader, MAC address, license key, etc. live. This memory has very small partitions that can't be written to, because they are smaller than the minimum block erase size the driver uses. To avoid the risk of users breaking their devices accidentally, no images are generated by the buildbot.
See SFP support for MikroTik RB922UAGS-5HPacD - #19 by rogerpueyo

Please give the image I sen you a try, so we can know whether it's the current support status, your device, the image you compiled...

Cheers!

Well your image boots and runs correctly, sysupgrade and all, as one would expect. So the question becomes which .config are you using? Can you share the .config you used to build the binaries so I can try to build one using same config? I have a nagging suspicion that maybe there is something funky with the build host configuration I have...

In terms of users not bricking their devices I understand the intent, but then on the device page it should probably say there is no prebuilt image and why, else ignorant people like myself will keep looking for it all over the place and try booting with wrong images etc etc. :laughing:

To build the image I just ran make config-clean and rm .config .config.old to clean up previous configuration. Then, in make menuconfig I selected:

Subtarget => MikroTik devices
Target Profile => MikroTik RouterBOARD 922UAGS-5HPacD

and ran make. I pasted the contents of the generated .config at https://pastebin.com/NHpMP8Yq, but it shouldn't be different from yours (please note I've got all the feeds installed).

I agree, I'll update it on the wiki when I have some minutes.

I searched a little more. The actual problem is:

commit 9d96b6fb720d9cecc7bd50b4f16dabe1b337f9f2
Author: Adrian Schmutzler <freifunk@adrianschmutzler.de>
Date:   Sun Dec 27 20:33:57 2020 +0100

    ath79/mikrotik: disable building NAND images
    
    The current support for MikroTik NAND-based devices relies on a
    gross hack that packs the kernel into a static YAFFS stub, as the
    stock bootloader only supports booting a YAFFS-encapsulated kernel.
    
    The problem with this approach is that since the kernel partition is
    blindly overwritten without any kind of wear or badblock management
    (due to lack of proper support for YAFFS in OpenWRT), the NAND flash
    is not worn uniformly and eventually badblocks appear, leading to
    unbootable devices.
    
    This issue has been reported here [1] and discussed in more detail
    here [2].
    
    [1] https://forum.openwrt.org/t/rb433-bad-sector-cannot-start-openwrt/71519
    [2] https://github.com/openwrt/openwrt/pull/3026#issuecomment-673597461
    
    Until a proper fix is found (or the stock bootloader supports other
    filesystems), we disable building these images to prevent unknowing
    users from risking their devices.
    
    Thanks to Thibaut Varène for summarizing the details above.
    
    Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>

The table of hardware entry for the device actually has a comment regarding the issue: https://openwrt.org/toh/hwdata/mikrotik/mikrotik_rb922uags-5hpacd

Q:
I can get DDM information from SFP? (rx level, etc) ethtool -m eth1 , for example ?
A: rogerpueyo
So far, I've got this:

root@OpenWrt:/sys/devices/platform/sfp/hwmon/hwmon0# for i in *; do echo "$i:"; cat $i; done
curr1_crit:
90
curr1_crit_alarm:
0
curr1_input:
16
curr1_label:
bias
curr1_lcrit:
1
curr1_lcrit_alarm:
0
curr1_max:
80
curr1_max_alarm:
0
curr1_min:
3
curr1_min_alarm:
0
device:
cat: read error: Is a directory
in0_crit:
3630
in0_crit_alarm:
0
in0_input:
3274
in0_label:
VCC
in0_lcrit:
2970
in0_lcrit_alarm:
0
in0_max:
3470
in0_max_alarm:
0
in0_min:
3130
in0_min_alarm:
0
name:
sfp
of_node:
cat: read error: Is a directory
power1_crit:
1995
power1_crit_alarm:
0
power1_input:
274
power1_label:
TX_power
power1_lcrit:
200
power1_lcrit_alarm:
0
power1_max:
1259
power1_max_alarm:
0
power1_min:
251
power1_min_alarm:
0
power2_crit:
631
power2_crit_alarm:
0
power2_input:
11
power2_label:
RX_power
power2_lcrit:
1
power2_lcrit_alarm:
0
power2_max:
501
power2_max_alarm:
0
power2_min:
1
power2_min_alarm:
0
subsystem:
cat: read error: Is a directory
temp1_crit:
90000
temp1_crit_alarm:
0
temp1_input:
45820
temp1_label:
temperature
temp1_lcrit:
-10000
temp1_lcrit_alarm:
0
temp1_max:
85000
temp1_max_alarm:
0
temp1_min:
-5000
temp1_min_alarm:
0
uevent:
OF_NAME=sfp
OF_FULLNAME=/sfp
OF_COMPATIBLE_0=sff,sfp
OF_COMPATIBLE_N=1

Yes, it works. Thank you.