Ath79: support D-Link DAP-2660 - Ethernet not working in Gigabit mode

Hi everyone,

I'm currently working on D-Link DAP-2660, which is similar to DAP-2695 (just with AC1200 and a white plastic case instead of plenum-rated metal).

Both wireless interfaces are working, I have a working factory-flashable image (tested on A2, but should also work for A1; the difference is apparently that A2 need some padding to make the image file bigger...), now the only issue is that I couldn't get the AR8035 Gigabit Ethernet PHY (AR8035) to work with 1000BASE-T, however it will work fine as long as the remote NIC is set to 10 or 100 Mbit/s fixed rate.

The PLL clock values for the GMAC were determined from the bootloader, i.e. entering recovery with different fixed link rates and dumping the corresponding GMAC PLL register values as written by the OEM uboot, as described here for DAP-2360.

// edit 2020-09-09: fixed green led mapping

But still no success :thinking:

Is there anything left to be configured in the .dts that i overlooked?
Does someone own this device and could help testing?

I have enabled wifi by default using dap2660 and dlinktest as ESSID / PSK, to avoid lock-out of the device. Beware that UART is 2.5 V for this device! (two CP2102 dongles died figuring this out :innocent: ) It is however safe to just connect RX and GND to read the bootlog, the 2.5V level is sufficient to be interpreted as high.

The DD-WRT folks seem to have this device supported and working, but not even using a devicetree based target in the first place... :laughing:

It looks to me that your DTS is messed up. What did you use as a base/template?

for example pll-data is not needed in your DTS and probably is the wrong values, it is provided by the included qca955x.dtsi

You said yourself

but your DTS is extremely different than that device

You should be using the values in the DTS for that device as a base for this one

The .dts is indeed redundant in some parts, as it is the result of trying multiple gmac configurations I found with other ath79 devices (e.g. the gmac-config node is probably irrelevant).

While the DAP-2695 is quite similar in terms of the software platform (i.e. flash layout and factory format), the major difference regarding ethernet ports is that it actually has a built-in gigabit switch for the two LAN ports, while DAP-2660 uses a single AR8035 PHY instead. There are other ath79 devices with this phy, but none of the configs worked.

The values for pll-data were extracted from the GMAC registers after being initialized by the OEM u-boot, so I assumed these would also apply to OpenWRT, but I will once again try different values here, especially the one from DAP-2695 (which also explicitly overrides the default one from the .dtsi).

Maybe it's also relevant how to set the rx/tx delays, though I hadn't figured out yet how to find those values, those are probably individual bits from the AR933X_GMAC_REG_ETH_CFG register?

The phy id 4 etc. seems to be correct, also 10/100 Mbit is working flawless. I can post a bootlog later containing the gmac and phy initialisation.

As long as the switch is the built-in switch and not an external chip, the qca955x.dtsi should have the right values. But your board is apparently unique and needs extra gmac config...

I am guessing you need to base your DTS after these boards, which seem to almost have what you're trying to do now

There is no built-in switch with the QCA955x series, the only QCA devices that actually have a gigabit switch built-in are IMHO from the ipq family, while QCA9533, QCA9563 etc. just come with fast ethernet switches (however the GMAC can be connected to an external gigabit phy or switch, e.g. DIR-842).

The devolo devices are actually were I stole the gmac-config snippet from, hoping it might change anything, but it didn't... :innocent:

It's curious that I've had the same problem with DAP-2360 (see the other thread linked above), I'm almost about to try to port this device to ar71xx just to see whether it works :laughing: :sweat_smile:

Currently, this is what DAP-2660 prints on boot regarding the phy:

[    0.418560] libphy: Fixed MDIO Bus: probed
[    0.425467] ag71xx 19000000.eth: invalid MAC address, using random address
[    0.749606] libphy: ag71xx_mdio: probed
[    0.755354] ag71xx 19000000.eth: connected to PHY at mdio.0:04 [uid=004dd072, driver=Atheros 8035 ethernet]
[    0.765802] eth0: Atheros AG71xx at 0xb9000000, irq 4, mode: rgmii
[    0.772275] i2c /dev entries driver
[    0.777462] NET: Registered protocol family 10
[    0.786299] Segment Routing with IPv6
[    0.790163] NET: Registered protocol family 17
[    0.794751] 8021q: 802.1Q VLAN Support v1.8

I'm gonna install DD-WRT now and check the bootlog just to see how they got Gigabit working with this device :sweat_smile:

Yeah you may have the same gmac config as the devolo series, but many of the dts params are different. Maybe just copy it directly?

This is very curious, I installed DD-WRT and it will initialize the GMAC PLL register for 1000BASE-T mode as 0x8f000000 (rather than 0x56000000 as in DAP-2695, 0x16000000 as in the qca955x dtsi, 0xae000000 as in the devolo dts or 0x96000000 which I had initially dumped from DAP-2660 uboot). Does DD-WRT alter the CPU frequency? :thinking:

`### PLL
root@DD-WRT:~# ./io -4 -l 0x50 0x18050000
18050000:  00021480 00213c00 01010000 00000400
18050010:  c00a07fe 000a0000 00000014 00000000
18050020:  00000520 000a03ac 8f000000 000010b3
18050030:  0a47f028 00004000 0523f828 41020ccc
18050040:  781003ff 003c103f 03000101 00000007

### GMAC
root@DD-WRT:~# ./io -4 -l 0x58 0x18070000
18070000:  00001001 00000000 0000000f 00000002
18070010:  0000ffff 00000000 1f018116 00001140
18070020:  00000049 00000060 00000040 00000000
18070030:  00000000 00000000 00000801 0000000e
18070040:  00000000 00000000 00000020 2396e54a
18070050:  04e29c45 00000000`

Even more curious: with DD-WRT the device will only work with gigabit, the link will also synchronize to 100Base-T but with no connectivity at all :sweat_smile: (the same problem as I have, just the other way round)

I'll do some more testing tomorrow :smile:

who knows what DD-wrt does, I don't trust their hackish kernel

Interesting, gigabit mode works when I set

pll-data = <0x8f000000 0x80000101 0x80001313>;

but not 100Base-T, so I set the latter values to the ones from the .dtsi:

pll-data = <0x8f000000 0x00000101 0x80001616>; also 100M is working, but not quite stable, sometimes this happens:

[   68.867323] ------------[ cut here ]------------
[   68.872050] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:448 dev_watchdo                                                                g+0x26c/0x274
[   68.880444] NETDEV WATCHDOG: eth0 (ag71xx): transmit queue 0 timed out
[   68.887066] Modules linked in: ath9k ath9k_common pppoe ppp_async iptable_nat                                                                 ath9k_hw ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt                                                                _MASQUERADE xt_FLOWOFFLOAD pppox ppp_generic nf_nat nf_flow_table_hw nf_flow_tab                                                                le nf_conntrack_rtcache nf_conntrack mac80211 ipt_REJECT cfg80211 xt_time xt_tcp                                                                udp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG slhc nf_rej                                                                ect_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 iptable_mangle iptable_filter                                                                 ip_tables crc_ccitt compat ledtrig_heartbeat nf_log_ipv6 nf_log_common ip6table                                                                _mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 gpio_butt                                                                on_hotplug
[   68.944595] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.52 #0
[   68.950432] Stack : 80650000 805f0038 00000000 00000000 805ef208 87c0bde4 806                                                                2615c 80625d43
[   68.958918]         80590c8c 00000000 807832d8 80630000 00200000 00000001 87c                                                                0bd98 9c226390
[   68.967400]         00000000 00000000 807b0000 000000c1 61696e74 00000000 2e3                                                                42e35 32202330
[   68.975883]         000000c1 2a1635c2 00000000 000e69d3 00000000 00000009 000                                                                00000 803dea3c
[   68.984366]         00000009 80630000 00200000 00000122 00000000 802f91a8 000                                                                00000 80780000
[   68.992851]         ...
[   68.995339] Call Trace:
[   68.997835] [<80069934>] show_stack+0x30/0x100
[   69.002355] [<800825d4>] __warn+0xc0/0x10c
[   69.006514] [<800826ac>] warn_slowpath_fmt+0x8c/0xac
[   69.011570] [<803dea3c>] dev_watchdog+0x26c/0x274
[   69.016362] [<800c42dc>] call_timer_fn.isra.34+0x20/0x90
[   69.021757] [<800c44f4>] run_timer_softirq+0x1a8/0x1f4
[   69.026987] [<80507fb0>] __do_softirq+0x110/0x298
[   69.031764] [<80298fc8>] plat_irq_dispatch+0x94/0xe4
[   69.036805] [<80065118>] handle_int+0x138/0x144
[   69.041406] [<80507d7c>] r4k_wait_irqoff+0x18/0x24
[   69.046274] ---[ end trace c2deb1844c7ca2c5 ]---

so apparently neither 0x80000101 nor 0x00000101 seems to be the right gmac pll setting for 100 Mbit :thinking: still wondering how to figure these out, when I can't trust the config written by the bootloader...

Maybe time to look at datasheets lol

I don't know if this make a difference, but are you testing that with hardware limited to 100base-t or are you trying to set it with software.

Do you know the bootloader's config? and how?

Indeed I've refrained from trusting the ubuntu network manager about this by now, and rather put an old 10/100 switch in between, and it was actually working with 100Mbit initially.

So, not quite sure if the problem was related to the PLL clock, or just me trying to bridge the wireless STA interface to LAN (to have it connected both to the internet as well as letting me access the device over WiFi, to see what happens when messing around with the ethernet port)… since it's so annyoing to change ip configuration on both ends all the time :pensive:

I should probably try iperf once again :slightly_smiling_face:

For qca955x the MII PLL register is at 0x18050028, so you could just run
md 0x18050028 1
from the uboot shell for each link speed, as described in the DAP-2360 thread linked above.

Ethernet is working fine now for Gigabit and 100BASE-T, but at 10BASE-T the connection is unstable (with retransmissions), however this also applies to the OEM firmware (as well as DD-WRT, which only supports Gigabit).