GRX500 ethernet bring-up on Fritz!Box 7560: CBM DQM output loses data_len - ideas?

For Developers
1

Hi all,

I'm working on an OpenWrt 5.10 ethernet driver for the AVM Fritz!Box 7560 (Intel Lantiq GRX500 SoC, DMA silicon V2.2) and am stuck on a very specific hardware-level blocker. I've narrowed the problem down thoroughly but can't crack the last step, so I'm hoping someone familiar with CBM/CQM internals can point me in the right direction.

What's working

The full TX pipeline is visibly functional end-to-end:

  • PHY: link up at 1 Gbps on 4 ports (GPHY firmware preserved via DTS reserved-memory).

  • CBM buffer alloc: FSQM_OFSC drains from 2047 as expected, buffers are allocated from SBA pool via EQM_CPU_NEW_SPTR.

  • CPU enqueue: EQM CPU port 0 accepts every descriptor; EQPC increments, OWN bit flipped from 0 to 1 in the readback.

  • TMU: dispatches from queue 16 to DQM port 6, dqpc6 increments 1:1 with enqueues.

  • CBM DQM: writes output descriptor into the DMA ring at 0x1E541000, alternating slot 0 and slot 1.

  • DMA2TX: CIS=0xe (EOP | DUR | DESCPT), cdptnr advances between 0x1E541000 and 0x1E541008 (slot 0 / slot 1 on the V2.2 2-DW stride).

The blocker

CBM zeros the 16-bit data_len field in DW3 when it writes the DQM output descriptor.

Example from one ping attempt:


CPU wrote : DW3 = 0x3100008A (SOP | EOP | byte_offset=2 | data_len=0x8A = 138 bytes)

CBM DQM output : DW3 = 0x31000000 (same SOP | EOP | byte_offset, data_len=0)

Every other bit propagates correctly. Only data_len is zeroed. The DMA consequently transfers 0 bytes per descriptor. GSWIP RMON TX counters stay at 0 across ports 0..10 even after dozens of TX attempts. ping fails 100%.

What's been ruled out

  • V2.2 vs V3.0 DMA silicon differences - handled (no writes to V3-only registers like DMA_CPDCNT / CH_POLL_EN / DMA_C_DP_TICK).

  • DMA_CDBA = 0x1E541000 pointing into CBM's DMADESC SRAM region - confirmed correct vs AVM's setup_DMA_channel() at cbm.c:4416-4441.

  • CPU-side MMIO write order (DW0 @ +0, DW1 @ +4, data_ptr @ +8, DW3 @ +12) - tried the BE-swapped variant and CBM refused every enqueue. Linear order is correct.

  • OWN=0 on input - confirmed convention (CBM flips to 1 on accept).

  • CBM_FIXED_RX_OFFSET = 194 data placement - byte_offset propagates correctly after this.

  • FDMA_PCTRL / SDMA_PCTRL / BM_PCFG on GSWIP internal ports 0..15 (not just 0..5).

  • Peer IP / ARP sanity.

What I've tried that regressed

  • MSEL=1 byte mode on CBM_EQM_CTRL and CBM_DQM_CTRL (AVM's cbm_counter_mode_set from datapath_misc.c:415-416). CBM still accepts, but TMU stops dispatching - queue 16 accumulates, dqpc6=0. MSEL=1 apparently requires additional configuration I haven't found.

The question

Where does CBM on V2.2 silicon actually propagate packet length from CPU enqueue to the DMA engine?

Hypotheses I haven't ruled out:

  1. CBM stores per-buffer length in an internal metadata table separate from the descriptor rings, and our CPU TX path needs to update it explicitly.

  2. TMU holds per-queue frame length state and the DMA2TX reads length from a TMU sideband (FPCR / frame-size-per-port register) rather than from the DQM descriptor.

  3. Some CBM register we haven't written selects between "propagate descriptor data_len" and "use fixed pool frame size" modes.

  4. On V2.2 silicon specifically, data_len lives at different DW3 bit positions than on V3.0, and our reads are at the wrong offset.

Anyone who has worked on CBM internals or successfully routed a packet CPU -> CBM -> GSWIP on a V2.2 GRX500 would save me a lot of time if they could point at the actual path length takes.

I've kept a detailed changelog of every version (v39 through v99) with register readbacks and decision trees if anyone wants to dig in - happy to share.

Thanks in advance!

2

Update:
data_len=0 in the DQM output descriptor is normal — length rides the TMU PPT2.BDYL sideband, not DW3. Confirmed by reading back a working DQM descriptor and it looks the same.

New blocker, same driver:

TMU queue 16 accumulates packets (q16 grows with each TX, eqpc advances, PPT2.BDYL holds the right length), but the scheduler never dispatches: dqpc6 = 0 permanently. Everything I can find that should gate dispatch is set:

QTHT[16][0]  = 0x80011320   (QE=1)
SBIT[48][0]  = 0x80010010   (SIE=1, IWGT=1, QSID=16)
SBOT[6][0]   = 0x80000006   (SOE=1, V=EPN, OMID=6)
EPMT[6]      = 0x80000006   (EPE=1, SBID=6)
TMU_CTRL     = 0x0000FF11   (MAXTB=0xFF, RPS=1, ACT=1)
CFG_DMA_EGP_6 = 0x00060101  (DQREQ | DQPCEN | EPMAP=6)
CBM_DQM_CTRL = 0x15, CBM_EQM_CTRL = 0x755
CFGCMD EP_ON|6 and SB_INPUT_ON|48 both returned VAL=1.

DMA2TX chan 1 is idle and healthy (CIS=0x04, waiting for descriptors that never come).

On GRX500 V2.2, what else is needed to make the TMU scheduler actually start dispatching from a queue once all enable bits are set?

Some per-port ready signal from CBM DQM back to TMU? A token-bucket init beyond TBE=0 / TBID=255? A global kick register? I've walked tmu_basic_init and init_cbm_dqm_dma_port in the AVM GPL dump and think I'm matching, but clearly something's missing.

3

The TMU-dispatch issue turned out to be a red herring. The real blocker: CBM rejects every CPU enqueue with disc_reason=8 (CFG_CPU_IGP_n bits [19:16]). ~40 hypotheses ruled out, compared across the six GPL trees on disk (AVM 07.11/07.30, BTHub6B, D7000v2, RAX40v1) and a cross-AVM-version diff.

What I've found since:

  • OWN-bit probe. Live EQM_CPU_DESC0+12 reads OWN=1 (HW released the slot back to CPU); the DISC_DESC3 latch — which captures the rejected descriptor — mirrors the as-fed bytes with OWN=0. So the reject is post-fetch, not pre-fetch classifier.
  • Inter-write CFG.DISC readback. Between each of our DW0/DW1/DW2/DW3 writes, cfg.disc=0. ~500 µs after DW3, it flips to 8. HW seems to evaluate the whole descriptor as a unit, asynchronously — no specific write triggers the reject.
  • DP-API call-chain audit. Every HW register write reachable from dp_register_subif/dev/alloc → cbm_dp_enable → queue_map_set → cbm_qidt_set is already replicated byte-for-byte in our v5.10 driver.

My best read at this point: the gate is in non-MMIO HW state — a CBM micro-controller, internal SRAM tables, or an undocumented "ready" flag set via a path not exposed in any of the six GPL trees.

Has anyone seen CFG_CPU_IGP_n.DISC=8 post-fetch on a GRX350 with otherwise-byte-equivalent register state?
Any known non-MMIO CBM init step (mailbox, SRAM scrub, micro-controller firmware load, …)?

4

Bit more general:

These are probably for older kernels (3.1, 4.9)
Be aware of potential difference between
PXB4395 AVM 7560 (GRX350, HW221)) (+ BTHub6B, D7000v2, RAX40v1)
and
PXB4583 AVM7590 (GRX550, HW226) (+7530, +7580, + K3C)

Do you have kernel 4.9 (or 3.10) running on the 7560 with openwrt with minimal patches at first? Please share. And then bump it up.

This is not for acceleration, is it?

5

Hi talos and jmv20101,

I've been following this thread with great interest — I also have a
Fritz!Box 7560 (GRX350, HW221) and I'm trying to get OpenWrt running on it.

My device details:

  • HWRevision: 221
  • bootloaderVersion: 1.3082
  • firmware_version: 1und1
  • memsize: 0x10000000 (256MB)
  • firstfreeaddress: 0x8523CBD0

I built an OpenWrt image targeting xrx200 but EVA returns
"553 Execution failed" — which confirms the xrx200/VR9 kernel
simply won't boot on GRX350.

talos: do you have a kernel 4.9 or any test image that actually
boots on the 7560? I'd be happy to test and provide UART output.
I have UART access available.

jmv20101: the greenwave/C4000XG assessment is very helpful,
thanks for sharing.

Any help appreciated!

6

This is to a 7560 also:

┌──(kali㉿kali)-[~/zram0]
└─$ ./eva_ramboot.py  192.168.178.1 K3C/bin/lantiq/openwrt-lantiq-xrx500-easy350_anywan_800m_owrt-uImage-initramfs

SETENV memsize 0x07bd8000
< 200 SETENV command successful
SETENV kernel_args_tmp mtdram1=0x87bd8000,0x88000000
< 200 SETENV command successful
MEDIA SDRAM
< 200 Media set to MEDIA_SDRAM
Traceback (most recent call last):
File "/home/kali/zram0/./eva_ramboot.py", line 42, in 
ftp.storbinary('STOR 0x%08x 0x88000000'%(haddr), img)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.13/ftplib.py", line 503, in storbinary
return self.voidresp()
~~~~~~~~~~~~~^^
File "/usr/lib/python3.13/ftplib.py", line 259, in voidresp
resp = self.getresp()
File "/usr/lib/python3.13/ftplib.py", line 254, in getresp
raise error_perm(resp)
ftplib.error_perm: 553 Execution failed.

but maybe this is using the wrong kernel choice in the K3C build.