Effect of "Set tcp_ecn to off" on ECN

How to disable ECN for cake on egress?

 tc -s -d qdisc show dev wan
qdisc cake 8012: root refcnt 2 bandwidth 36Mbit diffserv4 dual-srchost nat wash no-ack-filter no-split-gso rtt 100.0ms noatm overhead 0 mpu 64
 Sent 268496474 bytes 730574 pkt (dropped 430, overlimits 455600 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 4195968b of 4Mb
 capacity estimate: 36Mbit
 min/max network layer size:           40 /    1500
 min/max overhead-adjusted size:       64 /    1500
 average network hdr offset:           14

                   Bulk  Best Effort        Video        Voice
  thresh       2250Kbit       36Mbit       18Mbit        9Mbit
  target          8.1ms        5.0ms        5.0ms        5.0ms
  interval      103.1ms      100.0ms      100.0ms      100.0ms
  pk_delay        782us      658.3ms      377.1ms         73us
  av_delay         77us      654.4ms      176.5ms         19us
  sp_delay          3us        204us          7us          2us
  backlog            0b           0b           0b           0b
  pkts             9088       234208       442779        44929
  bytes          880615     86099850    175897581      6178735
  way_inds            0          262            0            0
  way_miss           13          553          432          267
  way_cols            0            0            0            0
  drops               0           33          397            0
  marks               0        30165        56978            0
  ack_drop            0            0            0            0
  sp_flows            0            7           10            0
  bk_flows            0            0            0            0
  un_flows            0            0            0            0
  max_len          1494         1514         1514         1392
  quantum           300         1098          549          300

qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 2416945301 bytes 1733446 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

the pk_delay/av_delay seems quite high for me?

Yes, the number seem too high, no idea why though. Cake does not allow to disable ECN at the moment, I believe the idea was that ECN needs to be negotiated anyways, one could simply disallow ECN at the endpoints inside one's own network. I have also at times desired a simpler one-step solution...
You can always try to use TC to set the ECN bitfield of egress packets to all zeros, though.

Also the memory usage:
memory used: 4195968b of 4Mb

Mmh, that is just a bit north of 4 * 1024 * 1024, so probably within the 4 MB limit, but I think that normally that should be lower if you are not actively stressing your link.