Divested-WRT: No-nonsense hardened builds for Linksys WRT series

Hi @SkewedZeppelin I see you just created a new build, and I was trying to replicate it when I realized the "config" file is missing in both "/latest" as well as " /20211101-00" folders. Is that accidental, or the build procedure changed and that's no longer needed?

Thanks in advance,

@wally_walrus
Managed to forget it. It is up now. Thanks!

I tested the multi dsa patch on caiman and mine shows interrupts on both eth0 and eth1

           CPU0       CPU1
.........
 45:       7194        445      MPIC   8 Level     eth0
 46:       8249       1137      MPIC  12 Level     eth1
.........

Offtopic:
Did someone checkout the new qosify package?
I have it running for a day now. Seems quite good?
Expect that it doesn't autostart at boot.

//edit
There is not that much of a difference in CPU utilisation.
Most interrupts still land on CPU0.

Interrupts for me are always pegged to CPU0 (mamba, rango). Been wondering as to any potential differential of performance / throughput RE. round-robin vs assigning user ports to particular CPU port.

Built and flashed qosify yesterday but have not gotten around to configuring and running quite yet.

1 Like

By default the patch maps one CPU to the lan ports and the other CPU the wan port.
So I expected that the interrupts would have been spread more evenly between the CPUs.

I guess the problem is still the mvneta driver? And the hardware queue mapping...?

Yes, the reason I haven't keep those patches is that I didn't see any actually benefit.
Something is missing.

Hi @SkewedZeppelin the master snapshot from a few days ago shows:

selinux-policy: update to version 1.0

Any chance you could look at this again?

TIA

@wally_walrus

Yes, I've been watching it.
I'll give it a try soon, but I still think there are some bits missing.

Hi @SkewedZeppelin, putting more effort in trying to replicate your builds and came across the following:

  • On Oct 27 I built an image but did not include 4724.patch since it was a "cherrypick" and not in your regular /patches folder, and found my build was made for kernel 4.10.75 (your build was already at 4.10.76)

  • Today I built again using the latest repo and 4724.patch is again NOT in your /latest folder, but I now ended up with 4.10.76

Question - can I conclude from the above that the repo master snapshot has now incorporated the patch thus kernel is at 4.10.76?

TIA

@wally_walrus

the number on the patch is the pull request ID on the OpenWrt GitHub:

You can see it was picked into master per the last comment.
The next build I do will likely include

as you see it is still open, not closed meaning it isn't included yet.

2 Likes

what is qosify please

readme

2 Likes

@SkewedZeppelin I downloaded your Multi-CPU image (thank you) however I found my upload was stuck at 89Mbps :frowning: on my WRT1900ACSv2

Looks like I need to either add kmod-sched or remove a commit as I previously found here -

I will build my own multi-CPU image using your config as a base however I am not sure what I need to do with the various patches - are you able to advise please?

Appreciate your and everyone else's time.

@oli - what hardware are you running or do you have your own build? As it appears you're not affected by this upload limit as I... :confused:

Cheers. :beers:

Edit: I should have read the thread more thoroughly as I missed this - seems I should wait

wrt1900acs v2 I gave all the details above!

I'll try to backport couple fixes in the mvneta. I do believe...every fix will improve our immune response. Thanks @SkewedZeppelin!

This one implement basic mqprio support.

--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -102,6 +102,8 @@
 #define      MVNETA_TX_NO_DATA_SWAP              BIT(5)
 #define      MVNETA_DESC_SWAP                    BIT(6)
 #define      MVNETA_TX_BRST_SZ_MASK(burst)       ((burst) << 22)
+#define        MVNETA_VLAN_PRIO_TO_RXQ                  0x2440
+#define      MVNETA_VLAN_PRIO_RXQ_MAP(prio, rxq) ((rxq) << ((prio) * 3))
 #define MVNETA_PORT_STATUS                       0x2444
 #define      MVNETA_TX_IN_PRGRS                  BIT(0)
 #define      MVNETA_TX_FIFO_EMPTY                BIT(8)
@@ -490,6 +492,7 @@
        u8 mcast_count[256];
        u16 tx_ring_size;
        u16 rx_ring_size;
+       u8 prio_tc_map[8];

        phy_interface_t phy_interface;
        struct device_node *dn;
@@ -4903,6 +4910,63 @@
        return phylink_ethtool_set_eee(pp->phylink, eee);
 }

+static void mvneta_clear_rx_prio_map(struct mvneta_port *pp)
+{
+       mvreg_write(pp, MVNETA_VLAN_PRIO_TO_RXQ, 0);
+}
+
+static void mvneta_setup_rx_prio_map(struct mvneta_port *pp)
+{
+       u32 val = 0;
+       int i;
+
+       for (i = 0; i < rxq_number; i++)
+               val |= MVNETA_VLAN_PRIO_RXQ_MAP(i, pp->prio_tc_map[i]);
+
+       mvreg_write(pp, MVNETA_VLAN_PRIO_TO_RXQ, val);
+}
+
+static int mvneta_setup_mqprio(struct net_device *dev,
+                              struct tc_mqprio_qopt *qopt)
+{
+       struct mvneta_port *pp = netdev_priv(dev);
+       u8 num_tc;
+       int i;
+
+       qopt->hw = TC_MQPRIO_HW_OFFLOAD_TCS;
+       num_tc = qopt->num_tc;
+
+       if (num_tc > rxq_number)
+               return -EINVAL;
+
+       if (!num_tc) {
+               mvneta_clear_rx_prio_map(pp);
+               netdev_reset_tc(dev);
+               return 0;
+       }
+
+       memcpy(pp->prio_tc_map, qopt->prio_tc_map, sizeof(pp->prio_tc_map));
+
+       mvneta_setup_rx_prio_map(pp);
+
+       netdev_set_num_tc(dev, qopt->num_tc);
+       for (i = 0; i < qopt->num_tc; i++)
+               netdev_set_tc_queue(dev, i, qopt->count[i], qopt->offset[i]);
+
+       return 0;
+}
+
+static int mvneta_setup_tc(struct net_device *dev, enum tc_setup_type type,
+                          void *type_data)
+{
+       switch (type) {
+       case TC_SETUP_QDISC_MQPRIO:
+               return mvneta_setup_mqprio(dev, type_data);
+       default:
+               return -EOPNOTSUPP;
+       }
+}
+
 static const struct net_device_ops mvneta_netdev_ops = {
        .ndo_open            = mvneta_open,
        .ndo_stop            = mvneta_stop,
@@ -4915,6 +4979,7 @@
        .ndo_do_ioctl        = mvneta_ioctl,
        .ndo_bpf             = mvneta_xdp,
        .ndo_xdp_xmit        = mvneta_xdp_xmit,
+       .ndo_setup_tc        = mvneta_setup_tc,
 };

 static const struct ethtool_ops mvneta_eth_tool_ops = {

Since moving to this build, I've got a problematic wifi client that gets stuck in disconnect, and then blacklists the router bssid.
Its a linux based client and I can see the wpa_supplicant logs at the client side. But it looks like openwrt's hostapd has been compiled with a lot of the lower level debug removed to reduce size. I don't see much of any management level messages at the router logs (and I moved to verbose debugging).

The client 'initially' connected and was running for some time (I don't know the conditions here, but there is some stickiness once things break). After some time, and possibly from power off/disconnects reconnects, the client can't get back on the network. I've tried clearing blacklist, disabling and renabling the network (client), rescanning, resetting wpa_supplicant, rebooting the client etc, nothing works, as it just quickly moves right back to disabled and blacklisted.

The router is sending ASSOC-REJECT (12) status code =1 to the client. I wish I could see the router side of the exchange in logging.

I've got a sanitized snippet of wpa_supplicant log below, just after the supplicant finds its ssid in scan and tries to connect.. Does anyone have any ideas?

1634772434.261811:   * freq_hint=2412
1634772434.261837:   * SSID - hexdump_ascii(len=6):
    6d 79 73 73 69 64                                       myssid            
1634772434.261907:   * IEs - hexdump(len=31): 30 14 01 00 00 0f ac 04 01 00 00 0f ac 04 01 00 00 0f ac 02 00 00 3b 07 51 51 73 76 79 7c 7d
1634772434.262011:   * WPA Versions 0x2
1634772434.262035:   * pairwise=0xfac04
1634772434.262056:   * group=0xfac04
1634772434.262077:   * akm=0xfac02
1634772434.262102:   * Auth Type 0
1634772434.277179: nl80211: Connect request send successfully
1634772434.277339: wlan0: Setting authentication timeout: 10 sec 0 usec
1634772434.277441: EAPOL: External notification - EAP success=0
1634772434.277494: EAPOL: External notification - EAP fail=0
1634772434.277524: EAPOL: External notification - portControl=Auto
1634772434.465961: nl80211: Event message available
1634772434.466289: nl80211: Drv Event 46 (NL80211_CMD_CONNECT) received for wlan0
1634772434.466356: nl80211: Connect event (status=1 ignore_next_local_disconnect=0)
1634772434.466430: wlan0: Event ASSOC_REJECT (12) received
1634772434.466533: wlan0: CTRL-EVENT-ASSOC-REJECT bssid=aa:bb:cc:dd:ee:ff status_code=1
1634772434.466619: wlan0: Radio work 'connect'@0x1517b0 done in 0.210513 seconds
1634772434.466674: wlan0: radio_work_free('connect'@0x1517b0): num_active_works --> 0
1634772434.466719: Added BSSID aa:bb:cc:dd:ee:ff into blacklist
1634772434.466755: Continuous association failures - consider temporary network disabling
1634772434.466908: wlan0: CTRL-EVENT-SSID-TEMP-DISABLED id=0 ssid="myssid" auth_failures=3 duration=30 reason=CONN_FAILED
1634772434.466972: wlan0: Blacklist count 29 --> request scan in 10000 ms

Just curious "since moving to this build" from what build? OEM, Master Snapshot, 21.02.1, 19.07.8, etc.?

1 Like

@phinn,

I moved from DavidC's last build. The client worked fine on that, but obviously there were a billion changes between 19.x and 21.02. Could still be the client doing something wrong that may have been tolerated in 19.x, but I need to figure out what the router doesn't like.

Are you familiar with this discussion? A few brave people are trying to solve/find a workaround for the wifi issues for WRT3200ACM/WRT32X.

They've currently been able to track down the problem and remedy the issue by replacing the current mac80211, with an older one (5.7.5), in a 21.02.1 build. If it works for master as well, maybe this is something we could try out here too?

2 Likes

I've updated to the latest build from the last David built on my WRT1900AC and the fan appears to be stuck on maximum. Any ideas?

It seems like there is a pwm fan control package but I think the kernel is too new to install.