CeroWrt II - would anyone care?

Community Builds, Projects & Packages
249

An end result that I really wanted was for ISPs to ship better gear, with a commonly maintained, constantly updated, quality OS like OpenWrt.

That was and remains kind of in conflict with some of the goals I've outlined in CeroWrt II so far - I vastly prefer "research" with no pesky end-users to deal with! :slight_smile:

But other goals - like for example, "one click ripe ipv6 spec compliance" do fit. If there was a true reference standard for a quality device, much like how the apple airports were viewed as such - then we could raise the bar for everyone.

Part of that, to me, is that proprietary offloads have got to go, the hardware needs to be designed and supported for a 10 year service lifetime, and so on.

250

Well, that essentially removes the price cap then, a few enthusiast will be willing to spend some money to play with cerowrt II, but even the then the cheaper the easier to participate.

I have a real question though, are these standards actually worth complying to? I see a few things maturing now in the IETF's tsvwg that will end up being official standards in spite of being "utter shite"... Are you sure that is different in IPv6-land? Then again, the best way to figure out whether the standards are worth their salt is to implement them and see how the resulting system behaves.

Yepp, getting enough sufficiently powerful CPUs in a router so they can handle what is thrown at them would be nice, that also limits the choice of CPUs considerably, mid to high end ARM/Intel/AMD with sufficient system and memory bandwidth. (Assuming this aims at at least routing at 1 Gbps rates... I like Jesper's approach from a few years back, when he ran 10Gbps ethernet with minimal sizes frames to really stress the device (and his logic, since most packet mixes are biased to much larger packets, testing with minimal packet size allowed testing routing capability far beyond the 10Gbps link he tested on); but I think he needed a multi socket multi CPU xeon system to pull this off).

Yes for a router that would be nice (side note, my refurbished wndr3700v2 bought initially at Fry's in Burbank (styled as if an UFO had crash landed in the store), to test cerowrt I is still up and running, "demoted" to AP duty but still operational), OR something where more modern replacements that are fully compatible can be expected (again pointing to x86). Something like the APU line only with a few beefy cores instead?

251

Just some random thoughts:

  • battery backed RTC, great
  • internal UPS by default, probably not as useful (globally-) as one might think at first.
    • blown fuse, just push it in, let the router reboot, done.
      the service interruption might not be nice, but at least your modem/ ONT/ switches and desktops will be affected the same, so no real need for the router to be more resilient than the rest of your network (it just needs to be able to reboot properly).
    • the whole street being powered off happens very (very) rarely where I'm located, even if it did happen often enough to consider, I'd then have to maintain power for more than just the router as well (see the blown fuse above). I realize that this situation might be severely different in other regions of the world, but there you do have to think about UPS/ independent generators anyways - and adding the router to the 'safe' circuit wouldn't be much of a burden.
    • some kind of emergency hitting the whole town, be it flooding, snow/ ice, storms, etc.
      Let's face it, even if your own house has a perfect backup, your ISP doesn't. Analogue phone lines -where they still exist- might have been built with battery backups, but have they been serviced within the last couple of decades - modern ones (VoIP/ SIP, cellular) don't even try and will go down quickly (30-60 minutes at best, if your cellular base station and its fibre backhaul routers are equipped with battery backup to begin with) in this case anyways (yes, I've been through a week of that, 2g cellular connectivity was gone within half an hour).
    • having a battery constantly on charge increases the risk of it catching fire, or for the cells to bulge and die over time - these things need regularly service.
  • on a small scale, especially if hardware design and maximizing profits isn't top priority, x86_64 is hard to beat and viable long beyond the half time of many of the past high-end mips/ ARM SOCs (case in point, the almost 10 year old ivy-bridge c1037u can do routing/ NAT and sqm/ cake at 1 GBit/s line speed easily, around 53% CPU usage on one core - without even hitting its maximum clockspeed).
252
1 Like
253

The scope of various sub-projects here is vast. If there is anyone out there that wants to take some of it on, or propose their own thing, both nlnet and the comcast innovation fund have been good sources for funding for me, and applying is very straightforward.

https://nlnet.nl/useroperated/ (due feb 1!!) - usually quite fast in the grant approval process.

And:

https://innovationfund.comcast.com/ (takes 4 months typically)

I am now on a small nlnet grant to deliver some bits described here:
https://nlnet.nl/project/CeroWRT-II/ and some of the work is gated on the other wonderful projects now ongoing. My hope is to be able to stay engaged in the openwrt universe for this entire release cycle as well as continue my advocacy work elsewhere.

Suggestions for other funding sources welcomed.

I'm also heavily involved in attempting to shift US policy and broadband funding towards lower latency, ipv6 enabled networks, most recently with the publication of this bitag report on network latency.

which I hope gains policymaker readership worldwide.

1 Like
254

Back to the original topic, some random things I would like to see:

  • Built-in NAT64/DNS64

IPv4 is here for the indefinite future, yes, new services from bad-at-technology organizations like Discord are still launching IPv4-only, but that doesn’t mean end-user devices have to be IPv4. MacOS has built-in NAT64. It shouldn’t be so hard on OpenWRT.

  • Subnetting, maybe also with mDNS-related adapters

Trash-IoT devices are often IPv4-only, but they don’t have to be on the same subnet as the end-user devices. The problem that comes to mind right away is when those trash devices communicate directly with the end-user device. HomeKit and IPP Everywhere don’t work easily when the trash device is on a separate subnet.

  • Multi-Gb routing and traffic shaping.

Though, at multi-Gb, I think a lot of the bottlenecks are transient and elsewhere in the network.

255

active and honest benchmarking against the best commercial products would be good, especially on our metrics - like latency to multiple stations, under load ( https://www.cs.kau.se/tohojo/airtime-fairness/ )- uptime, crashability, and so on.

eero, for example:

mikrotik, for another:

https://forum.mikrotik.com/viewtopic.php?t=179307

256

mesh protocol interop also.

257

This.

My ISP (Centurylink) can't seem to be bothered to put in anything faster than shitty old copper lines in my rural area. I'm not in a poor country either, I am one of the estimated 40+ million people in the US who do not have access to broadband internet (which considered to be 25 mbps down, 3 mbps up by the US gov).

Centurylink has collected tax dollars for improving infrastructure in rural areas, but they have still failed to meet even just the minimum requirements. - https://arstechnica.com/tech-policy/2020/01/centurylink-frontier-took-fcc-cash-failed-to-deploy-all-required-broadband/

Another thing is, Centurylink has also bought a whole bunch of companies. That money could be far better spent on upgrading rural infrastructure. - https://www.crunchbase.com/search/acquisitions/field/organizations/num_acquisitions/centurylink

"FCC’s latest report (released in January 2021) estimates across the U.S. 14.5 million people don’t have broadband -- and experts believe the actual number could be twice that or more. A BroadbandNow study released in February 2020 estimates that as many as 42 million Americans do not have the ability to purchase broadband internet. While the infrastructure bill dedicates funding for broadband, there’s so much more to do." - https://www.americanconnectionproject.com/

The digital divide is not talked about enough, and it does not just affect third world countries or "poor" people. Often it's spoiled brats who live in big cities have no idea what it's like in rural parts of the US (but yet they'll tell us to just "buy better internet" as if it exists... but it doesn't because ISPs just sit on their ass while collecting our taxes... or they'll just tell us to move but that's far from ideal either)... that is, until they travel out here and experience it for themselves. And you can bet some of those spoiled brats work on websites and fill them to the brim with intrusive advertising and trackers, adding to the pain of slow internet. The modern web is a bloated place.

BTW, without rural communities... huge industries like agriculture would take a hit. So if a bunch of people moves away from rural communities just so they can get faster internet... well you can say hi to higher prices on just about everything.

And it only gets worse in online gaming communities when toxic, spoiled city brats talk shit on people with high ping, ignoring the fact it's not even our fault (in other words, we're not manipulating our ping or cheating)... but it's our shitty ISPs that we basically have little to no power over. A lot of them even think that having a high ping gives us an advantage, but actually it usually doesn't (it just makes our in-game actions arrive late, which of course can hurt the number of kills we get, etc.). Some games will also kick you for high ping (even if it's just a fluctuation for a few seconds) and may even ban you, even if you had spent money on purchase the game in the first place.

258

Elsewhere - see broadband.money and a couple filings with the NTIA so far - I've been trying to get "better routers" on America's radar.

NTIA filing:

Upgrade in place idea. With the supply chains so tight, it would great to make a push for recycled routers:

259

matrix.org has a lot of useful looking stuff. Is there any support in openwrt?

1 Like
260

I just found cerowrt's old credits file, which was on the router, but not on the net. It's http, not https. But we were so amazing, in those 3+ years, and sigh, I'm so much older now. But my humble thanks to so many (that I could identify) that had helped, is here:

http://www.taht.net/~d/credits.html

5 Likes
261

Anything new on this one, guys?

262

far, far, far too distracted by fixing this regression: AQL and the ath10k is *lovely* - #479 by dtaht to be able to move forward in any way.

1 Like
263

Hi Dave @dtaht,

I have more follow-up questions, :thinking: after 6 months:

  1. Do the wifi "client" side drivers include any kind of bufferbloat fixes in Linux, BSD, macOS, iOS, Android. I thought Airtime Fairness (ATF) and Airtime Queue Limits (AQL) are on the wifi "AP" device. Do wifi "client" drivers include fq_codel? How does it work on wifi "clients"?

  2. Looks like Apple Macs mostly use Broadcom WiFI card which use brcmfmac (FullMAC) driver in Linux. Intel/Windows laptops predominantly use Intel WiFI cards. Do make-wifi-fast fixes work on FullMAC drivers like brcmfmac as AP or client in Linux or OpenWrt? How about iwlwifi (is it softMAC or fullMAC ?) as AP or client in Linux or OpenWrt?

  3. What kind of settings / parameters, if possible, can be changed on wifi "client" devices (running Windows 10/11, Linux, BSD, macOS, Android, iOS) to minimize bufferbloat / latency?

1 Like
264

re: 1 - The code from the AP side is fully generalizable down to the station side. ATF is not needed but also not used. AQL acts as a means of keeping a minimal amount of data in the device so fq_codel higher up can mediate. So all the wifi chips supported by these enhancements (ath10k, as are the intel wifi chips, are pretty common), benefit, on linux laptops. I presently use an ath10k, but am thinking of trying out a mt79 soon, on my laptops.

BSD has no support for fq_codel in wifi. I've not benchmarked it. As for windows, we did a bit of benchmarking recently that was not good.

No, these apis don't work on full macs. Broadcom is mostly in the woods still here.

  1. OSX and Ios does have fq_codel but it's pretty high up in the stack above the wifi here, and they've not actually implemented the codel portion of fq_codel when last I looked. More testing of uploads on iphones and osx is happening as a result of the new speedtest.net app for phones, and it's not looking good.

Clientside...

iwlwifi gained support 4? 5? years ago, but as one example, the rest of the intel driver for the ax200 chip (their wifi-6) chip was just awful at the time, I have not tried it since. As always, trying to recruit testers, the ax210 is rumored to be better. Mt79 currently has the most appeal for me on my next laptop. iwl can't be used as AP, usually.

The "ideas" do work on full macs. Gave a talk at broadcom, I think some listened. http://www.taht.net/~d/broadcom_aug9_2018.pdf

The big thing I still change is lowering the airtime allowed in a Be txop both at the AP and in the beacon to 2.5ms.

2 Likes
265

OK, it's been a year since we started speculating about doing a cerowrt II, and we ended up helping ship openwrt 22.03.1 instead. What still seems worth doing - or seems worth doing even more - out of this enormous discussion?

5 Likes
266

love the idea.

1 Like
267

Periodically I pop up hoping more folk have tackled any of the ideas discussed in this enormous thread. Anyone?

2 Likes
268

I started using Yggdrasil. It's an IPv6 overlay network that is fairly configuration free. I used it to get remote access to a device I was setting up at my sister's house, and a machine that runs my wife's camera on her research microscope on campus.

It provides authentication (IPv6 address derived from public key) and security (each connection end to end encrypted, though they don't talk much about the algorithm).

It's experimental but fairly stable now and very useful for accessing a device regardless of where it roams to and/or if it is inside ipv4 private address space. It restores end to end connectivity of the internet in a nice way.