Project idea: Open source network controller (Looking for feedback)

Community Builds, Projects & Packages
1

Hello :slight_smile:

I've been toying with the idea of building an open source network controller to manage OpenWrt devices in my house, basically an open source Ubiquity UniFi Controller or TP Link Armada Controller including iOS/Android app for full network management and monitoring.

I'm aware of and have tried OpenWISP but found it relatively cumbersome to setup and use, and quite low level and intricate compared to UniFi/Armada. I'm imagining building something much more high level, simple and easy to use.

I'd like to know if others would find something like this useful and interesting? If yes/no, please share why.

Your feedback and suggestions are much appreciated, thanks!

1 Like
2

I'd find that useful and interesting. I've considered OpenWISP for about 5 minutes when I learned about it and then realized the amount of work would involved in setting OpenWISP up and then discovering/maintaining templates. I've recently started using multiple routers as dumb AP's since I recently I moved to another house.

I set the main router and the dumb AP's up using a shell script with UCI commands. If I need to achieve something UCI can't provide, I use HERE documents or sed to get the job done. Currently I have separate shell scripts since I'm not using exactly the same hardware, I might put that all in one single shell script someday.

So setting up devices is one thing I'd be interested in, but also monitoring of wireless clients. On which AP are clients connected? How's the roaming history? I've currently got DAWN running in all AP's, all AP's also have the Luci-app for DAWN installed but it's not really intuitive to see in a simple overview (graphically) on which AP and radio/channel a client is connected, it's basically a big list of SSID's and MAC addresses with percentages. I need to stare at it for minutes to understand what's going on.

What's also something I'd be very interested in, is SQM usage; when getting SQM right, it involves knowledge of different qdiscs and how to interpret the values the tc -s qdisc command produces.

I'm not a developer, but what I would consider for such a project is to keep it simple and reuse existing techniques to get the job done. Would you consider UCI via SSH, or Ansible for instance? What would be required to run the controller itself? Is it possible to create it so lightweight it might run even on a OpenWRT router itself with 2 cores and at least 512MB of RAM for instance? Or do you need at least 2GB of RAM and 3GB of disk space for the controller software to run? Is it intended to run inside the home network, or is it (also) offered as a cloud service? I'd prefer to be fully in control of my own home network, no spying eyes, no cloud services that need monthly fees...

These are just some ramblings, don't know if this is what you're looking for?

1 Like
3

Thanks for the detailed reply, this is very useful.

So setting up devices is one thing I'd be interested in

Sounds great, this is the first use-case I'm working towards currently. A very simple declarative provisioning system.

but also monitoring of wireless clients. On which AP are clients connected? How's the roaming history? I've currently got DAWN running in all AP's, all AP's also have the Luci-app for DAWN installed but it's not really intuitive to see in a simple overview (graphically) on which AP and radio/channel a client is connected, it's basically a big list of SSID's and MAC addresses with percentages. I need to stare at it for minutes to understand what's going on.

I think this would certainly be a very useful feature, and shouldn't be too hard to implement.

What's also something I'd be very interested in, is SQM usage; when getting SQM right, it involves knowledge of different qdiscs and how to interpret the values the tc -s qdisc command produces.

This is an interesting case which I've not considered before, could most certainly implement this.

I'm not a developer, but what I would consider for such a project is to keep it simple and reuse existing techniques to get the job done. Would you consider UCI via SSH, or Ansible for instance? What would be required to run the controller itself? Is it possible to create it so lightweight it might run even on a OpenWRT router itself with 2 cores and at least 512MB of RAM for instance? Or do you need at least 2GB of RAM and 3GB of disk space for the controller software to run? Is it intended to run inside the home network, or is it (also) offered as a cloud service? I'd prefer to be fully in control of my own home network, no spying eyes, no cloud services that need monthly fees...

I'm planning to make the controller as simple and lightweight as possible at least initially. I'll most probably start with UCI over SSH as an initial provisioning mechanism, but could also look at re-purposing the OpenWISP backend for OpenWRT. I think at least initially, resource requirements would be very small, probably run-able on single core with >=256MB RAM, probably <100MB disk space so the controller could possibly run on larger OpenWRT devices, and other platforms like Pi's etc. Perhaps down the line I could offer some sort of cloud service, similar to UniFi, and charge a fair fee for that, but the software would otherwise be completely open source and self host-able.

These are just some ramblings, don't know if this is what you're looking for?

This is exactly the kind of feedback I am looking for, thank you very much!

1 Like
4

I think this would be awesome, and I hope this project happens...

but... I think that the size/scale/scope of the work involved to bring this to fruition has likely been vastly underestimated. A lot of this depends on the features that would be part of this system, how the user would interact with it, and what the provisioning system looks like.

So, for example:

  • building a GUI is complex, especially if the GUI will operate anything like Unifi/Omada (setting aside the UI visual design, just the building functional elements alone can be a big job).
  • Reading/writing the configuration is an obvious requirement, but other telemetry/statistics add many layers of complexity in terms of gathering and presenting the statistics.
  • If the system would work with a defined and limited set of devices, this would be challenging in itself, but doable. But if all devices supported by a specific OpenWrt version can be used, you have to have some way to understand the nuances of each one... for example, the number of physical ports and physical radios; how things are connected together (for example, a single 5-port switch inside a device vs a multi-port device that has individually routed ports with no switch chip), the capabilities of each chipset (how many SSIDs does a given wifi chip support, how many VLANs and what VLAN IDs can be used with each specific switch IC), PoE capabilities, and even things like the fact that there is currently a mix of DSA and swconfig based systems running on OpenWrt 21.02. You'd have to figure out if the per-device details come from a library/database of devices, or if this would be parsed and semantically understood by the system after reading stuff from the devices themselves).
  • There needs to be logic to keep track of globals in a network topology (VLANs, SSIDs, various profiles, etc.) and then it has to trickle down to the appropriate configuration elements for each subsystem (routing, firewall, switch ports, routed ports, radios, and so on).

Don't forget that Unifi (love it or hate it) has been developed over many years. It still has missing features, bugs, and other quirks, but it does generally work for most things. Omada has also been in development for a while, but really benefited from the work Ubiquiti had done since the latest version basically just copied the UI and feature set from Unifi. And both of these companies (and all the others who do enterprise single-pane-of-glass management systems) are for-profit ventures that have large teams of people working on this.

IMO, the best way to approach this is to start small... maybe only support a few devices in the initial version, keep the features limited to reading/writing the configuration of each device and presenting it in some simplified interface (either a simple GUI or a CLI based system). You'll find that even managing say 3 devices (2 APs and a router) from a singular interface will present a few challenges (for example, how do you present the user with a UI that helps setup, as an example, several routed networks and then setup switch ports and APs (SSID+password and VLAN associations) as a global constructs, while also allowing the radio power and channel to be set individually per AP and switch ports to be configured on a per-port basis.

This could be a really cool project, but it is way more complex than it seems at first glance, so start simple and build from there.

5

It's just a suggestion, but I think you could maybe talk to the OpenWISP people about your ideas and how they could become a part of OpenWISP project, might be a bit easier than building your own whole new thing from scratch by yourself. With OpenWISP you already have NetJSONconfig, scalability (OpenWISP recently got some Celery scheduling work done that makes managing even thousands of devices very smooth and responsive), and you can always put your own GUI built with whatever framework you want on top of Django.

6

I actually have built this system, though it is not open source, but a commercial product. We have been selling a variant of it that runs on FreeBSD for years now.

As I write this, we are 11 days from a product release of a completely new version, written completely in C, that is hosted on Openwrt linux.

I will tell you that I personally started the new C version something over 3 years ago, and for the last 12 months, one and sometimes two other people have been working on it with me. Until about 8 months ago I was not working on it full time, but since then I have been doing it full time.

It is a very involved project; not at all trivial. Our new product is built around a multithreaded, multiprocess controller, and is targeted at a specific market that can use it and that won't notice the features that are not yet fully implemented in the C version (the old FreeBSD product runs mostly in PHP).

So, I wish you luck doing this, but don't think it will be simple.

1 Like
7

I've been working on DAWN's core code, and equally find staring at the MAC address heavy view of what is happening rather tiring on the eyes.

I'm not traditionally a UI developer, but might take a look at DAWN-luci next. I'm initially thinking of focusssing on a refresh that is useful for info purposes on the "main" AP where DHCP correlation of MAC to IP to hostname is easily available. I've pondered whether it might be useful to have somehting like a list of the APs that each client can see, with a "Kick to..." button that would allow active user intervention - but can't imagine many people really want to be actively managing things at that level.

Do you think what you want can be managed within a single "capability" like DAWN, or would it need a higher-level, cross-feature aggregated control plane including SQM, etc?

1 Like
8

That would be awesome!

One of the biggest bang for buck/effort items would be exactly that.. To be able to see (intuitively) where each client is connected. And MACs are not intuitive.

Right now for example, I have a script running in CRON on each AP that populates device names in the Status->Overview page, but to your point, this is exactly what should be on the DAWN page. Some UI love of the DAWN page would really achieve 80% of the base need I feel. This includes better wrapping and visualization of the information on the page. Right now when viewed on Mobile, the DAWN page is almost unreadable.

PS. The script I run, denoted as a cron task:

*/1 * * * * arp-scan -qxIN -I br-lan | awk '{print $1}' | xarges fping -q -c1
1 Like
9

I’d be very happy if I could see in a glimpse where my clients are currently connected too. If a roaming history could be shown per client or per AP for say the last 7 days that would be awesome. In my opinion a “Kick to..” button is not needed, it’s mainly a client decision to roam. It would kind of defeat the purpose of running DAWN basically. If we get our WiFi network configured correctly and have some decent behaving clients, a “Kick to..” button should not be needed at all.

I consider a basic overview of which client is connected on which AP now and a history on a 5min based interval for about 7 days a very welcome addition to the DAWN luci app :+1: :pray:
And yes, I’d consider that a single capability of DAWN.

1 Like
10

It should be nice to have a good view and also that de aps work together link dawn

Here some inspiration for a nice overview (took it from virtual controller form aruba, also )

arubaexample
arubaexample1451×212 47 KB

11

I've built the first phase of this project now, which is a CLI paired with a JSON file which allows you to provision configuration onto multiple devices.

The JSON file is structured in the same format found in config files in /etc/config/*, so the syntax and config options should be familiar.

If you'd like to give it a try, see the Getting Started section in the GitHub repo here.

If anyone has any feedback or suggestions, please let me know :slight_smile:

1 Like
12

Nice, I was looking for a mix between Ansible and uci, where I can have my config files in Git and have an easy way to push my changes to my devices

1 Like