Use named sections in /etc/config/network

Initial configuration in /etc/config/network contains:

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0.1'

I would like it to be a named section:

config device 'br_lan'
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0.1'

probably using a prefix (e.g. device__), to avoid name collisions:

config device 'device__br_lan'
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0.1'

Same with eth0.2 device, and all blocks of switch and switch_vlan types.
(Blocks of type "interface" already use named sections)

Why ?
It would make using terraform provider easier,
openwrt_network_device could be configured with id = "br_lan" or id = "device__br_lan".

The current way of using id = "@device[0]" and hoping that br-lan is (and remains) the first device block is fragile.

It looks like it requires modification of config_generate.
Replacing add network device with something like set network.device__$sanitizedname=device should fix it (where sanitizedname has dashes replaced with underscores).

Before attempting the change, is there any reason not to do this ? Would it break anything ?

First of all, it's ugly. Apart of that, it means introducing an arbitrary identifier which is not used by anything in OpenWrt, just for the sake of accommodating a very narrow use case (Terraform provider updating an existing configuration), which pollutes the configuration's ID namespace (section names must be globally unique, regardless of type) and does not map cleanly to the related device (br-lan needs to be transcribed as br_lan).

I think the Terraform provider needs to learn to lookup uci sections by option values (e.g. "find section of type device where the name option is 'br-lan'") instead of modifying the way OpenWrt generates its default.

Yes it is ugly, using section type as prefix was intended to avoid any chance of name collisions.

uci -X show wireless shows no unnamed sections, and it uses prefix default_ for default_radio0 and default_radio1 named sections, that seems nicer.

Using config device 'default_device_lan' for br-lan and config device 'default_device_wan for eth0.2 is longer, but probably less ugly.
Sections switch could work without any prefix, and switch_vlan could use _vlanN as a suffix.

TP-Link Archer C2 AC750 would then have this in /etc/config/network:

config device 'default_device_lan'
        option name 'br-lan'
        option type 'bridge'
        option ipv6 '1'
        list ports 'eth0.1'

config device 'default_device_wan'
        option name 'eth0.2'
        option macaddr 'aa:bb:cc:dd:ee:ff'

config switch 'switch0'
        option name 'switch0'
        option reset '1'
        option enable_vlan '0'

config switch 'switch1'
        option name 'switch1'
        option reset '1'
        option enable_vlan '1'

config switch_vlan 'switch1_vlan1'
        option device 'switch1'
        option vlan '1'
        option ports '1 2 3 4 6t'

config switch_vlan 'switch1_vlan2'
        option device 'switch1'
        option vlan '2'
        option ports '0 6t'

(only showing sections which are unnamed as of OpenWrt 23.05)

Would the above be acceptable ?

Terraform

The goal is not to introduce arbitrary identifiers, but to have the initial configuration use only named sections, in order to have stable identifiers. Though it would be nice if they were the same on most OpenWrt devices.

Ideally, a terraform provider creates a resource by performing an API call, and in response it gets an id to be used for detecting drift between actual resource state and wanted configuration.
For some resources, the ID is part of resource description (e.g. name of S3 bucket), for others it is automatically generated (e.g. AWS EC2 instance id).

Initial OpenWrt configuration already has some resource already present, that requries importing them to terraform, in order to modify them or depend on their state.
At the moment, terraform provider requires import using the id, i.e. the name of the config section.
(For unnamed section, even terraform import openwrt_network_device.br_primary '@device[0]' works, but the resource still has id 'cfg030f15')

Until few hours ago I thought that terraform required providers to have an immutable id to identify a resource. That having the provider use name as id would make it impossible to change the name of any bridge. Not a huge issue with the initial br-lan bridge, but a problem for bridges created using Terraform. A user might want to rename br-guest2 to br-iot.

But according to this, providers using new Terraform Plugin Framework actually can support changing the id of a resource.

Still, using "name option" as resource id would be very far from ideal. It looks like luci api can get a section in one API call only by using section name (even cfgNUMBER), but cannot get a section based on one or more of its options' values.
For every bridge, the provider would have to get the whole network config. For switch_vlan the provider does not have anything like name, it would have to search for a section with both device and vlan options matching the resource, and use both options to construct some sort of unique id.

It would be a bit slower and needlesly more complex. Adding few strings to the initial configuration file seems like a much better option.

For completenes, using "name option" only for import, and then using unnamed section (cfgNUMBER) as the id would not work.

According to source it depends on:

  • position of the section within the file (used as first two digits)
  • section type
  • option names
  • values of string options

Removing br-lan would change cfgNUMBER id of eth0.2 device. Next terraform run, it would try to create new device for eth0.2, because it would look like it was deleted.

Modifying br-lan options should also change the cfgNUMBER, but experiment with renaming br-lan to br-notlan did not change it, it remained cfg030f15 even after reboot. No idea why.

I searched ‘terraform’ on the forum to se how big global problem we have and I only found two hits and this tread was one of them. The other one with somewhat similar question was this: https://forum.openwrt.org/t/uci-change-static-lease-and-deploy-static-leases-by-code/90352

There's currently no direct method to access the section by the option value, which makes automated configuration management a bit of a headache, at least that's how it looks in CLI:

device_proc() {
local CONF="${1}"
local NAME
config_get NAME "${CONF}" name
if [ "${NAME}" = "${DEVICE}" ]
then ...
fi
}
IFACE="lan"
. /lib/functions.sh
config_load network
config_get DEVICE "${IFACE}" device
config_foreach device_proc device

It may be useful to implement an easier way to address the matching section:

DEVICE="$(uci get network.lan.device)"
uci get network.@device[name="${DEVICE}"]

This:

might be an improvement, but it would not be enough.

In order to match correct vlan on a switch:

config switch_vlan
        option device 'switch1'
        option vlan '1'
        option ports '1 2 3 4 6t'

config switch_vlan
        option device 'switch1'
        option vlan '2'
        option ports '0 6t'

it would have to support matching multiple properties:

uci get network.@switch_vlan[device="switch1" && vlan='2']

That seems needlessly more complex, when the alternative is to just ensure that all sections in the initial configuration are named:

config switch_vlan 'switch1_vlan1' #named section
        option device 'switch1'
        option vlan '1'
        option ports '1 2 3 4 6t'

config switch_vlan 'switch1_vlan2' #named section
        option device 'switch1'
        option vlan '2'
        option ports '0 6t'

and then use just:
uci get network.switch1_vlan2

The issue is only with the sections in the initial configuration.

Any sections created by any automated configuration management should just be named sections (and configuration management should record the section name it used in its state).

Thinking about it a bit more, the section should probably be named switch1_vlan_wan (instead switch1_vlan2) to better match the remaining suggested section names.

There are only few places that generate unnamed sections in config_generate:

114 add network device
122 add network device
146 add network device
240 add network switch_vlan
267 add network switch_port
295 add network switch
313 add system system

It looks like I should be able to make a patch.

I am missing a real-life example of /etc/board.json file from a device that generates switch_port sections. Any suggestions ?

Well if you intend to coexist with user modified config, you need to properly support anonymous sections. If it is just about initial config/regenerating whole configs then the better course of action will be getting essential network base config from /etc/board.json and generating the entire /etc/config/network from scratch.

I stand by my opinion, adding section names is ugly and redundant, as well as error prone. Users will copy paste default sections to add further vlans and forget about adjusting the identifiers, leading to errors.

Configuring VLANs via the ui will add anonymous sections again etc.

2 Likes

my two cents: I have been manually adding section names because I thought it was neater but having them disappear makes it completely pointless. Thanks for confirming that indeed it is so :slight_smile:

My goal is to manage all OpenWrt APs and the OpenWrt using terraform only. With additional settings OpenWrt router (dhcp hosts) configured using additional terraform projects (e.g. proxmox homelab experiments). The coexistence with user modification was considered more on the level of changing section options, and then writing those values to terraform to be applied to all devices.

Took some more time to explore, if it is even possible to coexist with user modified config with unnamed sections. It looks like it is not possible.

For OpenWrt AP it is just /etc/config/network, but for OpenWrt router there are others (dhcp, firewall).

Even if something like uci get network.@switch_vlan[device="switch1" && vlan='2'] were to be implemented (probably somewhere around here), it would not be enough.

Assuming part of /etc/config/dhcp looks like this:

config host
        option mac '00:11:22:33:44:55'
        option name 'machine1'
        option dns '1'
        option ip '192.168.1.101'

config host
        option mac '00:11:22:33:44:aa'
        option name 'machine2'
        option dns '1'
        option ip '192.168.1.102'

Because there are no section names, it behaves like a database table without any primary key.

Any combination of options cannot be used as a composite key, because any one of the mac, name and ip options can be reasonably changed by the user.
e.g. replacement of wifi card (or re-creation of VM with new random mac), renaming of a machine, changing machine's allocated IP address (maybe moving it to different VLAN).

If a value of any option used in composite key changed, it would not be detected as the same resource, and terraform would (correctly) assume it destroyed and plan to create new host section with correct values.
(the user modified host section would no longer be managed by terraform and probably cause mac/name/ip collisions...)

cfgNUMBER cannot be used because it is not stable, deleting machine1 host is indistinguishable from deletion of machine2 host and changes in mac,nameand ip of machine1.
There is no way for terraform to track modifications to these sections - did the user remove one section, or changed options of all the following sections and deleted the last one ?

Looking at /etc/config/firewall, even the order of unnamed sections is significant. Changing priority of one rule (by moving) changes the cfgNUMBER identifier of the rules it moved over...

Thanks for the feedback. I assumed the issue was the ugly identifier (repetition of the section type in section name and the double underscore), not the addition of the name itself.

The idea was to make the default OpenWrt configuration easily importable into terraform.

With unnamed sections much more common than I thought, the only reasonable solution seems to be to just use a script to add names to all the unnamed sections in all the initial config files and run it before terraform import. It will be device specific, but without relying on cfgNUMBER section names, it should be possible to use terraform to manage OpenWrt devices relilably (assuming that firewall configuration is simple enough, that the order of rules does not matter).

I really hope that I'm just missing something obvious, because I do not see how any configuration management can work with unnamed sections without just ignoring user modifications and always overwriting the whole config file.

If unnamed sections were a filesystem, then (anyone) deleting a file in a directory would cause only the contents of the file to be deleted, not the file itself. Following files' contents would just shift to a one file prior and the last file in the directory would be removed. In such filesystem, it would be impossible to keep something like a symlink pointing to the same "file".

This can maybe be something.

To be hounest if you look at pending changes in Luci, deleting the old and rewrite is exactly what Luci does when you change something.

That also mean that you can’t for example change a list of something, uci simply makes a completely new list.

Well I tend to think of the configuration model in this way:

  • The entirety of the configuration is a bunch of entities
  • Entities are typed (the XXX part in config XXX)
  • Depending on the entity type, one or more options form the entities primary key;
    • For network interfaces the PK is the section name
    • For firewall zones it is option name
    • For DHCP reservations it is option mac
    • For switch-vlans and bridge-vlans it is option device + option vlan
    • For firewall rules it is the sum of all option values sans option comment since semantically it makes no sense to have two completely identical rules
    • ...

Which means that a configuration management that relies on stable identifiers for entities needs to have dedicated knowledge of all the various uci "entity types" (= uci section types) it wants to support in order to be able to synthesize stable identifiers regardless of whether these section happened to be named or not.

In the end, such an implementation would likely carry a form of schema definition to describe the various uci entities, like this pseudo code:

const entity_types = [
  {
    .name = "network interface",
    .flags = IS_NAMED,
    .uci_config = "network",
    .uci_type = "interface",
    .key = [ "$section_name" ],
    .options = [
      {
        .name = "proto",
        .type = "string",
        .enum = [ "none", "static", "dhcp", ... ]
      },
      {
        .name = "device",
        .type = "string"
      },
      ...
    ]
  },
  {
    .name = "bridge VLAN definition",
    .flags = IS_ANONYMOUS,
    .uci_config = "network",
    .uci_type = "bridge-vlan",
    .key = [ "device", "vlan" ],
    .options = [
      {
        .name = "device",
        .type = "string"
      },
      {
        .name = "vlan",
        .type = "integer"
      },
      ...
    ]
  },
  {
    .name = "firewall rule",
    .flags = IS_ANONYMOUS,
    .uci_config = "firewall",
    .uci_type = "rule",
     // PK = hash of all option values (sorted by option name) without the
     // "name" (freetext comment) option
    .key = [ "*", "!name" ],
    .options = [
      {
        .name = "src",
        .type = "string"
      },
      {
        .name = "dest",
        .type = "string"
      },
      ...
    ]
  },
  ...
];

Use a script to auto-rename sections according to their primary key:
I run this script to normalize the configuration before each automation. I wish this was the default.

the result:

network.device_br_lan
network.device_br_wan
network.device_eth1
network.device_wan
firewall.zone_lan
firewall.zone_wan
firewall.forwarding_lan_wan
firewall.rule_Allow_DHCP_Renew
firewall.rule_Allow_Ping
firewall.rule_Allow_IGMP

Now you can access each section by name, without jumping through hoops to find the section name. For example, to remove lan-to-wan forwarding:

uci delete firewall.forwarding_lan_wan

or disable ping from WAN:

uci set firewall.rule_Allow_Ping.enabled=0

The script:

#!/bin/sh

. /lib/functions.sh

normalize_section_name_cb() {
    local section="$1"
    local type name src dest section_name

    config_get type "$section" TYPE
    config_get name "$section" name

    case "$type" in
        zone)
            section_name="zone_$name"
            ;;
        forwarding)
            config_get src "$section" src
            config_get dest "$section" dest
            section_name="forwarding_${src}_${dest}"
            ;;
        rule)
            section_name="rule_$name"
            ;;
        device)
            section_name="device_$name"
            ;;
        *)
            return
    esac

    uci rename "$config.$section=$(echo $section_name | tr '-' '_')"
}

for config in network firewall; do
    config_load "$config"
    config_foreach normalize_section_name_cb
done

uci commit
1 Like