[network/utils & libs] version bump nftables 0.9.4 & libnftnl 1.1.6

For Developers
1

Beginning of December '19 source development released

  • nftables 0.9.3 [1]
  • libnftnl 1.1.5 [2]

Since each provides fixes for existing functionality it would be appreciated if a developer could be obliged to sponsor a PR for Master and 19.07

changelog nftables

Ander Juaristi (4):
netfilter: support for element deletion
evaluate: New internal helper __expr_evaluate_range
meta: Introduce new conditions 'time', 'day' and 'hour'
tests: add meta time test cases

Christian Göttsche (3):
statement: make secmark statements idempotent
src: add ability to set/get secmarks to/from connection
files: add example secmark config

Eric Garver (6):
cache: fix --echo with index/position
tests: shell: check that rule add with index works with echo
tests: shell: verify huge transaction returns expected number of rules
tests: shell: add huge JSON transaction
tests: shell: add huge transaction from firewalld
parser_json: fix crash on insert rule to bad references

Eric Jallot (10):
src: secmark: fix brace indentation and missing quotes in selctx output
src: parser_json: fix crash while restoring secmark object
src: obj: fix memleak in handle_free()
tests: shell: fix failed tests due to missing quotes
obj: fix memleak in parser_bison.y
flowtable: fix memleak in exit path
src: flowtable: add support for named flowtable listing
doc: fix missing family in plural forms list command.
src: flowtable: add support for delete command by handle
scanner: fix out-of-bound memory write in include_file()

Fernando Fernandez Mancera (5):
netlink_delinearize: fix wrong conversion to "list" in ct mark
src: add synproxy stateful object support
json: fix type mismatch on "ct expect" json exporting
json: tests: fix typo in ct expectation json test
tests: add stateful object update operation test

Florian Westphal (6):
src: json: add support for element deletion
src: evaluate: catch invalid 'meta day' values in eval step
evaluate: flag fwd and queue statements as terminal
src: meter: avoid double-space in list ruleset output
tests: check we can use "dynamic" set for lookups
expression: extend 'nft describe' to allow listing data types

Jeremy Sowden (11):
configure: remove unused AC_SUBST macros.
cli: remove unused declaration.
cli: add linenoise CLI implementation.
src: use -T as the short option for --numeric-time.
src: add --terse to suppress output of set elements.
doc: add missing output flag documentation.
main: add missing OPT_NUMERIC_PROTO long option.
main: remove duplicate output flag assignment.
py: add missing output flags.
src: add and use set_is_meter helper
doc: fix inconsistency in set statement documentation.

Michal Rostecki (1):
mnl: Fix -Wimplicit-function-declaration warnings

Pablo Neira Ayuso (15):
tests: shell: use-after-free from abort path
mnl: fix --echo buffer size again
libnftables: use-after-free in exit path
mnl: do not cache sender buffer size
tests: shell: delete flowtable after flush chain
libnftables: memleak when list of commands is empty
segtree: always close interval in non-anonymous sets
datatype: display description for header field < 8 bits
src: define flowtable device compound as a list
src: restore --echo with anonymous sets
src: add multidevice support for netdev chain
tests: shell: set reference from variable definition
segtree: restore automerge
netlink: off-by-one write in netdev chain device array
build: Bump version to v0.9.3

Phil Sutter (25):
parser_bison: Fix 'exists' keyword on Big Endian
mnl: Don't use nftnl_set_set()
monitor: Add missing newline to error message
tests/monitor: Fix for changed ct timeout format
rule: Fix for single line ct timeout printing
parser_json: Fix checking of parse_policy() return code
tproxy: Add missing error checking when parsing from netlink
main: Fix for misleading error with negative chain priority
Revert "main: Fix for misleading error with negative chain priority"
tests/py: Fix test script for Python3 tempfile
mnl: Replace use of untyped nftnl data setters
doc: Drop incorrect requirement for nft configs
libnftables: Store top_scope in struct nft_ctx
meta: Rewrite hour_type_print()
segtree: Check ranges when deleting elements
segtree: Fix get element for little endian ranges
cache: Reduce caching for get command
parser_bison: Avoid set references in odd places
files: Install sample scripts from files/examples
files: Drop shebangs from config files
scanner: Introduce numberstring
nft.8: Describe numgen expression
nft.8: Fix nat family spec position
tests/py: Set a fixed timezone in nft-test.py
segtree: Fix add and delete of element in same batch

Sergei Trofimovich (1):
nftables: don't crash in 'list ruleset' if policy is not set

Sven Auhagen (1):
mnl: remove artifical cap on 8 devices per flowtable

wenxu (1):
meta: add ibrpvid and ibrvproto support

changelog libnftnl

Ander Juaristi (2):
expr: meta: Make NFT_META_TIME_{NS, DAY, HOUR} known
expr: meta: Make NFT_DYNSET_OP_DELETE known

Eric Jallot (1):
flowtable: add support for handle attribute

Fernando Fernandez Mancera (1):
src: synproxy stateful object support

Manuel Messner (1):
flowtable: Fix symbol export for clang

Pablo Neira Ayuso (4):
flowtable: device array dynamic allocation
chain: multi-device support
flowtable: remove NFTA_FLOWTABLE_SIZE
build: libnftnl 1.1.5 release

Phil Sutter (11):
set: Export nftnl_set_list_lookup_byname()
obj: ct_timeout: Check return code of mnl_attr_parse_nested()
set_elem: Fix return code of nftnl_set_elem_set()
obj/tunnel: Fix for undefined behaviour
set: Don't bypass checks in nftnl_set_set_u{32,64}()
obj/ct_timeout: Avoid array overrun in timeout_parse_attr_data()
set_elem: Validate nftnl_set_elem_set() parameters
obj/ct_timeout: Fix NFTA_CT_TIMEOUT_DATA parser
libnftnl.map: Export nftnl_{obj,flowtable}_set_data()
Deprecate untyped data setters
utils: Define __visible even if not supported by compiler

[1] https://netfilter.org/news.html#2019-12-02-d
[2] https://netfilter.org/news.html#2019-12-02

2

My personal experience with open source projects is that if you have a personal itch it is often quicker if you scratch it yourself. Getting someone else to scratch it for you is dependant upon them finding the itch sufficiently insistent.

You may find a kind hearted person who can do the bump 'blind' and without a use case, but I have been personally bitten a few times doing 'innocuous bumps' on packages that I don't use and things coming back to bite me.

3

Not every user has the capability to submit a PR.

Sure, the user's dependency on the package maintainer is clear. Tough luck if the maintainer is not considering it worthwhile to go along.

Others may use those packages in question however, yet might not necessarily be in a position to contribute to package maintenance in the distro's repo. There might be developers that maintain package they may not deploy on their production nodes, except perhaps for testing purposes on testing nodes.

On the other hand it would seem that FW4 is on the agenda for this year and which supposedly deploys either or both packages and thus might be good reason to keep up with the source code development.

4

@ldir thanks for bumping libnftl in Master. Any plan to uplift to 19.07.x?

Unfortunately though the compile arg --without-json-parsing is imposed and thus:

  • rendering export of NFT rules to json impossible [3]
  • rendering deployment of libnftl-json impossible [4]

[3] https://bugs.openwrt.org/index.php?do=details&task_id=2821
[4] [fw4] libnftables JSON

5

libnftl was bumped purely to satisfy an issue related to an iptables bump. I didn't really 'enjoy' touching either of them in master (I hate touching stuff I don't understand/use), let alone backporting to a release.

6

The really odd thing is that '--without-json-parsing' isn't a legal config flag..at least not anymore.

Right, it's that nftables isn't being built with json support BY DEFAULT. So either we build nftables with json support by default, or some sort of variant option needs to be explored. Hmmmm.

7

Thanks for bumping NFT in Master :+1:

The json part is not clear though, on one hand there is

DEPENDS:=+PACKAGE_NFT_WITH_JSON:jansson

but then it is set not to support json

define Package/nftables/config
        config PACKAGE_NFT_WITH_JSON
                bool "Build nftables with json support"
                depends on PACKAGE_nftables
                default n
endef
8

Correct. Basically the nftables package built by the snapshot or 'release' builders does not have json support enabled.

If you build your own then you can enable the json option.

Ideally there would be two variants of the nftables package, say 'nftables' and 'nftables-json'. I attempted to implement the variants yesterday but got dependency errors and there's another issue lurking in that 'libjansson' is NOT part of the base openwrt repo but rather in the package feed.

Something I was hoping would be 'easy' low hanging fruit has escalated.

The 'easiest' way forward is to move jansson to base openwrt and enable json in nftables by default. But then we'll have two json related libraries in base openwrt.

1 Like
9

At least you made the effort since the maintainer cited does not seem active.

Backporting to the current stable branch (19.07.x) seems pointless with the way the kernel conf flags are unset in support for nftables [5] and thus rendering the deployment of nftables pointless.

I just had hoped that the json support would have been easier to implement since wanting to export current rules in the json format in order to deploy with libftnl but reckon have to write the rules in the json format from scratch then.

[5] https://bugs.openwrt.org/index.php?do=details&task_id=2360

10

It is definitely obsolete for libnftl since it does not even provide json parsing (any more), only libnftables (nowadays being integral part of nftables) does.

  • libnftnl is the low-level netlink library (responsible for creating/parsing the nf_tables netlink messages)
  • libnftables is the high-level library (contains most of the nftables functionality, such as parsing (nft rule format and json))

Not sure I understand - does a single jansson installation from the lib feed does not suffice?

A bit of egg vs. chicken. On one hand NFT is available (for deployment) in the distro but forgoes its json parsing functionality (mitigating the amount of bytes?) and on the other hand requires the user to customise the build conf with the aim of deploying NFT with said functionality.

11

jansson lives in the packages feed, it is not part of openwrt base, so phase1 builder (which creates the buildroot SDK etc) has a dependency on a lib that isn't in the base repo and will fail. Therefore as it currently stands we cannot build a json enabled version of 'nft' because the necessary lib isn't available to us at 'phase 1' build time. I think that whatever way this cat is skinned, jansson needs to move to base.

I have been trying to come up with a solution that builds 2 versions of nftables (one with json, one without) to satisfy the potential extra space argument, that is having problems with openwrt dependency handling aside from the fact jansson still would need to be moved.

It needs more brainpower and spare time.

1 Like
12

Progress of sorts, not ready yet, still working on it...and falling into loads of traps too.

https://git.openwrt.org/?p=openwrt/staging/ldir.git;a=commit;h=4dda00506e7b94fb1dba082afc81fdb478b5ff3a

1 Like
13

@ldir Thank you for the effort, now with the json support package available.

Meantime source development released

  • nftables 0.9.4 [6]
  • libnftnl 1.1.6 [7]
  • conntrack-tools 1.4.6 [8]
  • libnetfilter_conntrack 1.0.8 [9]

[6] https://lwn.net/Articles/816528
[7] https://lwn.net/Articles/816530
[8] https://www.spinics.net/lists/netfilter/msg59376.html
[9] https://www.spinics.net/lists/netfilter/msg59375.html

14

I'm going to take a bit of a breather from this for a while. I think I've done enough of late and don't want to run the risk of breaking things :slight_smile: