Firewall zone logging inside OpenWrt container

xorbug · September 16, 2021, 10:06pm

Hi all,
I have setup a couple of extra firewall zones on OpenWrt 19.07.8, and I enabled logging for them since I'd like too have an hint in case something odd is going on. The default reject-only logging behaviour is ok for me, but if I try and trigger a rejected connection I don't get any log (but the firewall does work and no connection is going through)...
Here my dmz zone as example:

config interface 'dmz'
        option proto 'static'
        option netmask '255.255.255.0'
        option delegate '0'
        option ipaddr '192.168.3.1'
        option ifname 'eth3'

config zone
        option name 'dmz'
        option forward 'REJECT'
        option output 'ACCEPT'
        option input 'REJECT'
        list network 'dmz'
        option log '1'
        option log_limit '10/minute'

and from an host inside that zone:

ping -c 3 192.168.3.1
PING 192.168.3.1 (192.168.3.1): 56 data bytes

--- 192.168.3.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

In tcpdump I do see:

23:55:32.510779 IP 192.168.3.22 > 192.168.3.1: ICMP echo request, id 4954, seq 2, length 64
23:55:32.510958 IP 192.168.3.1 > 192.168.3.22: ICMP 192.168.3.1 protocol 1 port 51127 unreachable, length 92

So it is getting rejected, but no sign of logging in the system log... am I missing something?
Thanks.

xorbug · September 17, 2021, 10:11am

Sorry for bumping, but anyone has any clue about this?

flygarn12 · September 17, 2021, 10:27am

There is one quality guaranteed option to test this logging function.
Turn it on for WAN. If nothing showed up then you are either behind a ISP firewall or something is wrong with the function.

flygarn12 · September 17, 2021, 10:30am

For inter zone logging “on the inside” I have always had bad luck getting something in the log?

xorbug · September 17, 2021, 10:53am

Thanks @flygarn12

Ok maybe I can test this one later (and no I'm not behind ISP firewall so this must be able to prove the latter)...

Though this test was intra-zone. Also the wiki page says:

In these settings, INPUT and FORWARDED packets are logged when REJECTED and written to system log [...]

flygarn12 · September 17, 2021, 11:06am

Yes, but then we have the question if one ping wakes the log function. (Like from the Hunt for red October! “One ping and one ping only”, sorry it’s friday afternoon now)

And I am not quite sure what the 10/minute filter actually are supposed to do. Do you have to do 11 pings in one minute to get a output to the log?
Or does it writes 10 lines every minute?

When I have seen the WAN log there are no logic in that filter?

xorbug · September 17, 2021, 11:25am

I had tested access to LuCI too, so didn't work with TCP either (the logging, the firwall is fine)... also the docs only talk about rejected packets regardless of their type.

Should be the latter... In iptables-extensions(8):

limit
This module matches at a limited rate using a token bucket filter. A rule using this extension will match until this limit is reached. It can be used in combination with the LOG target to give limited logging, for example.

vgaetera · September 17, 2021, 11:26am

iptables-save | grep -e dmz -e LOG

xorbug · September 17, 2021, 3:06pm

Hi @vgaetera,
I have redacted a couple of sensitive lines regarding forwards other zones... but they should not be relevant...

:zone_dmz_helper - [0:0]
-A PREROUTING -i eth3 -m comment --comment "!fw3: dmz CT helper assignment" -j zone_dmz_helper
:postrouting_dmz_rule - [0:0]
:prerouting_dmz_rule - [0:0]
:zone_dmz_postrouting - [0:0]
:zone_dmz_prerouting - [0:0]
-A PREROUTING -i eth3 -m comment --comment "!fw3" -j zone_dmz_prerouting
-A POSTROUTING -o eth3 -m comment --comment "!fw3" -j zone_dmz_postrouting
-A zone_dmz_postrouting -m comment --comment "!fw3: Custom dmz postrouting rule chain" -j postrouting_dmz_rule
-A zone_dmz_prerouting -m comment --comment "!fw3: Custom dmz prerouting rule chain" -j prerouting_dmz_rule
:forwarding_dmz_rule - [0:0]
:input_dmz_rule - [0:0]
:output_dmz_rule - [0:0]
:zone_dmz_dest_ACCEPT - [0:0]
:zone_dmz_dest_REJECT - [0:0]
:zone_dmz_forward - [0:0]
:zone_dmz_input - [0:0]
:zone_dmz_output - [0:0]
:zone_dmz_src_REJECT - [0:0]
-A INPUT -i eth3 -m comment --comment "!fw3" -j zone_dmz_input
-A FORWARD -i eth3 -m comment --comment "!fw3" -j zone_dmz_forward
-A OUTPUT -o eth3 -m comment --comment "!fw3" -j zone_dmz_output
-A zone_dmz_dest_ACCEPT -o eth3 -m comment --comment "!fw3" -j ACCEPT
-A zone_dmz_dest_REJECT -o eth3 -m limit --limit 10/min -m comment --comment "!fw3" -j LOG --log-prefix "REJECT dmz out: "
-A zone_dmz_dest_REJECT -o eth3 -m comment --comment "!fw3" -j reject
-A zone_dmz_forward -m comment --comment "!fw3: Custom dmz forwarding rule chain" -j forwarding_dmz_rule
-A zone_dmz_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
-A zone_dmz_forward -m comment --comment "!fw3" -j zone_dmz_dest_REJECT
-A zone_dmz_input -m comment --comment "!fw3: Custom dmz input rule chain" -j input_dmz_rule
-A zone_dmz_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
-A zone_dmz_input -m comment --comment "!fw3" -j zone_dmz_src_REJECT
-A zone_dmz_output -m comment --comment "!fw3: Custom dmz output rule chain" -j output_dmz_rule
-A zone_dmz_output -m comment --comment "!fw3" -j zone_dmz_dest_ACCEPT
-A zone_dmz_src_REJECT -i eth3 -m limit --limit 10/min -m comment --comment "!fw3" -j LOG --log-prefix "REJECT dmz in: "
-A zone_dmz_src_REJECT -i eth3 -m comment --comment "!fw3" -j reject

But I'm also starting to suspect something else... since I'm running it inside a container, may this setup impacts the ability to log? Therefore I also checked the kernel logs on the host but there aren't any such messages logged... but for sure I see the log modules loaded fwiw:

# lsmod | grep -i log
nf_log_ipv4            16384  4
nf_log_common          16384  1 nf_log_ipv4
xt_LOG                 20480  4
x_tables               53248  15 xt_conntrack,iptable_filter,nft_compat,xt_LOG,xt_tcpudp,xt_CHECKSUM,xt_nat,xt_comment,ipt_REJECT,iptable_raw,ip_tables,xt_limit,xt_MASQUERADE,iptable_mangle,xt_TCPMSS

vgaetera · September 17, 2021, 3:41pm

All chains show zeros for traffic counters.
There's nothing to log if there's no traffic.

xorbug · September 17, 2021, 4:05pm

Ok makes sense... i believe it must be related to the current setup. As i said in the first post the firewall is doing its job in rejecting the traffic which would otherwise pass through... I'll try to investigate and report back if I find something.

xorbug · September 17, 2021, 6:07pm

@vgaetera something doesn't add up because on the contrary of what iptables-save shows:

# iptables -L zone_dmz_src_REJECT -n -v
Chain zone_dmz_src_REJECT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   34  2736 LOG        all  --  eth3   *       0.0.0.0/0            0.0.0.0/0            limit: avg 10/min burst 5 /* !fw3 */ LOG flags 0 level 4 prefix "REJECT dmz in: "
   51  4164 reject     all  --  eth3   *       0.0.0.0/0            0.0.0.0/0            /* !fw3 */

Now I don't know if this is the right zone to take into consideration but i can even see the live packets counter incrementing in LuCI as I keep pinging and getting rejected...

vgaetera · September 17, 2021, 6:17pm

Let's check again:

iptables-save -c | grep -e LOG; logread -e REJECT

xorbug · September 17, 2021, 6:30pm

[0:0] -A zone_dmz_dest_REJECT -o eth3 -m limit --limit 10/min -m comment --comment "!fw3" -j LOG --log-prefix "REJECT dmz out: "
[34:2736] -A zone_dmz_src_REJECT -i eth3 -m limit --limit 10/min -m comment --comment "!fw3" -j LOG --log-prefix "REJECT dmz in: "

Nothing from logread...

vgaetera · September 17, 2021, 6:36pm

Try to specify a more flexible limit like 1000/sec.
Make sure the log buffer is large enough to catch the messages.

xorbug · September 17, 2021, 8:36pm

Still nothing... I have remote logging but I checked logread just in case, and there is nothing.

xorbug · September 18, 2021, 10:48am

@vgaetera, I bet you are curious now, so... my doubts were right:

I will try to experiment with those workarounds (namely that echo 1 > /proc/sys/net/netfilter/nf_log_all_netns if possible to avoid fiddling with that ulog and change the rules by hand), but at least we have understood the root cause (which is not OpenWrt).
Sorry in case I wasted your time guys, I will edit the thread title to make this info more available for searches.

edit: yes this is working now with sysctl net.netfilter.nf_log_all_netns=1 on the host. Beware that this is now bypassing the default kernel behaviour meant to protect against log spam from network namespaces.

system · September 28, 2021, 10:49am

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.