I do not think so. I believe kmsg will serialize them.
Wrong. Your script is invoked one per each interface in the fire and forget non-blocking mode, so the kernel does not wait until scripts finish and keep on setting up interfaces. So, yes the script is absolutely running many instances.
There are multiple problems here: the firewall gets reloaded several times during the network setup, so without proper planning and synchronization the approach will not work reliably.
Does not look like the OP is listening, though.
good point.
for reference to others re: firewall chatter
170007 iface ifup INTERFACE:lan
170008 firewall add INTERFACE:wan
170008 firewall add INTERFACE:wan6
170008 firewall add INTERFACE:lan
170008 iface ifup INTERFACE:loopback
170008 net add eth1 /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-2/2-2:1.0/net/eth1
170010 net add wlan0 /devices/platform/soc/fe300000.mmcnr/mmc_host/mmc1/mmc1:0001/mmc1:0001:1/net/wlan0
170010 iface ifup INTERFACE:wan
170011 net add br-lan /devices/virtual/net/br-lan
170012 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170012 firewall remove INTERFACE:wan6
170012 firewall remove INTERFACE:lan
170012 firewall remove INTERFACE:wan
170012 firewall add INTERFACE:lan
170012 firewall add INTERFACE:wan6
170012 firewall add INTERFACE:wan
170013 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170014 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170014 iface ifup INTERFACE:wan6
170015 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170015 firewall remove INTERFACE:lan
170015 firewall remove INTERFACE:wan6
170015 firewall remove INTERFACE:wan
170015 firewall add INTERFACE:wan
170015 firewall add INTERFACE:lan
170015 firewall add INTERFACE:wan6
170016 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170017 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170018 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170019 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170020 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170021 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170022 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170023 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170024 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170025 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170026 net add ifb4eth1 /devices/virtual/net/ifb4eth1
170027 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170028 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170029 net remove ifb4eth1 /devices/virtual/net/ifb4eth1
170030 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170031 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170032 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170033 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170034 net add ifb4eth1 /devices/virtual/net/ifb4eth1
170035 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170036 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170037 net remove ifb4eth1 /devices/virtual/net/ifb4eth1
170038 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170039 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170040 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170041 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170042 net add ifb4eth1 /devices/virtual/net/ifb4eth1
170043 net add TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
170044 net remove TMP_IFB_4_SQM /devices/virtual/net/TMP_IFB_4_SQM
171653 iface ifupdate INTERFACE:wan6
173335 iface ifupdate INTERFACE:wan6
175016 iface ifupdate INTERFACE:wan6
180656 iface ifupdate INTERFACE:wan6
181641 iface ifupdate INTERFACE:wan6
181950 iface ifupdate INTERFACE:wan6
182337 iface ifupdate INTERFACE:wan6
184017 iface ifupdate INTERFACE:wan6
185658 iface ifupdate INTERFACE:wan6
185811 iface ifupdate INTERFACE:wan6
Right, and let's now image each line is a script invocation in a very quick succession. I worked through a similar use case and stabilizing this kind of functionality is no easy task.
Now I got it, thanks for clarification.
#!/bin/sh
exec 511>/var/lock/etc-hotplug.d-net-my-script.lock
flock 511 || (echo "Failed to obtain a lock" ; exit 1)
echo hotplug/iface: $ACTION $INTERFACE >> /tmp/if_dbg
#echo hotplug/iface: $ACTION $INTERFACE >> /dev/kmsg
I did a couple reboots and it executed every time the hotplug scripts.
(I understand now they got executed all the time before, but the simultanious calls caused the problems)
root@test-r1:~# cat /tmp/if_dbg
hotplug/iface: ifup loopback
hotplug/iface: ifdown wan_3g
hotplug/iface: ifup lan
Yup, the absence of messages does not mean the scripts are not running. Mark the thread as resolved if the problem is solved.
I actually want to do a NETMAP to the IP address on the lan interface. Means I have to do it every time when the IP address changes there, or if an IP address comes up for the first time after a reboot.
Is there any better solution as to do this via a hotplug script in /etc/hotplug.d/net/99-netmap?
That is the best way I know. The problem is you have to wait for all the network changes to complete and the firewall restarts to stop, before you script actually does anything. It is challenging, because there are many concurrent invocations and only one can proceed while all the others need to finish without doing anything.
I suggest you use the flock approach and once the lock is obtained, you can create a file in /tmp/ if it does not exist. If the file exists, then the script exits without doing anything.
The lucky script can then wait for a minute or so (test this) and make all the config changes AND delete the file created in /tmp/, so that the next network reconfiguration could apply your changes again.
It seems it still works not reliable, even with writing to kmsg and using flock in the script.
This is my current script (I had to move it back to iface, since at the call in net the DHCP IP address was not availble).
#!/bin/sh
exec 511>/var/lock/etc-hotplug.d-ifup-99-netmap.lock
flock 511 || (echo "Failed to obtain a lock" ; exit 1)
echo hotplug/iface: $ACTION $INTERFACE >> /tmp/if_dbg
echo hotplug/iface: $ACTION $INTERFACE >> /dev/kmsg
if [ "$ACTION" = "ifup" -a "$INTERFACE" = "lan" ]; then
network0=`ubus call network.interface.lan status | grep \"address\" | grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.';`
network="$network0""0/24"
iptables -t nat -A PREROUTING -d 10.1.223.0/24 -i tun1 -j NETMAP --to $network
iptables -t nat -A POSTROUTING -s $network -o tun1 -j NETMAP --to 10.1.223.0/24
echo $ACTION $INTERFACE >> /tmp/lan_up_dbg
date +"%T" >> /tmp/lan_up_dbg
echo iptables -t nat -A PREROUTING -d 10.1.223.0/24 -i tun1 -j NETMAP --to $network >> /tmp/lan_up_dbg
echo iptables -t nat -A POSTROUTING -s $network -o tun1 -j NETMAP --to 10.1.223.0/24 >> /tmp/lan_up_dbg
echo "#!/bin/sh" > /tmp/lan_down
echo iptables -t nat -D PREROUTING -d 10.1.223.0/24 -i tun1 -j NETMAP --to $network >> /tmp/lan_down
echo iptables -t nat -D POSTROUTING -s $network -o tun1 -j NETMAP --to 10.1.223.0/24 >> /tmp/lan_down
chmod +x /tmp/lan_down
fi
if [ "$ACTION" = "ifdown" -a "$INTERFACE" = "lan" ]; then
echo $ACTION $INTERFACE >> /tmp/lan_up_dbg
date +"%T" >> /tmp/lan_up_dbg
/tmp/lan_down
rm /tmp/lan_down
fi
After reboot:
root@test-r1:~# ifconfig
br-lan Link encap:Ethernet HWaddr 00:0C:29:3B:DE:43
inet addr:192.168.90.157 Bcast:192.168.90.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe3b:de43/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1153 errors:0 dropped:0 overruns:0 frame:0
TX packets:744 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:277632 (271.1 KiB) TX bytes:484444 (473.0 KiB)
eth0 Link encap:Ethernet HWaddr 00:0C:29:3B:DE:43
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4306 errors:0 dropped:3 overruns:0 frame:0
TX packets:744 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:758604 (740.8 KiB) TX bytes:484444 (473.0 KiB)
Interrupt:19 Base address:0x2000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:22 errors:0 dropped:0 overruns:0 frame:0
TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2173 (2.1 KiB) TX bytes:2173 (2.1 KiB)
tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:10.102.0.53 P-t-P:10.102.0.53 Mask:255.255.0.0
inet6 addr: fe80::47df:661:4158:d8f/64 Scope:Link
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:0 (0.0 B) TX bytes:304 (304.0 B)
root@test-r1:~# cat /tmp/if_dbg
cat: can't open '/tmp/if_dbg': No such file or directory
root@test-r1:~# /etc/init.d/network restart
root@test-r1:~# cat /tmp/if_dbg
hotplug/iface: ifup loopback
hotplug/iface: ifdown wan_3g
hotplug/iface: ifup lan
hotplug/iface: ifup loopback
What I am doing wrong there?
Start by re-reading the thread. Good luck!
It actually looks to me as if the original problem that the iface script gets not always called during the boot is still present.
This is my current script and it should solve all the concurrency issues writing to files.
#!/bin/sh
echo hotplug/iface b_flock: $ACTION $INTERFACE >> /dev/kmsg
exec 511>/var/lock/etc-hotplug.d-iface-99-netmap.lock
flock 511 || (echo "Failed to obtain a lock" ; exit 1)
echo hotplug/iface a_flock: $ACTION $INTERFACE >> /dev/kmsg
echo hotplug/iface: $ACTION $INTERFACE >> /tmp/if_dbg
if [ "$ACTION" = "ifup" -a "$INTERFACE" = "lan" ]; then
network0=`ubus call network.interface.lan status | grep \"address\" | grep -oE '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.';`
network="$network0""0/24"
iptables -t nat -A PREROUTING -d 10.1.223.0/24 -i tun1 -j NETMAP --to $network
iptables -t nat -A POSTROUTING -s $network -o tun1 -j NETMAP --to 10.1.223.0/24
echo $ACTION $INTERFACE >> /tmp/lan_up_dbg
date +"%T" >> /tmp/lan_up_dbg
echo iptables -t nat -A PREROUTING -d 10.1.223.0/24 -i tun1 -j NETMAP --to $network >> /tmp/lan_up_dbg
echo iptables -t nat -A POSTROUTING -s $network -o tun1 -j NETMAP --to 10.1.223.0/24 >> /tmp/lan_up_dbg
echo "#!/bin/sh" > /tmp/lan_down
echo iptables -t nat -D PREROUTING -d 10.1.223.0/24 -i tun1 -j NETMAP --to $network >> /tmp/lan_down
echo iptables -t nat -D POSTROUTING -s $network -o tun1 -j NETMAP --to 10.1.223.0/24 >> /tmp/lan_down
chmod +x /tmp/lan_down
fi
if [ "$ACTION" = "ifdown" -a "$INTERFACE" = "lan" ]; then
echo $ACTION $INTERFACE >> /tmp/lan_up_dbg
date +"%T" >> /tmp/lan_up_dbg
/tmp/lan_down
rm /tmp/lan_down
fi
It sometimes calls the script during the boot and sometimes not.
root@test-r1:~# reboot
root@test-r1:~# dmesg | grep hotplug
[ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
root@test-r1:~# reboot
root@test-r1:~# dmesg | grep hotplug
[ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[ 11.621435] hotplug/iface b_flock: ifup loopback
[ 11.621863] hotplug/iface a_flock: ifup loopback
[ 11.625733] hotplug/iface b_flock: ifdown wan_3g
[ 11.626261] hotplug/iface a_flock: ifdown wan_3g
[ 11.651263] hotplug/iface b_flock: ifup lan
[ 11.651881] hotplug/iface a_flock: ifup lan
In both situations was the lan interface up and had an IP address.
It happens not very often (maybe ever 10-20 times of rebooting) that the script gets not executed during the boot, but if it happens, then it also gets not executed on the shutdown.
I was able to take a sceenshot on the shutdown after the script did not work during the boot.
If the script was executed during the boot, then it also gets executed during the shutdown.
If I have the system running after the script got not executed during the boot and I restart the network, then the script gets executed and then also later on the shutdown.
Is there anything else what I could check if I have the system running after a boot which did not execute the scripts?
here is what i recommend you do... it will be beneficial for you in any implementation anyway...
half of your issue is that you are NOT VALIDATING and debugging much at the firewall level.
(p.s. i marked fantom-x's answer as correct on your behalf as it imho most accurately and adequately answers the question you initially asked)
Thanks, I will have a look on this.
Independent from that, that there is be a better solution to do my netmap, why is this iface event not called reliably?
Hotplug scripts which get not executed during the boot could cause other problems as well.
For example gets the firewall script also not called in this situation.
root@test-r1:~# logread | grep "Reloading firewall"
No log entry for the firewall (created by /etc/hotplug.d/iface/20-firewall) if this happens with the not-executed scripts during boot.
root@test-r1:~# /etc/init.d/network restart
root@test-r1:~# logread | grep "Reloading firewall"
Sun Jun 21 12:48:45 2020 user.notice firewall: Reloading firewall due to ifup of lan (br-lan)
root@test-r1:~# reboot
root@test-r1:~# logread | grep "Reloading firewall"
Sun Jun 21 12:49:22 2020 user.notice firewall: Reloading firewall due to ifup of lan (br-lan)
After another reboot, on which it did execute my script, the firewall entry was created in the log.
(I suppressed for testing all the executions of iptables commands in my script)
I think it is not solved, since the iface script gets not executed on every boot.
you have return to this hypothesis already before... and twice you been convinced that it was flawed. i'm not saying that you are wrong. but unless you provide "definitive evidence" that substantiates this theory... and explain in detail why such a theory is true... at this stage in the game I think you have to trust the advice you've been given and know that it IS run EVERY time on BOOT. and invest some time writing detailed scripts to prove or disprove other possible influences and advice before immediately oversimplifying things again. or even better... diversify your efforts so as not to let one "method" stop you from achieving your end result.
unticked... please remember to mark one when you feel you have found the answer you are looking for.
Add this line before the the first echo command, reboot, and see if there is anything in that file.
exec >>/tmp/exec.log 2>&1
Also, remove kmsg and use /tmp/if_dbg to monitor the script.
UPDATE: I do not know the sequence of things, but it is possible that kmsg is not always fully setup yet by the time the hot plug scripts are executed, so using a file is a more reliable solution.
I did some further tests.
Installed a new and clean OpenWRT instance in VMware.
Only changed its static lan IP address to DHCP.
Created a test script /root/test.sh
#!/bin/sh
sleep 30
if logread | grep -F "Reloading firewall"; then
reboot
fi
Added a call for this script to /etc/rc.local.
This script issues a reboot after 30s if there is "Reloading firewall" firewall detected in the log, which gets created by the /etc/hotplug.d/iface/20-firewall script which came with OpenWRT.
This was running for more than an hour and it never stopped rebooting.
Means there was no problem running the hotplug scrips.
Then I installed these modules, but changed nothing else. Same modules are installed on the other instance which causes the problems.
base-files busybox comgt-ncm dnsmasq dropbear e2fsprogs flock htop ip6tables iptables-mod-dnetmap iptables-mod-nat-extra kernel kmod-3c59x kmod-8139too kmod-bnx2 kmod-button-hotplug kmod-e100 kmod-e1000 kmod-e1000e kmod-igb kmod-ipt-offload kmod-natsemi kmod-ne2k-pci kmod-nf-nathelper-extra kmod-pcnet32 kmod-r8169 kmod-sis900 kmod-tg3 kmod-usb-net-rndis kmod-via-rhine kmod-via-velocity logd luci-app-wol luci-proto-3g luci-ssl miniupnpc mkf2fs mtd natpmpc ntpclient odhcp6c odhcpd-ipv6only openvpn-openssl partx-utils picocom ppp ppp-mod-pppoe ppp-mod-pptp px5g-mbedtls uci urandom-seed urngd usb-modeswitch usbutils
I did not change anything in the configuration, but after the installation of these modules my test script stopped after a couple reboots.
Means to me that probably one of these packages causes a problem to the execution of the iface hotplug script.
Now, you can start bisecting the list to foggier out which package makes this happen.
The reason messages could be missing is because the logger might not be fully operational yet. The more packages you install, the more work the router has to do on startup, the higher probability that the logger starts later sometimes.
The best approach is to use flock and write in your own file: the file system is definitely up and operational when the scripts are invoked.
this might help you... replace / make a new version of /sbin/hotplug-call
#!/bin/sh
export HOTPLUG_TYPE="$1"
. /lib/functions.sh
PATH="/usr/sbin:/usr/bin:/sbin:/bin"
LOGNAME=root
USER=root
export PATH LOGNAME USER
export DEVICENAME="${DEVPATH##*/}"
filterset() {
while read THING; do
case $THING in
"PS"*|"LINENO"*|"HOSTNAME"*) continue; ;;
"USER"*|"TERM"*|"SHLVL"*) continue; ;;
"PPID"*|"PATH"*|"OPTIND"*) continue; ;;
"IFS="*|"N="*|"'") continue; ;;
"PWD="*|"HOME="*|"_C"*) continue; ;;
"LOGNAME="*|"hlogfile"*|"_C"*) continue; ;;
*) echo $THING; ;;
esac
done
}
STMP=`date +%H%M%S`
hlogfile="/tmp/HOTPLUG-sbin-hotplug-call-$HOTPLUG_TYPE"
hlogfilec="/tmp/HOTPLUG-sbin-hotplug-call-cumulative"
STMP=`date +%H%M%S`
if ! ps w | grep '/sbin/logd' | grep -v 'grep'; then STMP="${STMP}-nologd"; fi
echo "sbin-hotplugcall-$STMP $HOTPLUG_TYPE" > /dev/kmsg
case $HOTPLUG_TYPE in
ntp) :; ;;
firewall) echo `date +%H%M%S` ${*} ${DEVPATH##*/} $ACTION INTERFACE:$INTERFACE >> $hlogfilec; ;;
net) echo `date +%H%M%S` ${*} $ACTION $DEVICENAME $DEVPATH >> $hlogfilec; ;;
iface) echo `date +%H%M%S` ${*} $ACTION INTERFACE:$INTERFACE $DEVPATH >> $hlogfilec; ;;
*) set | filterset >> $hlogfile; ;;
esac
echo /sys/$DEVPATH >> $hlogfile.$DEVICENAME.alluevent
cat /sys/$DEVPATH/uevent >> $hlogfile.$DEVICENAME.alluevent
if [ \! -d /etc/hotplug.d/$1 ]; then
echo "sbin-hotplugcall-$STMP $HOTPLUG_TYPE NODIR-/etc/hotplug.d/$1" > /dev/kmsg
echo "sbin-hotplugcall-$STMP $HOTPLUG_TYPE NODIR-/etc/hotplug.d/$1" >> $hlogfilec
fi
if [ \! -z "$1" -a -d /etc/hotplug.d/$1 ]; then
for script in $(ls /etc/hotplug.d/$1/* 2>&-); do (
[ -f $script ] && . $script
); done
fi