Netatmo HomeKit native devices, not able to update iOS Home app when isolated (but work online/using Netamo app)

[giuliomagnifico]

@spence

169.254.x.x are self assigned local_only addresses.

https://en.wikipedia.org/wiki/Link-local_address

Commonly used when a host has no valid ip address config or DHCP.
If you changed the network connectivity without causing a physical link loss on the hosts, this could be expected. Simple devices sometimes don't have code to deal with that situation, or perhaps any network change without a reboot.

Try causing a wifi re-association for the Netatmo devices if that is easy or just reboot them.

Yes this is what I was saying, if they don't have an IP, how they can talk? I'll try the reboot when they are inside the lan and I will check if they get a valid

[quote="spence, post:19, topic:149133"]
please do a capture on the WAX206 with this:
tcpdump -ennv -i br-lan
and verify that the IP address on the WAX206, 192.168.1.3 , shows up as a source in some packets[/quote]

Yes but it shows up only in the (current) SSH session with the Mac I'm using, nothing else...

17:48:40.904910 80:cc:9c:eb:8d:37 > f8:e4:3b:a3:5f:54, ethertype IPv4 (0x0800), length 1326: (tos 0x4a,ECT(0), ttl 64, id 25025, offset 0, flags [DF], proto TCP (6), length 1312)
    192.168.1.3.22 > 192.168.1.10.50721: Flags [P.], cksum 0x8870 (incorrect -> 0x4fbc), seq 20940704:20941964, ack 13177, win 1002, options [nop,nop,TS val 3344280471 ecr 547442437], length 1260

@spence

If you can capture some packets "from" the interface on WAX206 directly connected to the LAN then it looks like Avahi is only working in one direction.
I noticed there was no proxied mDNS traffic from 'iot' network on the br-lan capture.
Are there any devices doing mDNS on the 'iot' network? If there are, then maybe avahi is currently only forwarding from 'Netatmo'

It's what I'm trying to say, Avahi is not working as expected, and I think the host inside 'iot' interface are working only because there's the Avahi istance running on Homebridge, because all the queries are sent to them but from them

@spence

I don't have any suggestions but please work on changing the Avahi config / firewall rules so Avahi forwards from 'Netatmo' to 'LAN' as well.
Check with tcpdump -ennv -i br-lan port 5353 should show nDNS traffic from source 192.168.1.3.

I see only traffic from other sources like smart plugs/TV/etcc..and the Homebridge server where Avahi is running and is running fine:

Homebridge server

192.168.1.5.5353 > 224.0.0.251.5353: 0 PTR (QM)? _hap._tcp.local. (33)
18:10:39.713126 e4:5f:01:b3:3a:8d > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 1248: (tos 0x0, ttl 255, id 38789, offset 0, flags [DF], proto UDP (17), length 1234)

Random iot device

192.168.1.109.5353 > 224.0.0.251.5353: 0*- [0q] 3/0/2 Giulios-iPhone.local. (Cache flush) AAAA fe80::109a:cc7a:4eaf:87d5, Giulios-iPhone.local. (Cache flush) A 192.168.1.109, Giulios-iPhone.local. (Cache flush) AAAA fd91:e81d:cbd3:4623:1076:22e4:f83b:8ceb (153)

About the rule... hmmm but all the traffic can already go from 'lan' to 'netatmo', what rule I can try to add other than this that is already in firewall?

config rule
	option name 'LAN to Netatmo'
	option src 'lan'
	option target 'ACCEPT'
	option dest 'Netatmo'
	list proto 'all'

I also have to go out, I'll think on it!

Thanks!

EDIt:

@spence

ps -w

Uhm, Avahi is running as I've always noticed but I noticed only now that assigned to the useless 'nobody' user! This could be the issue (?) no permissions?

19252 nobody    2684 S    avahi-daemon: running [WAX206.local]

[spence]

@giuliomagnifico

I see only traffic from other sources like smart plugs/TV/etcc..and the Homebridge server where Avahi is running and is running fine:

Homebridge server

192.168.1.5.5353 > 224.0.0.251.5353: 0 PTR (QM)? _hap._tcp.local. (33)
18:10:39.713126 e4:5f:01:b3:3a:8d > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 1248: (tos 0x0, ttl 255, id 38789, offset 0, flags [DF], proto UDP (17), length 1234)

Random iot device

192.168.1.109.5353 > 224.0.0.251.5353: 0*- [0q] 3/0/2 Giulios-iPhone.local. (Cache flush) AAAA fe80::109a:cc7a:4eaf:87d5, Giulios-iPhone.local. (Cache flush) A 192.168.1.109, Giulios-iPhone.local. (Cache flush) AAAA fd91:e81d:cbd3:4623:1076:22e4:f83b:8ceb (153)

These are all on the 'lan' network (192.168.1.0/24) and not the 'iot' network (192.168.5.0/24). We know Avahi proxies them to network 'Netatmo'. I am interested to know what mDNS hosts are in network 'iot' that we should see reflected to the other networks.
.
.
.
What I was suggesting is that after adjusting the config so avahi might work better / differently then look for reflected mDNS traffic exiting interfaces on the WAX206. You may have another tool to make that easier. Didn't you show me something like avahi-browse -arp ?
.
.
.

@giuliomagnifico

Uhm, Avahi is running as I've always noticed but I noticed only now that assigned to the useless 'nobody' user! This could be the issue (?) no permissions?

Since Avahi IS working reflecting mDNS from 'Netatmo' to 'lan' I would not think permissions are an issue. Why would "nobody" be allowed read to access of 'Netatmo' and write access to 'lan' but not read access from 'lan' or read/write access to 'iot'.

Are there mDNS / Homekit devices active on 'iot'?

For netstat, try a broader, more inclusive set of options but grep for '5353':
netstat -np |grep 5353
If nothing shows up then run netstat -anp and look for anything interesting. There might be something tied to ubusd or netflow etc. I don't know how avahi runs. Maybe there is revealing info in /etc/rc.d or /etc/init.d.
.
.
.
As for Avahi config, I didn't find much info on openwrt.org/docs/ but from https://linux.die.net/man/5/avahi-daemon.conf I found allow-interfaces=.

A recent posting of your Avahi config does not have a line for
allow-interfaces=
It may be good to add that back to section [server] and leave the list empty.
Try it and look for proxied mDNS, either with 'avahi-browse -arp` or tcpdump.

Have you been watching logread and dmesg for any related messages, especially avahi startup and dhcp events. If you use dnsmasq for dhcp, I think you can turn on event logging for things like unusual client requests in the config file and reload the config.

[spence]

@giuliomagnifico

After investigating I'm starting think that Avahi is not working as expected with these Netatmo devices, I try to explain:

When the Netatmo devices are isolated from the lan, I absolutely can't see any query when I listen from the WAX206 with avahi-browse -arp.

As soon as I bond the Netatmo wireless interface to the br-lan interface, they appear!

I am thinking that it is more likely that avahi is only reflecting from network 'lan' to network 'Netatmo` and not from 'Netatmo' or 'iot' to 'lan'.

To test this, identify a system that is known to be sending mDNS traffic and seen by WAX206 and move it to network 'Netatmo'. Better yet, I saw your iPhone mDNS traffic reflected from 'lan' to 'Netatmo'.

09:24:33.909046 02:0c:43:26:60:30 > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 304: (tos 0x0, ttl 255, id 45888, offset 0, flags [DF], proto UDP (17), length 290)
    192.168.6.1.5353 > 224.0.0.251.5353: 0 [3q] [5n] ANY (QM)? GiulioM-bM-^@M-^Ys iPhone._rdlink._tcp.local. ANY (QM)? Giulios-iPhone.local. ANY (QM)? 70:b3:06:1d:5e:d9@fe80::72b3:6ff:fe1d:5ed9-supportsRP._apple-mobdev2._tcp.local. (262)

09:24:34.160194 02:0c:43:26:60:30 > 01:00:5e:00:00:fb, ethertype IPv4 (0x0800), length 304: (tos 0x0, ttl 255, id 45899, offset 0, flags [DF], proto UDP (17), length 290)
  192.168.6.1.5353 > 224.0.0.251.5353: 
    0 [3q] [5n]
     ANY (QM)? GiulioM-bM-^@M-^Ys iPhone._rdlink._tcp.local.
     ANY (QM)? Giulios-iPhone.local.
     ANY (QM)? 70:b3:06:1d:5e:d9@fe80::72b3:6ff:fe1d:5ed9-supportsRP._apple-mobdev2._tcp.local.
    (262)

(I added some line breaks in the second packet for easy reading)
Did you have the Home app open on your iPhone at around 9:24 this morning?
.
.

Try moving your iPhone to the 'Netatmo' network, verify that it has a good IP address and see if it shows up in avahi-browse and or in tcpdumps.

Also test to see if Avahi is reflecting to 'iot'. I don't think we did that yet. I had that on my todo list from yesterday.

[giuliomagnifico]

Ehy @spence you won't trust me but.... I solved the issue. In a very stupid way, and this is not the first time I have a similar experience with OpenWrt (I also wrote it before, about bounding/assigning an interface to a device, example.

I was simply searching a way to allow the traffic rom netatmo to lan as you wrote:

I don't have any suggestions but please work on changing the Avahi config / firewall rules so Avahi forwards from 'Netatmo' to 'LAN' as well.

And I changed this, just for curiosity (since the Netatmo zone is already assigned to the wl0-ap0 interface):

That is the same as add list device

config zone
	option name 'Netatmo'
	option output 'ACCEPT'
	list network 'Netatmo'
	option forward 'ACCEPT'
	option input 'REJECT'
	list device 'wl0-ap1'

My previous setting was

config zone
	option name 'Netatmo'
	option output 'ACCEPT'
	list network 'Netatmo'
	option input 'REJECT'
	option forward 'REJECT'

This fixed all.

I have no idea of what is the difference in doing this, I mean assign a zone to an interface or specify the interface from a zone, but that is all.

Now here in Italy are the 11PM so I'm to tired to think to something else rather than sleep

Tomorrow I'll reply to your post, I'll turn off the netatmo interface and I'll put the Netatmo devices inside the iot WLAN, do some tcpdump, I'll investigate better, etc...

That's absurd... I spent one week just to add a stupid option to assign a stupid

list device 'wl0-ap1'

to a firewall zone that is already assigned to the 'wl0-ap0` interface.

That emoticon explain my mood!

I don't know if it's a weird behviour with LuCi or what else, but is way faster and simpler create a new firewall zone when you create the new interface with LuCi.... but if this don't/will not work, then is pointless, you still have to go inside the firewall and specify to which device this zone must be assigned.

Anyway I learned lots of new things/stuff doing this debug, thanks to you! So I'm happy to fixed it in this "stupid way"

[spence]

Yay!!!! Congratulations!!

[spence]

@giuliomagnifico

Ehy @spence you won't trust me but.... I solved the issue. In a very stupid way, and this is not the first time I have a similar experience with OpenWrt (I also wrote it before, about bounding/assigning an interface to a device, example.

I was simply searching a way to allow the traffic rom netatmo to lan as you wrote:

I don't have any suggestions but please work on changing the Avahi config / firewall rules so Avahi forwards from 'Netatmo' to 'LAN' as well.

And I changed this, just for curiosity (since the Netatmo zone is already assigned to the wl0-ap0 interface):

I believe it! I am not surprised that it is something like that. I was working with you to find the point where it breaks so you could focus on finding a solution for a specific problem.

Your solution looks unusual to me. I never used swconfig so I only had to learn the DSA way and other "new" config styles and did not need to unlearn old openwrt ways. It was interesting learning openwrt after 26 years of cisco and other vendors before that.
.
.
.
I'm sure it is already your plan but after testing in network 'iot', you can set mDNS reflection to be limited to 'lan' and 'iot' so 'guest' can not see nor serve records, unless you want that to work for screen casting to your tv or something.

Assuming that the rest of your setup for iot isolation goes smoothly, feel free to bring up anything you mentioned earlier that I did not address to your satisfaction or new things as well.

[giuliomagnifico]

@spence

I believe it! I am not surprised that it is something like that. I was working with you to find the point where it breaks so you could focus on finding a solution for a specific problem.

Yes thanks again!

@spence

Your solution looks unusual to me.

...don't tell it to me... this solution looks absurd, because the Netatmo interface was already assigne to the Netamo zone of the firewall. So why add a device to a firewall zone if an interface is assigned to the same zone?

@spence

I never used swconfig so I only had to learn the DSA way and other "new" config styles and did not need to unlearn old openwrt ways. It was interesting learning openwrt after 26 years of cisco and other vendors before that.

But this is a DSA router/AP, not swconfig, with my old swconfig things were "more logical", is the second time that this DSA config is incomprensibile to me. But that's the (new) way! And I like learn new things

@spence

I'm sure it is already your plan but after testing in network 'iot', you can set mDNS reflection to be limited to 'lan' and 'iot' so 'guest' can not see nor serve records, unless you want that to work for screen casting to your tv or something.

Yes thanks, already done, after assignign the same wireless interface to the 'iot' firewall zone, I've put the Netatmo device inside the 'iot' zone, now they're working as expected.

@spence

Assuming that the rest of your setup for iot isolation goes smoothly, feel free to bring up anything you mentioned earlier that I did not address to your satisfaction or new things as well

Oh thanks, at the moment everything is perfect, the only next step I'm planning is to add the TV to a VLAN, because it's wired connected to the hub, but I have to chose if I have to use the Netgear hub to make a untagged VLAN or a tagged one in the R4S I don't like both ways, the interface in the Netgear witch is ugly and I don't like the tagged VLANs but since I use the TV for airplay content sharing, maybe is better leave it inside the lan with other devices to avoid further messes

I'll write a post on my blog for this issue, maybe could be useful for someone else, since it was a very weird and not logical solution!

Edit: another weird behavior

I'm cleaning up the Netatmo interface, zone, etcc... but I discovered that if I don't specify also the subnet IP inside the firewall (not only the attached device), the setup isn't working.

config zone
	option name 'iot'
	option output 'ACCEPT'
	option input 'REJECT'
	list device 'wl0-ap0'
	list network 'iot'
	option forward 'REJECT'
	list subnet '192.168.5.0/24'

With the Netatmo zone it was working without specifying anything... becuse the wl0-ap0 device is already assigned to the iot zone that has 192.168.5.0/24 has IP. I have no idea...

[Finn]

Hi Giulio.
just picked up this thread from our wax206 thread and haven't gone through the entire post - yet.
FYI
There shall be no firewall settings/setup on the AP (and no dhcp or dns) - these things You can disable in system->startup - those services shall be handled by the router only

[giuliomagnifico]

Hi Finn, thanks for the reply, I solved the issues yesterday, you can read only the last post: Netatmo HomeKit native devices, not able to update iOS Home app when isolated (but work online/using Netamo app) - #24 by giuliomagnifico

Anyway yes, use the firewall on the router only is another way but I have to create lots of tagged VLANs and the router should handle more work. Instead using the firewall access point I demand some work to it and I can have another layer of security.

Now my network is made in this way, with 3 isolated subnets and no VLANs

[Finn]

I think I recall You as having a Nanopi R4S as Your router - it's piece of cake to run many vlans for that device.
I am running 3 vlans (found out how to set it up from reading Your posts amongst others) - lan, IOT and game and yet another vlan will be added soon.

My setup: WAN -> NanopiR4S (OpenWrt) -> Netgear 308Tgs (Openwrt) - Netgear wax206 AP (Openwrt)

Most security is in the firewall (router) and in here I control what which vlan can regarding access to the internet - Lan full access to wan and IOT, IOT only outgoing access to wan no access to Lan same with Game (no one can access IOT or game from WAN) but that requires me to allow dhcp and dns requests (in traffic rules) so IOT and Game can receive information when needed and maybe this may have caused Your issues not getting any ip's ? just a guess

Ciao

[giuliomagnifico]

Yes I also have the small great R4S (that also @spence has IIRC). And thanks. We're on the same exact hardware, except that I have another Netgear switch (very similar, the GS108E)

Your config is perfect, whatever to use VLAN or subnet is a personal choice. Both have advantages and disadvantages! My network is similar to your, only using subnets, by the way my issue wasn’t the IPs/DHCP, was something related to specific explain every subnet and device attached to the a firewall zone. Also if it’s already selected an interface that is assigned to the same subnet/device. And I had to specify a new route and gateway for the subnets to the R4S to route the mDNS packets for the Homebridge server… just a mess for Apple stuff

Anyway I’m writing a new post blog on my actual home setup, because I made some hardware changes, like two big fans inside the """server cabinet* """ I will write also the details about this issue and the configuration of the WAX206.

The R4S is rugged/dust-proof

[spence]

@giuliomagnifico

this solution looks absurd, because the Netatmo interface was already assigne to the Netamo zone of the firewall. So why add a device to a firewall zone if an interface is assigned to the same zone?

Did you reboot your WAX206 after adding or making network or firewall changes for iot or Netatmo? When I was working on adding several VLANs / L3 networks to my setup, I found that the added setup didn't work after a commit. Even reloading the firewall config and reloading the network config didn't help. Restarting network service didn't help either. Rebooting the device got the new config working each time. I repeated this a couple of times adding new networks but did not try to analyze the root issue.

@giuliomagnifico

Edit: another weird behavior

I'm cleaning up the Netatmo interface, zone, etcc... but I discovered that if I don't specify also the subnet IP inside the firewall (not only the attached device), the setup isn't working.

config zone
	option name 'iot'
	option output 'ACCEPT'
	option input 'REJECT'
	list device 'wl0-ap0'
	list network 'iot'
	option forward 'REJECT'
	list subnet '192.168.5.0/24'

With the Netatmo zone it was working without specifying anything... becuse the wl0-ap0 device is already assigned to the iot zone that has 192.168.5.0/24 has IP. I have no idea...

.
.
I'll share the basics of my working setup in case it sparks an idea for a change to your setup. I am not trying to get you to use VLANs. Just sharing my working config that does not add list device or list subnet to my zone stanzas.
I use VLANs in my config. My wired interfaces are defined in a bridge and my 5 L3 'lan' side networks/interfaces all follow this strategy:

In my /etc/config/network file, each config bridge-vlan stanza defines a 'vlan' which is also a virtual device:

config bridge-vlan
	option device 'sw_switch'
	option vlan '51'
	list ports 'eth1:t'

In my /etc/config/network file, each config interface stanza defines a "L3 network" and maps it to a virtual device for the vlan.

config interface 'vl51_guest'
	option proto 'static'
	option device 'sw_switch.51'
	list ipaddr '192.168.51.1/24'

To add a specific wifi SSID to be part of the 'network',( and zone by way of the config interface stanza), the option network line in the config wifi-iface stanza in the /etc/config/wireless file defines that binding:

config wifi-iface 'wifinet10'
	option device 'radio0'
	option mode 'ap'
	option encryption <redacted>
	option isolate '1'
	option key <redacted>
	option ieee80211w <redacted>
	option wpa_disable_eapol_key_retries '1'
	option network 'vl51_guest'
	option ssid 'Spence_guest'
	option ifname 'wl5g-vl51-gu'

I currently only have one network per zone but I did successfully test with multiple "untrusted" networks in a single zone.
In my /etc/config/firewall file, The 'list network' line is the reference to the config interface label in the network file to bind everything in that network to the zone.

config zone
	option output 'ACCEPT'
	option input 'REJECT'
	option forward 'ACCEPT'
	option name 'guest'
	list network 'vl51_guest'

config forwarding
	option src 'guest'
	option dest 'wan'

config rule
	option name 'Allow-Guest-input-DHCP'
	option src 'guest'
	option dest_port '67'
	option target 'ACCEPT'

config rule
	option name 'Allow-Guest-input-ICMP'
	list proto 'icmp'
	option src 'guest'
	option target 'ACCEPT'
	list dest_ip '192.168.51.1'

config rule
	list proto 'udp'
	option src 'guest'
	list dest_ip '192.168.51.1'
	option target 'ACCEPT'
	option name 'Allow-Guest-input-DNS'
	option dest_port '53'

.
.
.
If I remember to, I will try adding a wifi SSID to a separate IP interface/network in its own zone without involving any wired ports or dot1q VLANs and see if I run into a similar issue to yours but today I am going outside for a long walk to enjoy this warm winter afternoon.

Happy OpenWrt-ing!

[giuliomagnifico]

@spence

Did you reboot your WAX206 after adding or making network or firewall changes for iot or Netatmo? When I was working on adding several VLANs / L3 networks to my setup, I found that the added setup didn't work after a commit. Even reloading the firewall config and reloading the network config didn't help. Restarting network service didn't help either. Rebooting the device got the new config working each time. I repeated this a couple of times adding new networks but did not try to analyze the root issue

Yes I noticed this too, only a bit late maybe, less frequent for the firewall (generally a restart of it was working), but very often for the network. And that's weird because I also was restarting every time after a change the network. Maybe there's something like a "kernel cache of the routing" for the DSA devices?

@spence

I'll share the basics of my working setup in case it sparks an idea for a change to your setup. I am not trying to get you to use VLANs. Just sharing my working config that does not add list device or list subnet to my zone stanzas.
I use VLANs in my config. My wired interfaces are defined in a bridge and my 5 L3 'lan' side networks/interfaces all follow this strategy

My old setup, with the R7800 swconfig, was working without all these 'list this/that', now I have also to add the subnet of an interface that HAS the same subnet. I don't know, had I known earlier I would have started at the beginning with the VLANs, now I'm planning to rebuild all again with a VLAN from the router to the AP. Anyway I can also add now a VLAN, keep the subnets but remove the firewall zones/rules on the AP, and create the rules for the VLAN on the R4S/router.

Thanks for posting your setup, since I can use it as inspiration for this (why are you forwarding the ICMP on the Guest? )

@spence

If I remember to, I will try adding a wifi SSID to a separate IP interface/network in its own zone without involving any wired ports or dot1q VLANs and see if I run into a similar issue to yours but today I am going outside for a long walk to enjoy this warm winter afternoon.

Yes you can try, have fun and in case let me know what happens.

@spence

Happy OpenWrt-ing!

You too and thanks again for all the help again!