Adding a 2nd Pihole to DNSJack

I've had the DNSHIJACKv4 previously posted here working fine for sometime. I've long wanted to get a 2nd DNS server running for redundancy, every now and then the Pi has glitched and everything stops until I figure it out. Long story short I recently upgraded my NAS so I can run a Docker instance of Pihole. The original was 192.168.70.250, the new is .251 it's resolving okay albeit most traffic still goes through #1. However, I can see from the log on #1 that queries made to #2 are being redirected to #1 by the router.
Here's my hijack rules

# DNSHIJACKv4 start of section ####
# Log and hijack to Pihole
iptables -t nat -N dnshijack
#iptables -t nat -I dnshijack -j LOG --log-prefix "dnshijack4 "
iptables -t nat -A dnshijack -j DNAT --to-destination 192.168.70.250
# allow Piholes to query internet
iptables -t nat -A dnshijack -j DNAT --to-destination 192.168.70.250
# allow Pihole1 to query internet
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p udp --dport 53 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p tcp --dport 53 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p udp --dport 853 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p tcp --dport 853 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
#
iptables -t nat -A dnshijack -j DNAT --to-destination 192.168.70.251
# allow Pihole2 to query internet
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p udp --dport 53 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p tcp --dport 53 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p udp --dport 853 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p tcp --dport 853 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
# allow queries to Router, this is where the Hijack goes
iptables -t nat -A prerouting_lan_rule -p tcp --dport 53 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -p udp --dport 53 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT -j ACCEPT
iptables -t nat -A prerouting_lan_rule -p tcp --dport 853 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT -j ACCEPT
iptables -t nat -A prerouting_lan_rule -p udp --dport 853 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT -j ACCEPT
# anything else is hijacked
iptables -t nat -A prerouting_lan_rule -p udp --dport 53 -m comment --comment "HiJack DNS" -j dnshijack
iptables -t nat -A prerouting_lan_rule -p tcp --dport 53 -m comment --comment "HiJack DNS" -j dnshijack
iptables -t nat -A prerouting_lan_rule -p udp --dport 853 -m comment --comment "HiJack DNS" -j dnshijack
iptables -t nat -A prerouting_lan_rule -p tcp --dport 853 -m comment --comment "HiJack DNS" -j dnshijack
# fix "reply from unexpected source"
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p tcp -m tcp --dport 53 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p udp -m udp --dport 53 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p tcp -m tcp --dport 853 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p udp -m udp --dport 853 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
# DNSHIJACKv4 end of section ####

I added the 4 lines with MAC 02:42:c0:a8:46:fb which I've confirmed with both arp and wireshark is that for the 2nd Pihole. I suspect obviously I have missed something here. I realise too I should probably modify the last section somehow to fix unexpected replies having just pasted it in

From the iptables man page:

In your case, every hijacked packet will be redirected to 192.168.70.250 because it will match the first rule in the dnshijack chain. If the posted configuration is correct, that rule should be even duplicated.

Run iptables -t nat -nvL dnshijack to check the rules' hits. You should see some only on the first one.

If your idea is to create some kind of load balancing, you could use the iptables statistic module as discussed here.

You should create post routing rules for the Pi-hole2 as well.

Thanks. As you say when checking there are no hits for the second rule. It was not necessarily load balancing but that I wanted a degree of redundancy having had a few outages where the existing Pihole has failed. I guess load balancing would provide that in that the client will try the other if no response from the first. I have changed the hijack so that it is as follows

# DNSHIJACKv4 start of section ####
# Log and hijack to Pihole
iptables -t nat -N dnshijack
# allow Piholes to query internet
iptables -t nat -A dnshijack -j DNAT --to-destination 192.168.70.250
#
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p udp --dport 53 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p tcp --dport 53 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p udp --dport 853 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source dc:a6:32:91:0c:51 -p tcp --dport 853 -m comment --comment "Allow Pi-Hole Ethernet" -j ACCEPT
#
iptables -t nat -A dnshijack -j DNAT --to-destination 192.168.70.251
# allow Pihole2 to query internet
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p udp --dport 53 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p tcp --dport 53 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p udp --dport 853 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -m mac --mac-source 02:42:c0:a8:46:fb -p tcp --dport 853 -m comment --comment "Allow Pi-Hole2 Ethernet" -j ACCEPT
# allow queries to Router, this is where the Hijack goes
iptables -t nat -A prerouting_lan_rule -p tcp --dport 53 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT
iptables -t nat -A prerouting_lan_rule -p udp --dport 53 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT -j ACCEPT
iptables -t nat -A prerouting_lan_rule -p tcp --dport 853 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT -j ACCEPT
iptables -t nat -A prerouting_lan_rule -p udp --dport 853 -d 192.168.70.1 -m comment --comment "Allow Router" -j ACCEPT -j ACCEPT
# anything else is hijacked
iptables -t nat -A prerouting_lan_rule -p udp --dport 53 -m comment --comment "HiJack DNS" -j dnshijack
iptables -t nat -A prerouting_lan_rule -p tcp --dport 53 -m comment --comment "HiJack DNS" -j dnshijack
iptables -t nat -A prerouting_lan_rule -p udp --dport 853 -m comment --comment "HiJack DNS" -j dnshijack
iptables -t nat -A prerouting_lan_rule -p tcp --dport 853 -m comment --comment "HiJack DNS" -j dnshijack
# fix "reply from unexpected source"
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p tcp -m tcp --dport 53 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p udp -m udp --dport 53 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p tcp -m tcp --dport 853 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.250 -p udp -m udp --dport 853 -m comment --comment "DNS Pi-hole MASQUERADE" -j MASQUERADE
# Pihole 2
iptables -t nat -A postrouting_lan_rule -d 192.168.70.251 -p tcp -m tcp --dport 53 -m comment --comment "DNS Pi-hole2 MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.251 -p udp -m udp --dport 53 -m comment --comment "DNS Pi-hole2 MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.251 -p tcp -m tcp --dport 853 -m comment --comment "DNS Pi-hole2 MASQUERADE" -j MASQUERADE
iptables -t nat -A postrouting_lan_rule -d 192.168.70.251 -p udp -m udp --dport 853 -m comment --comment "DNS Pi-hole2 MASQUERADE" -j MASQUERADE
# DNSHIJACKv4 end of section ####

and then added to the /etc/config/firewall

config redirect
        option target 'DNAT'
        option name 'Pihole'
        option src 'lan'
        option src_dport '53'
        option dest_ip '192.168.70.250'
        option dest_port '53'
        option reflection '0'
        option dest 'lan'
        option extra '-m statistic --mode nth --every 2 --packet 0'
        list proto 'tcp'
        list proto 'udp'
        option src_ip '192.168.70.0/24'
        
config redirect
        option target 'DNAT'
        option name 'Pihole2'
        option src 'lan'
        option src_dport '53'
        option dest_ip '192.168.70.251'
        option dest_port '53'
        option reflection '0'
        option dest 'lan'
        list proto 'tcp'
        list proto 'udp'
        option src_ip '192.168.70.0/24'

Obviously restarted firewall but still everything goes to the first instance. If I query #2 directly from my desktop with nslookup bbc.co.uk 192.168.70.251 I get the response from #2 but the log on #1 shows the query redirected there and

Chain dnshijack (4 references)
 pkts bytes target     prot opt in     out     source               destination         
   20  1307 DNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0            to:192.168.70.250
    0     0 DNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0            to:192.168.70.251

Seems everything is going to #1 not sure what it is I have missed, I checked and the extra iptables package already installed.

I took the opportunity today to disable the existing Pihole and a lot of things are fine, PC mobile etc, however the devices which insist on disregarding the advertised DNS such as Chromecast are failing as the Hijack doesn't send to #2 so I have some redundancy.

These rules will be inserted into the zone_lan_prerouting chain, but you have other rules in prerouting_lan_rule chain that take precedence.

Remove the rules created in /etc/config/firewall.
Verify that the iptables-mod-ipopt package is installed.
Make your custom rules look like this

iptables -t nat -N dnshijack
iptables -t nat -A dnshijack -m statistic --mode nth --every 2 --packet 0 -j DNAT --to-destination 192.168.70.250
iptables -t nat -A dnshijack -j DNAT --to-destination 192.168.70.251

and check if the statistics module is used in the first rule

Chain dnshijack (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0            statistic mode nth every 2 to:192.168.70.250
    0     0 DNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0            to:192.168.70.251

It's simple load balancing without failover.
The requests will be distributed between the two Pi-holes, but if one of them is down, the client will get a connection timed out message.

2 Likes

Okay done that.

Chain dnshijack (4 references)
 pkts bytes target     prot opt in     out     source               destination         
    6   334 DNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0            statistic mode nth every 2 to:192.168.70.250
    5   274 DNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0            to:192.168.70.251

Looks like it is doing as expected.

Thanks for your help.
I was thinking that a client would retry on failure and would hopefully then get the other server. I am home alone on Saturday so can make some experiments without upsetting my wife. In any event having redundancy should hopefully stop the total failure which is a nuisance especially if I am not home.

1 Like

Got me scratching my head still. Now that I had a second Pihole up it gave me the chance to update the original to the latest Pi OS and check that I did really have redundancy. As soon as I disable the second Pihole the first can no longer resolve a query. Previously #1 was getting about 2/3 of the total traffic. I am going to let it run for 24 hours and see f that's still the case, as far as I know the new install is functionally identical to the old.

EDIT - I'll leave this in case it helps anyone else. It's an issue with unbound & Debian 11 this thread and the linked one explain it.