Unbound does not resolve local hostnames

colini · January 7, 2023, 3:56pm

Hello,

I've installed and configured unbound in the "parallel dnsmasq" setup described here:

https://github.com/openwrt/packages/blob/master/net/unbound/files/README.md#parallel-dnsmasq

Seems to work except unbound forwards requests for local (unqualified) hostnames to internet nameservers and resolution fails. If I qualify the hostname with the local domain (".lan") then it works:

$ dig u6-lite-closet

; <<>> DiG 9.10.6 <<>> u6-lite-closet
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 27216
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;u6-lite-closet.			IN	A

;; AUTHORITY SECTION:
.			993	IN	SOA	a.root-servers.net. nstld.verisign-grs.com. 2023010700 1800 900 604800 86400

;; Query time: 5 msec
;; SERVER: 192.168.10.1#53(192.168.10.1)
;; WHEN: Sat Jan 07 09:48:20 CST 2023
;; MSG SIZE  rcvd: 118

With ".lan":

$ dig u6-lite-closet.lan

; <<>> DiG 9.10.6 <<>> u6-lite-closet.lan
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 17467
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;u6-lite-closet.lan.		IN	A

;; ANSWER SECTION:
u6-lite-closet.lan.	120	IN	A	192.168.99.10

;; Query time: 6 msec
;; SERVER: 192.168.10.1#53(192.168.10.1)
;; WHEN: Sat Jan 07 09:49:28 CST 2023
;; MSG SIZE  rcvd: 63

dnsmasq DNS is still running on port 1053, and it resolves "local" names w/o qualification.

dnsmasq on port 1053:

$ dig -p 1053 u6-lite-closet    

; <<>> DiG 9.10.6 <<>> -p 1053 u6-lite-closet
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62629
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;u6-lite-closet.			IN	A

;; ANSWER SECTION:
u6-lite-closet.		0	IN	A	192.168.99.10

;; Query time: 6 msec
;; SERVER: 192.168.10.1#1053(192.168.10.1)
;; WHEN: Sat Jan 07 09:51:06 CST 2023
;; MSG SIZE  rcvd: 59

My config file (only edited via Luci)

/etc/config/unbound

config unbound 'ub_main'
	option dns64 '0'
	option domain 'lan'
	option edns_size '1232'
	option extended_stats '0'
	option hide_binddata '1'
	option interface_auto '1'
	option localservice '1'
	option manual_conf '0'
	option num_threads '1'
	option protocol 'default'
	option rate_limit '0'
	option rebind_localhost '0'
	option rebind_protection '1'
	option recursion 'default'
	option resource 'default'
	option root_age '9'
	option ttl_min '120'
	option ttl_neg_max '1000'
	option unbound_control '0'
	option verbosity '1'
	list iface_wan 'wan'
	option enabled '1'
	option validator '1'
	option validator_ntp '1'
	option listen_port '53'
	list iface_trig 'lan'
	list iface_trig 'wan'
	option dhcp_link 'dnsmasq'

config zone 'auth_icann'
	option enabled '0'
	option fallback '1'
	option url_dir 'https://www.internic.net/domain/'
	option zone_type 'auth_zone'
	list server 'lax.xfr.dns.icann.org'
	list server 'iad.xfr.dns.icann.org'
	list zone_name '.'
	list zone_name 'arpa.'
	list zone_name 'in-addr.arpa.'
	list zone_name 'ip6.arpa.'

config zone 'fwd_isp'
	option enabled '0'
	option fallback '1'
	option resolv_conf '1'
	option zone_type 'forward_zone'
	list zone_name 'isp-bill.example.com.'
	list zone_name 'isp-mail.example.net.'

config zone 'fwd_google'
	option enabled '0'
	option fallback '1'
	option tls_index 'dns.google'
	option tls_upstream '1'
	option zone_type 'forward_zone'
	list server '8.8.4.4'
	list server '8.8.8.8'
	list server '2001:4860:4860::8844'
	list server '2001:4860:4860::8888'
	list zone_name '.'

config zone 'fwd_cloudflare'
	option enabled '0'
	option fallback '1'
	option tls_index 'cloudflare-dns.com'
	option tls_upstream '1'
	option zone_type 'forward_zone'
	list server '1.1.1.1'
	list server '1.0.0.1'
	list server '2606:4700:4700::1111'
	list server '2606:4700:4700::1001'
	list zone_name '.'

I'm running OpenWrt 22.03.2, r19803-9a599fee93 on x86.

Any ideas?

Thanks,
Colin

K2DLS · January 7, 2023, 7:51pm

The resolver on the client system needs to know to append your "local" domain to searches.

1 -- Are your DNS servers on the client system set via DHCP?
2 -- Let's see what /etc/resolv.conf looks like on the system in question.

colini · January 7, 2023, 11:50pm

Thanks for taking a look. I have several systems on different OS's. I observed the same behavior on them all.

For these systems the DNS server is set via DHCP on my openwrt router. I have a couple OS's, here are the resolv.conf files

On MacBook:

$ cat /etc/resolv.conf

#
# macOS Notice
#
# This file is not consulted for DNS hostname resolution, address
# resolution, or the DNS query routing mechanism used by most
# processes on this system.
#
# To view the DNS configuration used by this system, use:
#   scutil --dns
#
# SEE ALSO
#   dns-sd(1), scutil(8)
#
# This file is automatically generated.
#
search lan
nameserver 192.168.10.1

Per the comments, the output (truncated) from scutil

$ scutil --dns

DNS configuration

resolver #1
  search domain[0] : lan
  nameserver[0] : 192.168.10.1
  if_index : 16 (en0)
  flags    : Request A records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)

<other stuff removed>

DNS configuration (for scoped queries)

resolver #1
  search domain[0] : lan
  nameserver[0] : 192.168.10.1
  if_index : 16 (en0)
  flags    : Scoped, Request A records
  reach    : 0x00020002 (Reachable,Directly Reachable Address)

On an Ubuntu system directly connected to the router:

$ cat /etc/resolv.conf

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search lan

Like the mac, Ubuntu seems to maintain resolv.conf only as a courtesy. The output from resolvectl:

$ resolvectl status

Global
       Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (ens18)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.30.1
       DNS Servers: 192.168.30.1
        DNS Domain: lan

Link 3 (br-8ac72e8366fd)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 4 (docker0)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

<other veth clipped>

And on the openwrt router itself:

# cat /etc/resolv.conf

search lan
nameserver 127.0.0.1
nameserver ::1

These look right to me. How can I verify that my clients are appending ".lan" correctly? tcpdump or is there an easier way?

Thanks,
Colin

K2DLS · January 8, 2023, 12:02am

Do you have nslookup installed?

What does nslookup u6-lite-closet return?

svrocket · January 8, 2023, 2:31am

I dont have the easy answer for you, but I struggle with similar issues.
I am "expert level admin" of unbound on fedora/rhel. I am evaluating moving to either unbound on openwrt or dnsmasq on openwrt.

I find the behavior to be different, depending on which client binary you are asking the question with.
yet another avenue of difference is traditional /etc/resolv.conf vs systemd's stub-resolv behavior.

Consider:

root@openwrt ll /usr/bin/hostip /usr/bin/resolveip /usr/bin/dig /usr/bin/host /usr/bin/nslookup
-rwxr-xr-x 1 root root 101K Nov 24 2020 /usr/bin/dig
-rwxr-xr-x 1 root root 81K Nov 24 2020 /usr/bin/host
-rwxr-xr-x 1 root root 77K Nov 24 2020 /usr/bin/hostip
lrwxrwxrwx 1 root root 17 Nov 11 2020 /usr/bin/nslookup -> ../../bin/busybox
-rwxr-xr-x 1 root root 4.0K Nov 24 2020 /usr/bin/resolveip

there is also /usr/bin/drill, if you want to play with that.

basically, 4 out 5 behave as expected, and busybox's nslookup is crazy/bad/wrong.
but dont blame it on openwrt - I can say, nslookup behaves in similar ways on fedora/rhel.
host + dig are my go-to, I ignore nslookup these days

Also same point about your choice of opkg installation:
dnsmasq dnsmasq-full (dnsmasq-dhcpv6)
vs unbound (libunbound)

but unbound users definitely want
luci-app-unbound

if you dont have it already. You might be just missing a tick-box in the Luci GUI

K2DLS · January 8, 2023, 1:47pm

@colini Can you also provide the output of:

uci show dhcp.@dnsmasq[0]
uci show network.lan.dns_search

jkool702 · January 8, 2023, 2:57pm

@svrocket & @K2DLS:

I have had unbound up and running on my router for over a year and it works fantastic. Performance is stellar. It is used in recursion mode (not forwarding mode), has DNSSEC enabled, and has a huge adblock block list (with ~600k entries) incorporated with it.

I spent a long time getting it setup and working well (after much trial and error). I was never able to get unbound + dnsmasq (serial or parallel) to work, but unbound + odhcpd works fantastic. It also doesnt cripple unbounds performance, since dnsmasq is...well...not the quickest program.

Here are all (i think) of the relevant config stuff from my setup. A few notes:

Im using the "hybrid UCI" configuration, where the configuration is mostly defined in UCI but then specific unbound configuration options are tweaked / overridden in unbount_{ext,srv}.conf
The router is an exceptionally beefy R9000. For more "standard" consumer-grade routers youll likely want to turn down the recursion strength and the memory usage. Also, unbound is serving 3 lan-side networks: lan, IoT and guest. You may not need IoT or guest.
I compiled the router firmware from source and added some extra goodies (e.g. the subnetcache module, libevent). IIRC, without libevent the "outgoing range" and perhaps the "num queries per thread" are limited to at most 1024, and in practice need to be a bit less that that.

/etc/config/unbound

config unbound 'ub_main'
        option interface_auto '1'
        option hide_binddata '1'
        option listen_port '53'
        option extended_luci '1'
        option localservice '1'
        option dhcp4_slaac6 '1'
        option add_extra_dns '0'
        option num_threads '1'
        option rate_limit '0'
        option rebind_protection '1'
        option rebind_localhost '1'
        option root_age '5'
        option ttl_min '120'
        option ttl_neg_max '1000'
        option validator '1'
        option validator_ntp '1'
        option verbosity '1'
        option enabled '1'
        option extended_stats '1'
        option dhcp_link 'odhcpd'
        option recursion 'default'
        option resource 'default'
        option domain 'lan'
        option domain_type 'static'
        option unbound_control '2'
        option protocol 'ip6_local'
        option manual_conf '0'
        option add_wan_fqdn '0'
        list trigger_interface 'IoT'
        list trigger_interface 'guest'
        list trigger_interface 'lan'
        list trigger_interface 'wan'
        option add_local_fqdn '4'
        option edns_size '1232'
        option dns64 '1'
        list iface_lan 'IoT'
        list iface_lan 'guest'
        list iface_lan 'lan'
        list iface_wan 'wan'
        list iface_wan 'wan6'
        list iface_trig 'IoT'
        list iface_trig 'guest'
        list iface_trig 'lan'
        list iface_trig 'wan'

config zone 'auth_icann'
        option enabled '1'
        option fallback '1'
        option url_dir 'https://www.internic.net/domain/'
        option zone_type 'auth_zone'
        list server 'lax.xfr.dns.icann.org'
        list server 'iad.xfr.dns.icann.org'
        list zone_name '.'
        list zone_name 'arpa.'
        list zone_name 'in-addr.arpa.'
        list zone_name 'ip6.arpa.'

/etc/config/dhcp

config odhcpd 'odhcpd'
        option maindhcp '1'
        option leasefile '/var/lib/unbound/odhcpd/dhcp.leases'
        option leasetrigger '/usr/lib/unbound/odhcpd.sh'
        option loglevel '4'

/etc/unbound/unbound_srv.conf

  so-reuseport: yes
  outgoing-port-avoid: 32400 #; Plex
  outgoing-port-avoid: 1194 #; OpenVPN UDP port
  outgoing-port-avoid: 5060-5061 #; SIP Signaling UDP port
  outgoing-port-avoid: 9001 #; TOR Relay
  outgoing-port-avoid: 9030 #; TOR Relay
  msg-buffer-size: 65552
  outgoing-range: 8192
  num-queries-per-thread: 4096
  outgoing-num-tcp: 32
  incoming-num-tcp: 32
  rrset-cache-size: 32m
  msg-cache-size: 16m
  stream-wait-size: 16m
  key-cache-size: 8m
  neg-cache-size: 2m
  ratelimit-size: 4m
  ip-ratelimit-size: 4m
  http-query-buffer-size: 8m
  http-response-buffer-size: 8m
  infra-cache-numhosts: 16384
  use-caps-for-id: no
  prefetch: yes
  prefetch-key: yes
  rrset-roundrobin: yes
  target-fetch-policy: "-1 4 3 2 1 0"
  minimal-responses: yes
  aggressive-nsec: yes
  disable-dnssec-lame-check: no
  hide-trustanchor: yes
  harden-short-bufsize: no
  harden-large-queries: no
  harden-glue: yes
  harden-below-nxdomain: yes

I think thats all the relevant config. To use DNSSEC youll need to set it up and have unbound-anchor. I want to say to use odhcpd you need unbound-control (and of course the full odhcpd package) too. The readme linked above has good setup info though. Good luck!

jkool702 · January 8, 2023, 4:45pm

@colini I meant to tag you too

svrocket · January 9, 2023, 12:08am

thats cool, jkool. I'll prolly borrow your pasted config.

jkool702 · January 13, 2023, 4:19am

Also, heres a little setup script that I wrote some time ago to get DNSSEC up and running. This should do all the setup needed for DNSSEC.

mkdir -p /etc/unbound/root-anchors
unbound-control-setup -d /etc/unbound
wget https://data.iana.org/root-anchors/icannbundle.pem -O /etc/unbound/root-anchors/icannbundle.pem
wget https://data.iana.org/root-anchors/root-anchors.p7s -O /etc/unbound/root-anchors/root-anchors.p7s
wget https://data.iana.org/root-anchors/root-anchors.xml -O /etc/unbound/root-anchors/root-anchors.xml
wget https://www.internic.net/domain/named.root -O /etc/unbound/root.hints
cp /var/lib/unbound/root.key /etc/unbound
unbound-anchor -a /etc/unbound/root.key -c /etc/unbound/root-anchors/icannbundle.pem -x /etc/unbound/root-anchors/root-anchors.xml -s  /etc/unbound/root-anchors/root-anchors.p7s -r /etc/unbound/root.hints
chown -R unbound:unbound /etc/unbound
service unbound restart

colini · January 15, 2023, 10:07pm

Thanks everyone for taking a look at this. I finally got to look into it again today and it now seems to be working. Some observations:

My setup is the "Parallel dnsmasq" as described in the README file.
You have no hope of getting this to work just clicking in Luci You must read the README page carefully.
The suggested dnsmasq setting list dhcp_option 'option:dns-server,0.0.0.0' is necessary, otherwise dnsmasq does not provide a DNS server or "search" setting to clients. I didn't see a way to do this via Luci, just edit /etc/config/dhcp directly.
After making changes, you may have to fully restart unbound and/or dnsmasq for the integration scripts to work.

Re: my original issue, I now think the problem was "I don't know how to use dig correctly". I didn't realize by default dig does not read /etc/resolv.conf at all, so it will not append a search path to your queries based on that file. You have to specify +search explicitly. Now I know!

Thanks for all the help,
Colin

system · January 25, 2023, 10:08pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.