Https-dns-proxy not working on latest snapshots/22.03.1

Hi,

As the title says https-dns-proxy doesn't seem to be working on the latest snapshots.
I upgraded this morning to (i think) r20877 and couldn't find any websites. By stopping https-dns-proxy i got his working again.
A short time later R20885 was available which i installed. Again no website could be found.
Now back to R20775 which just works.

BTW device is RT3200.

Yeah it's related to wolfssl fixes (see advisory thread)

1 Like

Thanks, didn't know that.
Thought that was all settled after the libwolfssl5.5.1 update.

I was hoping that this package was working again under 22.03.1 and latest snapshot (R20932), but it still seems to be broken.
Is there a fix in the make?

1 Like

According to this: OpenWrt 22.03.1 first service release - #27 by stangri

It is working fine, but I am having the same problem with 22.03.1 and https-dns-proxy not working. I'm considering a factory reset to see if that helps.

I'm on RPi4 and even on 22.03.1 https-dns-proxy isn't working. Even on snapshot builds. See this thread

Yeah, it is definitely not working fine, even with out-of-box defaults. Not sure why some seem to not have the problem.

@stangri Is there anything we can do to help troubleshoot this? There's at least a dozen or so of us observing this. It's broken even with fully default settings (of https-dns-proxy at least), and broke sometime at or after the wolfssl update. It doesn't crash, it just doesn't do anything. The processes run, logread looks fine, dmesg is clear...I'm not sure where to look now.

Even more oddly, dig will return a valid response, but ping will hang...on an uncached lookup. (This isn't on-router, I'm talking about dnsmasq clients.) I see a TON of queries and returns going out over 443, so that part seems to be working, but the answer never gets back to the client...it just times out.

Reddit thread here: https://www.reddit.com/r/openwrt/comments/y1iyd7/help_httpsdnsproxy_isnt_working_anymore/

+1 broken for me since upgrade to OpenWrt 22.03.2 Xiaomi Redmi Router AX6S

Yes!

  1. Do the curl/opkg still work with https requests?
  2. Have you tried setting verbosity to 2 and seeing what happens in the log?
  3. Are there https-dns-proxy instances listed in ubus call service list output?
  4. Where's the log of https-dns-proxy starting?
  5. What's the network config on the client you're testing from?

I build my own images, so I've upgraded to 23.02.1 without keeping old settings and I don't have any problems with https-dns-proxy, I'm guessing something may have happend to underlying curl library during upgrade.

As to help:
I downgraded to 22.03.1. Only extra packages installed are (luci-)https-DNS-proxy and luci-app-statistics.
https-DNS-proxy is in standard setup (Cloudfare and Google) and running.

  1. I don't know how to test this. But "auc" and "opkg update" fails.
root@RT3200_Router:~# auc
auc/0.3.1-1
Server:    https://sysupgrade.openwrt.org
Running:   22.03.1 r19777-2853b6d652 on mediatek/mt7622 (linksys,e8450-ubi)
No data available (61)
root@RT3200_Router:~# opkg update
Downloading https://downloads.openwrt.org/releases/22.03.1/targets/mediatek/mt7622/packages/Packages.gz
Failed to send request: Operation not permitted
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.1/targets/mediatek/mt7622/packages/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/base/Packages.gz
Failed to send request: Operation not permitted
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/base/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/luci/Packages.gz
Failed to send request: Operation not permitted
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/luci/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/packages/Packages.gz
^[[DFailed to send request: Operation not permitted
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/packages/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/routing/Packages.gz
Failed to send request: Operation not permitted
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/routing/Packages.gz

Downloading https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/telephony/Packages.gz
Failed to send request: Operation not permitted
*** Failed to download the package list from https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/telephony/Packages.gz

Collected errors:
 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.1/targets/mediatek/mt7622/packages/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/base/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/luci/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/packages/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/routing/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

 * opkg_download: Failed to download https://downloads.openwrt.org/releases/22.03.1/packages/aarch64_cortex-a53/telephony/Packages.gz, wget returned 4.
 * opkg_download: Check your network settings and connectivity.

  1. What and where to set this parameter?
  2. Yes, i can see 2 instances (i edited the output and left only dnsmasq and dns-proxy).
/etc/config$ ubus call service list
{
	"dnsmasq": {
		"instances": {
			"cfg01411c": {
				"running": true,
				"pid": 10312,
				"command": [
					"/usr/sbin/dnsmasq",
					"-C",
					"/var/etc/dnsmasq.conf.cfg01411c",
					"-k",
					"-x",
					"/var/run/dnsmasq/dnsmasq.cfg01411c.pid"
				],
				"term_timeout": 5,
				"respawn": {
					"threshold": 3600,
					"timeout": 5,
					"retry": 5
				},
				"jail": {
					"name": "dnsmasq",
					"procfs": false,
					"sysfs": false,
					"ubus": true,
					"log": true,
					"ronly": false,
					"netns": false,
					"userns": false,
					"cgroupsns": false,
					"console": false
				},
				"mount": {
					"/bin/ubus": "0",
					"/etc/TZ": "0",
					"/etc/dnsmasq.conf": "0",
					"/etc/ethers": "0",
					"/etc/group": "0",
					"/etc/hosts": "0",
					"/etc/passwd": "0",
					"/tmp/dhcp.leases": "1",
					"/tmp/dnsmasq.d": "0",
					"/tmp/hosts": "0",
					"/usr/bin/jshn": "0",
					"/usr/lib/dnsmasq/dhcp-script.sh": "0",
					"/usr/share/dnsmasq/dhcpbogushostname.conf": "0",
					"/usr/share/dnsmasq/rfc6761.conf": "0",
					"/usr/share/dnsmasq/trust-anchors.conf": "0",
					"/usr/share/libubox/jshn.sh": "0",
					"/var/etc/dnsmasq.conf.cfg01411c": "0",
					"/var/run/dnsmasq/": "1"
				}
			}
		}
	},
	"https-dns-proxy": {
		"instances": {
			"instance1": {
				"running": true,
				"pid": 10354,
				"command": [
					"/usr/sbin/https-dns-proxy",
					"-r",
					"https://cloudflare-dns.com/dns-query",
					"-a",
					"127.0.0.1",
					"-p",
					"5054",
					"-b",
					"1.1.1.1,1.0.0.1",
					"-4",
					"-u",
					"nobody",
					"-g",
					"nogroup"
				],
				"term_timeout": 5,
				"data": {
					"firewall": [
						{
							"type": "redirect",
							"target": "DNAT",
							"src": "lan",
							"proto": "tcp udp",
							"src_dport": "53",
							"dest_port": "53",
							"reflection": false
						},
						{
							"type": "rule",
							"src": "lan",
							"dest": "*",
							"proto": "tcp udp",
							"dest_port": "853",
							"target": "REJECT"
						}
					]
				},
				"respawn": {
					"threshold": 3600,
					"timeout": 5,
					"retry": 5
				}
			},
			"instance2": {
				"running": true,
				"pid": 10355,
				"command": [
					"/usr/sbin/https-dns-proxy",
					"-r",
					"https://dns.google/dns-query",
					"-a",
					"127.0.0.1",
					"-p",
					"5053",
					"-b",
					"8.8.8.8,8.8.4.4",
					"-4",
					"-u",
					"nobody",
					"-g",
					"nogroup"
				],
				"term_timeout": 5,
				"respawn": {
					"threshold": 3600,
					"timeout": 5,
					"retry": 5
				}
			}
		}
	},
}
  1. I only see these two lines in the log:
Sun Oct 16 11:00:34 2022 user.notice https-dns-proxy: Starting service ✓✓
Sun Oct 16 11:00:40 2022 user.notice https-dns-proxy: Stopping service ✓
  1. I don't know exactly what you're asking. I have 2 RT3200. One as router, the other as AP/switch. Both are connected via copper.My computer is connected to the AP/switch. This is the windows 11 (22H2) config of the network.
Link speed (Receive/Transmit):	1000/1000 (Mbps)
IPv6 address:	xxxx:xxxx:xxxx:xxxx::100
xxxx:xxxx:xxxx:xxxx:ddff:8547:8b01:2de0
xxxx:xxxx:xxxx::100
xxxx:xxxx:xxxx:0:5a15:32c9:612f:2c6c
Link-local IPv6 address:	xxx::xxxx:xxxx:120f:7e4e%11
IPv6 DNS servers:	xxxx:8e20:91a5::1 (Unencrypted)
IPv4 address:	192.168.1.100
IPv4 DNS servers:	192.168.1.1 (Unencrypted)
Primary DNS suffix:	lan
Manufacturer:	Intel
Description:	Intel(R) I211 Gigabit Network Connection
Driver version:	13.0.14.0
Physical address (MAC):	xx-xx-xx-xx-xx-xx

Please let me know if you need more tests.

My problem is on a Belkin RT3200 as well, maybe there's something specific to this device or a driver it uses that is tripping it up?

Not sure if there's anything in particular that's common between this and the other reported devices in this thread?

How exactly and what from?

One way is to run curl https://openwrt.org, but given the logs you have provided, it already looks like the installed libcurl doesn't support https requests. I don't know how you ended up with this issue.

It may be helpful if you have provided the output of opkg list-installed.

  1. Yes
  2. In debug, it fills up the log REALLY fast, but I do see this repeated over and over and over and over and over:

Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [W] 1665925206.811833 https_client.c:351 7A96: curl request failed with 0: No error
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [W] 1665925206.811839 https_client.c:353 7A96: curl error message: Error reading ca cert file F��� - mbedTLS: (- 0x3E00) PK - Read/write of file failed
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [W] 1665925206.811843 https_client.c:380 7A96: No response (probably connection has been closed or timed out)
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811847 https_client.c:435 7A96: CURLINFO_NUM_CONNECTS: 1
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811849 https_client.c:447 7A96: CURLINFO_EFFECTIVE_URL: https://dns.google/dns-query
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811852 https_client.c:482 7A96: Times: 0.000064, 0.002606, 0.000000, 0.000000, 0.000000, 0.002715
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [I] 1665925206.811862 https_client.c:504 7A96: Response was faulty, skipping DNS reply.
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811865 main.c:84 Received response for id: 7A96, len: 0
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811952 https_client.c:205 5E99: * Error reading ca cert file F��� - mbedTLS: (-0x3E00) PK - Read/ write of file failed
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811969 https_client.c:587 Released used io event: 0xffffcc08c0d0
Sun Oct 16 09:00:06 2022 daemon.info https-dns-proxy[5693]: [D] 1665925206.811991 https_client.c:112 curl closed socket: 9

which almost looks like a NULL pointer to the CA cert (which are installed).

  1. Yes (with the verbosity flag removed again):
   "https-dns-proxy": {
           "instances": {
                   "instance1": {
                           "running": true,
                           "pid": 6679,
                           "command": [
                                   "/usr/sbin/https-dns-proxy",
                                   "-r",
                                   "https://cloudflare-dns.com/dns-query",
                                   "-a",
                                   "127.0.0.1",
                                   "-p",
                                   "5054",
                                   "-b",
                                   "1.1.1.1,1.0.0.1",
                                   "-4",
                                   "-u",
                                   "nobody",
                                   "-g",
                                   "nogroup"
                           ],
                           "term_timeout": 5,
                           "data": {
                                   "firewall": [
                                           {
                                                   "type": "redirect",
                                                   "target": "DNAT",
                                                   "src": "lan",
                                                   "proto": "tcp udp",
                                                   "src_dport": "53",
                                                   "dest_port": "53",
                                                   "reflection": false
                                           },
                                           {
                                                   "type": "rule",
                                                   "src": "lan",
                                                   "dest": "*",
                                                   "proto": "tcp udp",
                                                   "dest_port": "853",
                                                   "target": "REJECT"
                                           }
                                   ]
                           },
                           "respawn": {
                                   "threshold": 3600,
                                   "timeout": 5,
                                   "retry": 5
                           }
                   },
                   "instance2": {
                           "running": true,
                           "pid": 6680,
                           "command": [
                                   "/usr/sbin/https-dns-proxy",
                                   "-r",
                                   "https://dns.google/dns-query",
                                   "-a",
                                   "127.0.0.1",
                                   "-p",
                                   "5053",
                                   "-b",
                                   "8.8.8.8,8.8.4.4",
                                   "-4",
                                   "-u",
                                   "nobody",
                                   "-g",
                                   "nogroup"
                           ],
                           "term_timeout": 5,
                           "respawn": {
                                   "threshold": 3600,
                                   "timeout": 5,
                                   "retry": 5
                           }
                   }
           }
   },
  1. Without verbose logging, it's pretty quiet:

Sun Oct 16 09:10:52 2022 user.notice https-dns-proxy: Starting service ✓✓
Sun Oct 16 09:11:04 2022 user.notice https-dns-proxy: Stopping service ✓

  1. Tried from both servers with static IPs and DHCP clients, both on the same subnet as the router.

This is a bug in upstream package which was fixed about 20 hours ago: CURL stopped working for https after latest woflssl patch (21.02) - #50 by stangri

Thanks much! Will install the fixed package as soon as it reaches main and retest.

A workaround for anyone having the problem:

  • Add the following to /etc/config/https-dns-proxy under each 'config https-dns-proxy' stanza (that's a tab in front, NOT spaces!):
   option ca_certs_file '/etc/ssl/certs/ca-certificates.crt'
  • Restart https-dns-proxy via LuCI or '/etc/init.d/https-dns-proxy restart' from the command line

EDIT: A previous version of this workaround suggested explicitly installing the ca-certificates package. This is NOT necessary. ca-bundle, as required by https-dns-proxy, is sufficient.

3 Likes

That's done the trick for me, thank you!

This worked. Thank you!

Check, works.
Thanks.

Please note the workaround is still required for 22.03.2 at this time pending an updated https-dns-proxy package.

3 Likes