PiHole in a docker, no internet access

I just had an unexpected shutdown and docker refuses to load after that, this is what I'm getting:

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Tried everything to turn it on, nothing works. Is it this delicate?

For me unexpected shutdowns caused some issues with the storage being mounted in read only mode maybe due to a weird corruption which pretty much stops everything from working. I suggest to check storages and check the kernel logs (dmesg or logread), see if these are OK and no serious errors, hopefully if you find an error it may be in a post here on the forum.

For restarting docker, try the following diagnosis:

  • start docker with the below and check if docker is running again
/etc/init.d/docker restart
  • if the above didn't work, first check that there aren't any instances of docker running and then start docker manually and check for errors in the output:
# check if docker is not running, output should only showing 
# the grep command meaning no other processing that match 
# 'docker' are running
ps | grep docker
# start docker manually and check for any errors in the output, 
# the below command will run in the foreground so you'll need 
# another ssh session if you want to check things while keeping 
# the first session with docker running on it. To stop docker 
# process try hitting Ctrl+C or Ctrl+D.
/usr/bin/dockerd --config-file=/tmp/dockerd/daemon.json
 opkg install docker
Package docker (20.10.16-1) installed in root is up to date.
/etc/init.d/docker restart
-ash: /etc/init.d/docker: not found

dmesg had a lot of stuff in it, not sure what's relevant or not. It's not my first time dealing with dockers in Linux based systems, I've never had something like this happen. Would removing docker and reinstalling solve this? If so, what would happen to everything I set up until now?

dmesg | grep corrupt
[    4.130069] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
fsck
-ash: fsck: not found

 * wfopen: /usr/sbin/fatlabel: Read-only file system.
 * wfopen: /usr/sbin/fsck.fat: Read-only file system.
 * wfopen: /usr/sbin/mkfs.fat: Read-only file system.

Same issue, it would seem that it is quite common to get the system to boot as read only from the short search I ran.

How did you solve this issue on your machine?

It only happened to me a couple of times and I didn't get the chance to write down the steps due to being in rush. I recall having to run the fsck.ext4 on the ext4 partition that had the root file system. Something like the below (this only applies if using x86/x86_64 with ext4 fs, not squash fs):

# to check the path of the partition, use fdisk on the disk path then type the letters p then q.
# type m for more help. Here it is assumed that /dev/sda is the disk in use but it could 
# a different path (in most cases it's usually /dev/sda)
fdisk /dev/sda
# use fsck to resolve corruption with the partition based on the path shown in fdisk, it 
# should be the partition that does not have boot flag on. Answer the questions with y.
fsck.ext4 /dev/sda2
reboot

If the above didn't work due to command not found, then I guess you'll have to use a "Linux LiveCD" or a "Bootable Gparted image" and mount the disk to it to be able to run the above commands and fix the corruption.

Thanks, I already tried that, I'll have to reinstall as it installed in a GPT instead of MBR, and the destined computer only has Legacy bios anyway.

The question is how do I avoid this happening in the future, where the answer is not a UPS.

docker exec -it pihole ping 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
From 192.168.1.3 icmp_seq=1 Destination Host Unreachable
From 192.168.1.3 icmp_seq=2 Destination Host Unreachable
From 192.168.1.3 icmp_seq=3 Destination Host Unreachable

Reinstalled, did the guide step by step again, still no ping outside.

Having the same issue: no internet connection form inside a container. There are some comments in /etc/config/dockerd about it being incompatible with fw4; that could be why it is not working at this time.

1 Like

Thanks for the tip. :partying_face:

On my NanoPi5 with Docker i use OpenWrt 22.03.0-rc6 and I also have the problem that i can't send a ping from my Docker container to the outside.

In /etc/config/dockerd I found this line

config firewall 'firewall
        option device 'docker0'
        #list blocked_interfaces 'wan'

I commented out the list blocked_interfaces and restarted it. After that everything worked fine.

Please do not use in productive environments!

1 Like

Is it because it is now exposed to the Internet?

1 Like

It's because I don't know exactly what the setting does. I still have to try that out. I use the NanoPi R5S here anyway only in a protected LAN.

In my config for dockerd i miss this line

#	option extra_iptables_args '--match conntrack ! --ctstate RELATED,ESTABLISHED' # allow outbound connections

so i think it's better to do this

config firewall 'firewall'
	option device 'docker0'
	list blocked_interfaces 'wan'
	option extra_iptables_args '--match conntrack ! --ctstate RELATED,ESTABLISHED' # allow outbound connections

That work for me. Can someone with iptables knowledge confirm this? Thanks.

That is a good reason :slight_smile: I myself moved to LXC containers: they just work and do not require messing with firewall rules.

1 Like

I tried LXC container today. so much better than the Docker tinkering. Yesterday lost several hours with Docker / Firewall (OpenWrt 22.03.0-rc6).

Today about two hours and my unbound with ad blocker is running and not a problem with the firewall :slight_smile:

I will probably replace the Docker containers with LXC containers. Thanks for the tip!

hi,

  1. dockerd officially supports iptables only not nftables which is the core fw element in 22.03.
  2. dockerd service requires some fix (search in the forum for the possible solution) to work in 22.03 properly.
  3. blocked_interface + extra_iptables_args are the right way: first tells disallow communication between dockerd network stack and those interfaces in the list; 2nd tells, as explained by the comment, to make outgoing traffic from docker network and accept reply traffic. obviously if you don't want to allow wan (or any other interface) to communicate with dockerd network stack just use blocked_interface option.

LXC is indeed a good alternative if you are not short of storage and if you're happy to do manual upgrades of OS + app instead of docker pull & recreate & run sequence. there are benefits on both sides, use what is better for you. or you can even use both parallel.

2 Likes

LXC feels so much light weight vs Docker and takes less space as well. Just like in Docker, it is possible to control resources and priorities and do other tricks.

Would having iptables legacy or the iptables to nftables package (I can't seem to remember the exact name honestly) help to make it work then?

see point 2 - it works (more or less) if you do those fixes.

1 Like

Circling back on this one. I've followed the same tutorial listed in OP and still have internet. It looks like @grrr2 said something can work, with fixes, but need to search for those fixes. I've also read the other thread about nftables and dockerd, but still have no luck

Are there a definitive list of fixes which allows pihole to be run via macvlan with OpenWRT and nftables?

Thanks!

hi,

why you need macvlan?

what if you just use --network=host, fire up pihole, adjust ports as needed (e.g. move default dnsmasq to port 54 instead of 53 allowing pihole to bind port 53 etc).
i use adguard home with this kind of approach and it works. i could even use adguard as dhcp server, which i don't (*), so dnsmasq is still there but that's up to you.

(*) because it is very limited, only supports one interface for example.