Procd capabilities support in 21.02.2

For Developers
1

I am writing my first procd init script. The service in question needs CAP_NET_RAW and CAP_NET_ADMIN capabilities.

I've tried running it as nobody and setting the capabilities:

root@OpenWrt:/etc/init.d# cat fdfd
#!/bin/sh /etc/rc.common

USE_PROCD=1
START=90
STOP=15

start_service() {
        procd_open_instance
        procd_set_param command /usr/bin/fdfd
        procd_set_param file /etc/fdf-config.json
        procd_set_param user nobody
        procd_set_param capabilities /etc/capabilities/fdfd.json
        procd_close_instance
}

root@OpenWrt:/etc/init.d# cat /etc/capabilities/fdfd.json
{
        "bounding": [
                "CAP_NET_RAW",
                "CAP_NET_ADMIN"
        ],
        "effective": [
                "CAP_NET_RAW",
                "CAP_NET_ADMIN"
        ],
        "ambient": [
                "CAP_NET_RAW",
                "CAP_NET_ADMIN"
        ],
        "permitted": [
                "CAP_NET_RAW",
                "CAP_NET_ADMIN"
        ],
        "inheritable": [
                "CAP_NET_RAW",
                "CAP_NET_ADMIN"
        ]
}

But I'm getting an Operation not permitted error when trying to set the SO_BINDTODEVICE socket option, which requires CAP_NET_ADMIN.

If I comment out the init script lines that set the user and capabilities then the process runs as root and works just fine.

Am I doing something wrong? Does procd capabilities support actually work in 21.02.2?

Thanks!

2

OK, I've figured out that capabilities only work with jails, so my init script now looks like this.

#!/bin/sh /etc/rc.common

USE_PROCD=1
START=90
STOP=15

start_service() {

        procd_open_instance
        procd_set_param command /usr/bin/fdfd
        procd_set_param file /etc/fdf-config.json

        if [ -x /sbin/ujail -a -e /etc/capabilities/fdfd.json ]; then
                procd_add_jail fdfd log requirejail
                procd_set_param user nobody
                procd_set_param capabilities /etc/capabilities/fdfd.json
                procd_set_param no_new_privs 1
        fi

        procd_close_instance
}

I've installed procd-ujail, and now my service just hangs when I try to start it. Nothing is logged and netstat shows that none of its listening sockets are open. Furthermore, it doesn't respond to SIGTERM, only SIGKILL.

Running ujail from the command line has similar results, i.e.:

root@OpenWrt:/etc/init.d# /sbin/ujail -n fdfd -U nobody -C /etc/capabilities/fdfd.json -c -l -E -dddd -- /usr/bin/fdfd

Nothing logged, no sockets, doesn't respond to SIGTERM, etc.

How can I figure out what is going on?

3

Try strace and let me know what you see when it hangs.

4

It's stuck in epoll_pwait(), but I can't figure out what it's waiting for.

Here are the last few lines of strace output. Note that this shows what happens when I try to stop it with SIGTERM and then use SIGKILL.

5197  mmap2(NULL, 45864, PROT_READ, MAP_PRIVATE, 8, 0) = 0x77d84000
5197  close(8)                          = 0
5197  munmap(0x77d84000, 45864)         = 0
5197  close(7)                          = 0
5197  munmap(0x77eb4000, 42304)         = 0
5197  epoll_pwait(3, 0x77e12418, 10, 1804856406, NULL, 16) = -1 EINTR (Interrupted system call)
5197  --- SIGTERM {si_signo=SIGTERM, si_code=SI_USER, si_pid=5199, si_uid=0} ---
5197  sigreturn({mask=[]})              = -1 EINTR (Interrupted system call)
5197  epoll_pwait(3,  <unfinished ...>) = ?
5197  +++ killed by SIGKILL +++

The full command was:

strace -f -o /tmp/ujail.strace /sbin/ujail -n fdfd -U nobody -C /etc/capabilities/fdfd.json -c -l -E -r /usr/lib/fdf-filters -dddd 
 -- /usr/bin/fdfd -d -l

I put the full strace output at https://pastebin.com/1cptaz16, and you can find the fdfd source code at https://github.com/ipilcher/fdf if needed.

5

Another data point. I noticed that sysntpd runs in a jail when ujail is installed, so I restarted the service (stop, wait, start), and I'm seeing the exact same behavior — nothing in the log and it doesn't respond to SIGTERM.

Mar 30 10:09:30 OpenWrt procd: Instance sysntpd::instance1 pid 5304 not stopped on SIGTERM, sending SIGKILL instead

Looks like this may be a general issue with ujail. :grimacing:

6

Can you try using procd package from master branch and see if the issue is fixed there?
It is possible that some fixes still have to be picked into 21.02 as well.

Particularly, I suspect you might be hitting what has been fixed by
https://git.openwrt.org/?p=project/procd.git;a=commit;h=bb95fe8df7115ab57792369c768927eafc042161
as well as
https://git.openwrt.org/?p=project/procd.git;a=commit;h=1056fc4cf36a99f9ca2bf0857d9291e8d33d95ad

7

Can you try using procd package from master branch and see if the issue is fixed there?

Is it possible to do that without reinstalling OpenWrt? I don't have a spare device on which to do that. I supposed I could check into running an x86 build in a VM, but ...

Particularly, I suspect you might be hitting what has been fixed by
https://git.openwrt.org/?p=project/procd.git;a=commit;h=bb95fe8df7115ab57792369c768927eafc042161
as well as
https://git.openwrt.org/?p=project/procd.git;a=commit;h=1056fc4cf36a99f9ca2bf0857d9291e8d33d95ad

Looking at those patches, it doesn't look likely that either of them is related to what I'm seeing. Specifically, if you look at the strace output, you won't see fork() or clone() called anywhere. For some reason, the actual service isn't even being run.

8

You can sysupgrade to a pre-release 22.03-SNAPSHOT build for that device or even just replace procd (which you would have to build using the 21.02.2 SDK). In case something goes wrong you can use failsafe mode to recover.

The first of those two patches fixes the issue that prevents jailed services from being stopped.
The second commit fixes a memory corruption on the stack which can very well explain the behavior you are seeing.

9

Just wanted to circle back on this. I didn't have time to test the pre-release, but I did uninstall procd-ujail and then reinstall it later for some testing. To my surprise, it's working now, so it looks like the latest package (2021-03-08-2cfc26f8-1) has whatever fixes are required to make it work.

10

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.