Thank you again for digging into all those issues.
Your patch 010-make-ujail-tty-writable.patch can be applied as-is imho, unless there is something in OCI standard regarding making this configurable (eg. by having tty group and only allowing rw access to members of that group).
Hooks not working is quite strange, I haven't tested it in a while, but I did test them excessively when implementing support for hooks. As nobody else has ever tested (esp. hooks) apart from me when implementing them, it can very well be that there are still problems. The biggest being the obvious lack of documentation of the various convenience hacks (such as per-container netifd instance, automatic handling of containers /etc/resolv.conf by netifd(!), ...) which can of course be in conflict with things done by hooks when using other container environments.
Regarding nsenter: I got a similar minimal implementation for testing, I should have included it as a package. Your implementation now is much more beautiful and of course we should take that now
The idea was kinda that in the end you should not be needing it, of course. uxc attach ... should be enough.
Regarding uxc exec: I think it can be very useful, though the implementation should be cleaned up a bit more. Listing all the different namespace types there once again feels wrong, esp. given that this list is getting longer from time to time, and we will then have to edit it an various places (eg. to add support for CLONE_NEWTIME now that kernel is new enough to support it).
Regarding user namespaces: While user namespaces generally do work afaik, there are problems with configurations combining cgroups and user namespaces. I've checked other implementations and found their work-arounds for the same issue quite ugly, and then the "nice" solution was around the corner CLONE_INTO_CGROUP (since Linux 5.7). Now that we have left behind Linux 5.4 we could mitigate the problem by using that.
ubus already supports passing open file handlers, and this is how uxc attach $containername as well as uxc start --console $containername works. In the same way we can also implement more logging -- and the console-pass-through code anyway needs some cleaning as you already noticed.
I will dedicate my day tomorrow to validate/fix hooks functionality and which ever of the other problems you mentioned I manage to reproduce and fix. As they are numerous, maybe we should have a checklist to tick them off and comment (I think that's enough, no need to overkill with spending time on issue-trackers).
Bugs:
- hook executable existence and permissions aren't checked
- hook process hangs (uloop/waitpid problem?)
Maybe bugs:
-
/dev/ttyaccess requires membership in gid 5 (tty). Make this configurable somehow (spec?).
Missing:
-
uxc exec(patch suggested) - Documentation for
netifdintegration, make/etc/resolv.confhandling more configurable