Trying to make Domoticz userdata persistent

Installing and Using OpenWrt
1

I've searched the forum, and the web, and have not found any related topic.

The default userdata location for Domoticz on OpenWrt is '/var/lib/domoticz/'. Since that filesystem is volatile, all my hardware and devices configured in Domoticz are lost after a reboot. So, I changed the path for userdata in /etc/config/domoticz by commenting out the default and adding a persistent path:

# option userdata '/var/lib/domoticz/'
option userdata '/mnt/share/storage/domoticz/'

But now, 'service domoticz start' cause Domoticz to go into a crash loop:

Fri Aug  2 13:56:11 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 13:56:11 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 13:56:17 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 13:56:17 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 13:56:22 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 13:56:22 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 13:56:28 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 13:56:28 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 13:56:33 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 13:56:33 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 13:56:39 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 13:56:39 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 13:56:39 2024 daemon.info procd: Instance domoticz::instance1 s in a crash loop 6 crashes, 1 seconds since last crash
Fri Aug  2 15:07:22 2024 daemon.info domoticz: Domoticz is starting up....
Fri Aug  2 15:07:22 2024 daemon.info domoticz: Domoticz running...
Fri Aug  2 15:07:22 2024 daemon.info procd: Instance domoticz::instance1 s in a crash loop 7 crashes, 0 seconds since last crash

I don't find any 'domoticz_crash.log' files anywhere on the system.

I have allowed '/etc/init.d/domoticz' to create the userdata directory structure which also changes ownership of the userdata directory:

ls -l /mnt/share/storage/
drwxrwx---    2 domoticz domoticz      4096 Aug  2 11:06 domoticz

ls -l /mnt/share/storage/domoticz/
lrwxrwxrwx    1 root     root            21 Aug  2 11:06 plugins -> /etc/domoticz/plugins
lrwxrwxrwx    1 root     root            21 Aug  2 11:06 scripts -> /etc/domoticz/scripts

If I invoke 'domoticz' directly from the command line, it runs without any apparent error, but images and/or CSS files don't load in the web interface at port 8080. (And it creates a 'plugins' directory in '/usr/bin' for some reason.)

Is there a "correct" way to make Domoticz userdata persistent?

Netgear WNDR4300

OPENWRT_BOARD="ath79/nand"
OPENWRT_ARCH="mips_24kc"
OPENWRT_RELEASE="OpenWrt 22.03.5 r20134-5f15225c1e"

2

Is there ujail in the domoticz init.d file ?

You should upgrade.

3

There is no "ujail" in the '/etc/init.d/domoticz' file.

I tried uploading the 'domoticz' init file, but apparently text files are not allowed:

Sorry, the file you are trying to upload is not authorized (authorized extensions: jpg, jpeg, png, gif, heic, heif, webp, avif).

I am always hesitant to mess with firmware lest something goes wrong. And, I haven't familiarized myself with how to do an OpenWrt upgrade with OpenWrt firmware. Maybe I should.

4

You can always c&p it into the post, just use the </> button for preformatted text.

But if there's no ujail, it shouldn't be a permission thing.

5

Okay. Here is the content of '/etc/init.d/domoticz':

#!/bin/sh /etc/rc.common

START=99
USE_PROCD=1
PROG=/usr/bin/domoticz
PIDFILE=/var/run/domoticz.pid

start_domoticz() {
	local section="$1"
	local loglevel sslcert sslpass sslwww syslog userdata

	config_get loglevel "$section" "loglevel"
	config_get sslcert "$section" "sslcert"
	config_get sslkey "$section" "sslkey"
	config_get sslpass "$section" "sslpass"
	config_get ssldhparam "$section" "ssldhparam"
	config_get sslwww "$section" "sslwww"
	config_get syslog "$section" "syslog"
	config_get userdata "$section" "userdata" userdata /var/lib/domoticz

	[ -n "$loglevel" ] && procd_append_param command -loglevel "$loglevel"
	[ -n "$syslog" ] && procd_append_param command -syslog "$syslog"

	[ -d "${userdata}" ] || {
		mkdir -p "${userdata}"
		chmod 0770 "$userdata"
		chown domoticz:domoticz "$userdata"
	}

	# By default, ${userdata}/scripts is a symlink to /etc/domoticz/scripts
	# and the two dzVents directories under there which Domoticz will actually
	# write to at runtime are symlinked back to /var/lib again.
	[ -d "${userdata}/plugins" ] || ln -sf /etc/domoticz/plugins "${userdata}/plugins"
	[ -d "${userdata}/scripts" ] || ln -sf /etc/domoticz/scripts "${userdata}/scripts"
	for DIR in data generated_scripts; do
		[ -d /var/lib/domoticz/dzVents/$DIR ] || {
			mkdir -p /var/lib/domoticz/dzVents/$DIR
			chown domoticz.domoticz /var/lib/domoticz/dzVents/$DIR
		}
	done
	procd_append_param command -userdata "$userdata"

	[ -n "$sslcert" -a "${sslwww:-0}" -gt 0 ] && {
		procd_append_param command -sslcert "$sslcert"
		procd_append_param command -sslwww "$sslwww"
		[ -n "$sslkey" ] && procd_append_param command -sslkey "$sslkey"
		[ -n "$sslpass" ] && procd_append_param command -sslpass "$sslpass"
		[ -n "$ssldhparam" ] && procd_append_param command -ssldhparam "$ssldhparam"
	} || procd_append_param command -sslwww 0
}

start_service() {
	procd_open_instance

	procd_set_param command "$PROG"
	procd_append_param command -noupdates
	procd_append_param command -approot /usr/share/domoticz/

	config_load "domoticz"
	config_get_bool disabled "$section" "disabled" 0
	[ "$disabled" -gt 0 ] && return 1
	config_foreach start_domoticz domoticz

	procd_set_param pidfile "$PIDFILE"
	procd_set_param respawn
	procd_set_param stdout 0
	procd_set_param term_timeout 10
	procd_set_param user "domoticz"

	procd_close_instance
}
6

It does appear to be a permission issue. I changed the next to last command in '/etc/init.d/domoticz' from:
procd_set_param user "domoticz"

to:
procd_set_param user "root"

and it works without crashing.

Now, to figure out exactly what is the permission issue...

7

If you run it as root, and it creates files, they'll have the wrong owner, when you try to run it using the init.d script.

8

I was actually running it as root using the init.d script by changing the user in this line: procd_set_param user "domoticz"
But, yes, any files created by 'root' would have the wrong owner. I did that only as a test.

Further testing shows that:

  1. Running as user 'domoticz' with the default userdata path in the volatile '/var/lib/domoticz' works fine,
  2. Running as user 'root' with my preferred userdata path on persistent storage works fine,
  3. Running as user 'domoticz' with my preferred userdata path does not work.

Raising the loglevel to "error" shows that it cannot open the database file when running with condition #3 above. Under the other two conditions, it appears that 'domoticz.db' is copied from '/usr/bin' to the userdata path but not under condition #3. (Why the db file is installed to /usr/bin I cannot fathom.)

I guess I'm going to have to take a look at the source code.

9

Soft link the incorrect path, to the desired one ?

10

I thought about that. But, 'var/lib/domoticz' needs to be an actual directory because it holds runtime data:

# By default, ${userdata}/scripts is a symlink to /etc/domoticz/scripts
# and the two dzVents directories under there which Domoticz will actually
# write to at runtime are symlinked back to /var/lib again.
[ -d "${userdata}/plugins" ] || ln -sf /etc/domoticz/plugins "${userdata}/plugins"
[ -d "${userdata}/scripts" ] || ln -sf /etc/domoticz/scripts "${userdata}/scripts"
for DIR in data generated_scripts; do
        [ -d /var/lib/domoticz/dzVents/$DIR ] || {
                mkdir -p /var/lib/domoticz/dzVents/$DIR
                chown domoticz:domoticz /var/lib/domoticz/dzVents/$DIR
        }
done

Another option I've considered is creating a 'procd' service stop script that would copy the working db in '/var/lib/domoticz' to persistent storage, and then modify the init script to copy the saved db to the volatile working directory on start up. I would need to learn how to use 'procd' first though, and I think it's a bit of a kludge work-around.

It just seems to me that it should work when one simply changes the 'userdata' path to their preferred path. And, I'm trying to determine the "correct" fix so others don't have to go through this like I am. It surprises me that this issue hasn't been brought up more. Perhaps that is because others have taken the easy, and less secure, route choosing to run the application as root: [SOLVED]Domoticz at 18.06.2 - #14 by sobolkz

11

I fail to see the soft link problem....

12

I was thinking that the runtime data should be cleared when the system is rebooted, and that a soft-link in the volatile working directory might cause circular links. But now that I think about it more, all it would do is cause the runtime data to be stored persistently; and just might work.

So, I soft-linked '/var/lib/domoticz' to '/mnt/share/storage/domoticz'. And even copied the db to the persistent storage giving it domoticz:domoticz ownership.:

root@OpenWrt:~# ll /var/lib/domoticz
lrwxrwxrwx    1 root     root            27 Aug  3 13:38 /var/lib/domoticz -> /mnt/share/storage/domoticz/

root@OpenWrt:~# ll /var/lib/domoticz/
-rw-r--r--    1 domoticz domoticz    401408 Aug  3 13:46 domoticz.db
drwxr-xr-x    4 root     root          4096 Aug  3 13:39 dzVents/
lrwxrwxrwx    1 root     root            21 Aug  3 13:39 plugins -> /etc/domoticz/plugins/
lrwxrwxrwx    1 root     root            21 Aug  3 13:39 scripts -> /etc/domoticz/scripts/

It still doesn't work:

Sat Aug  3 13:48:18 2024 daemon.info domoticz: Domoticz is starting up....
Sat Aug  3 13:48:18 2024 daemon.info domoticz: Domoticz running...
Sat Aug  3 13:48:18 2024 daemon.err domoticz: Error opening SQLite3 database: unable to open database file
Sat Aug  3 13:48:18 2024 daemon.err domoticz: Domoticz(pid:5612, tid:5612('-')) received fatal signal 6 (Aborted)
Sat Aug  3 13:48:18 2024 daemon.err domoticz: siginfo address=0x15ec
Sat Aug  3 13:48:18 2024 daemon.err domoticz: Failed to start gdb, will use backtrace() for printing stack frame
Sat Aug  3 13:48:18 2024 daemon.info procd: Instance domoticz::instance1 s in a crash loop 6 crashes, 1 seconds since last crash

I don't get it. I must be doing something wrong with this.

I'm going to upgrade to OpenWrt 23.05 and try the Domoticz 23.05 version, though I don't see much difference between that source and the 22.03 version source. If there is still an issue, I will raise it at Bug Reports. Hopefully, I can work with the maintainer to get it sorted out.

In the meantime, I'll likely resort to what at least one other has done: run it as root.

Thanks for trying to help. :slightly_smiling_face:

13

Just in case anyone cares to follow along, I've submitted a bug report: