Jshn wiki page

Talk about Documentation
1

I created a page with description of jshn

I just gathered all the samples that I found so the article can be simplified and improved.
For example one example from this thread

Also if anyone can add more descriptions for the jshn utility that would be great because now they are just repeats themselves:
Options:

  • -r <message> parse from string <message>
  • -R <file> parse from file <file>
  • -w write to stdout
  • -o <file> write to file <file>
  • -p <prefix> set prefix
  • -n no newline
  • -i indent
5 Likes
2

@stokito I really appreciate the wiki page that you created. It helped me write a simple shell script for fetching the date (timestamp) corresponding to a minimum near-time future value (price) of electricity in Finland, in a 2-column JSON array of prices:

#!/bin/sh
set -e
. /usr/share/libubox/jshn.sh
date=$(date -u +%Y-%m-%dT%H:00:00.000Z)
json_load "$(curl -s "https://sahkotin.fi/prices?start=$date")"
dump(){ eval "echo -e \$$1_value\\\\t\$$1_date";}
json_for_each_item dump prices|
awk 'BEGIN{v="NaN";d="$date";}{if($1<0+v){v=$1;d=$2;}}END{print d}'

I wonder if my dump() is considered a hack, or if it is valid usage of the library. In the wiki page, there currently is no example of traversing arrays of structures like this. One more extension to the wiki page might be an example to traverse a multi-dimensional array, if that is supported.

I would also like to publish a more complete version of the script somewhere. It would be a cron job that would interact with a Tasmota device. Is there perhaps a GitHub repository of such examples, related to OpenWrt?

3

If you need array functionality, you can use my POSIX-arrays library. It is implemented in shell language and doesn't call any external binaries, so it's generally fast for small'ish arrays (up to about 2k elements on x86). It supports both indexed and associative arrays, and there are examples of translation from Bash.

github.com/friendly-bits/posix-arrays

4

I am rather unfamiliar with what could be considered idiomatic JSON. I realized that two-dimensional arrays, as opposed to arrays of tuples, would make it harder to express things like this:

jq '.["prices"]|min_by(.value).date'

With the exception of empty input, this filter should be roughly equivalent to the jshn based one. It would introduce a dependency on jq, which is it is not installed by default on my system.
@antonk The "Some more details" section of your library documentation immediately makes me think of a similar eval trick. I rather seldomly write shell scripts, therefore I sought an opinion whether there is any better solution to this problem. I didn’t want to go straight ahead and augment the wiki page with something controversial.

5

I'm not really sure what you are saying but regarding shell scripts, they have lots of pitfalls, so being reluctant of implementing complex things in a shell script is generally well justified. On the other hand, working with arrays via external binaries may get very slow very quickly. The idea of my library is to provide a relatively easy to use and safe API for working with arrays in a shell script, so you can use arrays without being an expert on shell scripts. You are welcome to use what is right for you.

6

The curl needs to be installed but the OpenWrt out of the box has uclient-fetch which is a stripped wget clone. So you can use:

  PRICES_JSON=$(wget -qO - "https://sahkotin.fi/prices?start=$date")
  exit_status=$?
  if [ $exit_status -ne 0 ]; then
     >&2 echo "error $exit_status"
     exit $exit_status
  fi
  json_load "$PRICES_JSON"

Everything else I put on the wiki page
https://openwrt.org/docs/guide-developer/jshn#parse_list_of_objects

Please review and comment.
I also created a separate topic to help with shell scripts Writing shell scripts support topic

1 Like
7

@stokito Thank you, your version is illustrating the usage of the library functions better. I was not aware that curl is not part of the core installation. I accidentally bought the gl.inet Flint while I intended to get the MT-GL6000 (Flint 2), for which someone has implemented stock OpenWrt.

I think that in my particular application I would rely on set -e to abort the script in case of any failing command. Typically, I would expect it to be a missing Internet connection, or an error during parsing some misformatted data, which in my final script would likely lead to some date arithmetics complaining about an invalid argument.

In fact, I would prefer set -eu for even more safety, but this would trigger an error due to an uninitialized variable in /usr/share/libubox/jshn.sh. If I initialize JSON_PREFIX= before sourcing the library, the next complaint would be about the variable JSON_UNSET, which I don’t think can be fixed that easily.

Normally, my final script would invoke sleep for 0 to 23 hours and send a command to start a warm water boiler at the cheapest time. The script would be invoked every 24 hours by cron. There will be a manual override in case the automation should fail.

1 Like
8

Maybe you can use the HomeAssistant for this? It's written in Python so it will be problematic to use it on a router but technically speaking this is possible,
Be careful with such things: you may burn you house :slight_smile:

9

I posted my solution in https://github.com/arendst/Tasmota/discussions/21915, including implementing the JSON logic in both jq and jshn (using my original eval and awk hack).

HomeAssistant might actually run on the gl.inet Flint 2, but I prefer to keep things small and simple. Physically, I think that this is safe, backed by a thermostat and a pressure valve. Only the control of the 3-phase contactor has been replaced with a CE approved WiFi enabled switch (okay, running possibly non-approved third-party open source firmware).