Package jq has no regex support

Feature Requests
1

Hello. I am trying to write a script to parse a JSON file from mediathekviewweb API to script some movie downloading. However, this is working on my PC but not on my device. After some debugging I stumbled upon an unexpected error:

user@device:~$ cat resp10.txt | jq '.result.results | .[0].title | scan("foo")'
jq: error (at <stdin>:0): jq was compiled without ONIGURUMA regex libary. match/test/sub and related functions are not available.

Unfortunately, installing the oniguruma package does not help. I would like to ask for a second package version with onigurma as a dependency, so jq can perform regular expressions. I am not experienced enough to do this, I never compiled anything, I am used to scripting. I can work around this by using python and its json module, but this is an overkill of resources.

May I ask if someone is so kind to do this for me at some time?
Thank you all!

2

What device is this running on??

ubus call system board
1 Like
3 
{
	"kernel": "5.15.167",
	"hostname": "ArchipelWEI",
	"system": "Qualcomm Atheros QCA9533 ver 2 rev 0",
	"model": "GL.iNet GL-AR300M (NOR)",
	"board_name": "glinet,gl-ar300m-nor",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "23.05.5",
		"revision": "r24106-10cc5fcd00",
		"target": "ath79/nand",
		"description": "OpenWrt 23.05.5 r24106-10cc5fcd00"
	}
}
4

You may be able to accomplish your goal with the already installed tools on the router, if you are willing for it to be semi non-portable. Check out jsonfilter:

$ ubus call system board  | jsonfilter -e '$.kernel' -e '$.release.target'
5.15.167
x86/64
5

Thank you for this suggestion.

$ curl -s -H "Content-Type: text/plain" -d '{ "queries" : [ {"fields" : ["topic"], "query" : "Sturm der Liebe"} ] , "size" : 9999 }' "https://mediathekviewweb.de/api/query" > resp.txt
$ cat resp.txt | jsonfilter -e '$.result.results[@.title="Episode 3740"].size'
452984832
$ cat resp.txt | jq '.result.results[] | select(.title=="Episode 3740").size'
452984832

Well, I do not see any advantage over jq or I am too blind to see how I can use regex with jsonfilter. Althouh I could deal without for my specific use case it would be less generic. I prefer the more powerful jq and would like to use regex. I also want to use the functions scan (fails now), select and inside. The possibility to combine them in one statement is way easier and more efficient than doing this in shell loops. I guess this is what you mean with "semi non-portable".

Therefore, I would like to continue this feature request.

6

Json parsing can be done with sed. For complex tasks this may be difficult but probably possible, as a last resort.

7

Yes, absolutely. As I said, I can work around this. Still, jq is a powerfull language and I would like to use it in its full feature-set.

I mean, we do have packages like mtr-json and mtr-nojson and a difference in size of 739 bytes (at least for my platform). The size difference for jq and jq-oniguruma would be sth. like 96641 bytes vs. 96641+170733 bytes. This is still better than higher level languages, enough to provide 2 packages and powerful enough to beat sed/awk in first place.

I may be a rare case, I can understand that. Still I think it is reasonable, is it?

8

You could open an issue in the OpenWrt packages repo and tag the jq mainater who appears to be @ratkaj (on github).

9

That looks like a good idea to me. I just created my first github account and will create an issue the next days - when I am sure not to sth. stupid there.

10

There is precedent for having packages with different features versus size tradeoffs. For example, there are 9 different versions of wpad, supporting different TLS libraries and enabled features. So perhaps you can propose a "jq-full" package that has everything enabled.

1 Like