LuCI rewrite in ucode - testers wanted

Yeah, the logic is quite naive atm, it simply checks if there's a luasrc/ directory in a package. If there is, a dependency on luci-lua-runtime is added. Independently of this LuCI ucode rewrite we could already start converting rpcd backend scripts to ucode or shell (whatever is preferred).

Could you give me a quick list of packages in your .config which do not explicitly depend on Lua but still require it?

I noticed just the statistics and upnp (plus wireguard that I didn't actually trigger).

The implicit dependencies found so far seem to be init/rpcd/status scripts having lua in shebang.

After disabling the packages with explicit dependencies, the build config is small:

perus@ub2204:/Openwrt/r7800$ grep "luci" .config.minimal 
CONFIG_PACKAGE_luci-ssl-openssl=y
CONFIG_PACKAGE_luci-app-sqm=y
CONFIG_PACKAGE_luci-app-wol=y
CONFIG_PACKAGE_luci-proto-ppp=y

Implicit dependencies found:

CONFIG_PACKAGE_luci-app-upnp=y
CONFIG_PACKAGE_luci-app-statistics=y
CONFIG_PACKAGE_luci-app-wireguard=y

Disabled due to explicit dependencies:

####CONFIG_PACKAGE_luci-theme-material=y
####CONFIG_PACKAGE_luci-theme-openwrt-2020=y
####CONFIG_PACKAGE_luci-app-adblock=y
####CONFIG_PACKAGE_luci-app-commands=y
####CONFIG_PACKAGE_luci-app-ddns=y
####CONFIG_PACKAGE_luci-app-nlbwmon=y
####CONFIG_PACKAGE_luci-app-uhttpd=y
1 Like

I pushed ucode conversions for luci-app-upnp, luci-app-statistics and luci-app-wireguard to the PR branch.

2 Likes

Thanks jow,
at the first glance the stats work normally again, without lua in the build :wink:

Hmmm, system->startup->initscripts is empty for me, no init scripts listed. Anyone else?

Anything reported by ubus call luci getInitList on the cli? If not, any error visible in logread afterwards?

ubus call luci getInitList
Command failed: Unknown error

Nothing in logread.

Fixes for conntrack listing and @ldir's init script listing issue pushed. Also converted luci-theme-material to ucode (untested yet).

4 Likes

Thanks @jow
The init scripts evaluation (startup page) seems to work, and also the conntrack listing in realtime graphs works.

massive work @jow

If we rewirte them to ucode what will be L mean from Luci?

Lightweight :stuck_out_tongue:

6 Likes

On latest master i get this error message now:

Runtime error: Unable to dlopen file '/usr/lib/ucode/luci/core.so': /usr/lib/ucode/luci/core.so: undefined symbol: crypt
In module(), file /usr/share/ucode/luci/dispatcher.uc, line 1, byte 1:
  called from anonymous function (/www/cgi-bin/luci:7:21)

 `// Copyright 2022 Jo-Philipp Wich <jo@mein.io>`
  ^-- Near here

Is this error related to this change?

Likely. Is this a custom build? If yes, what's the diffconfig.sh output?

Can't post it here because of the char limit.

Ah, a glibc build, as suspected. Please try the following fix:

diff --git a/modules/luci-base/src/Makefile b/modules/luci-base/src/Makefile
index 896aeb0a38..ad309e5c6b 100644
--- a/modules/luci-base/src/Makefile
+++ b/modules/luci-base/src/Makefile
@@ -10,7 +10,7 @@ lib/plural_formula.c: lib/plural_formula.y contrib/lemon
 lib/lmo.c: lib/plural_formula.c
 
 core.so: lib/luci.o lib/lmo.o lib/plural_formula.o
-       $(CC) $(LDFLAGS) -shared -o $@ $^
+       $(CC) $(LDFLAGS) -shared -lcrypt -o $@ $^
 
 version.uc:
        echo "export const revision = '$(LUCI_VERSION)', branch = '$(LUCI_GITBRANCH)';" > $@
1 Like

Works. Thank you.

Is there any place from where we could see examples of ut files? the ucode page is not very clear. For instance all the environment variables and how to access uci config files is not covered iin the ucode page.

For instance, the lua instructions:

-- UCI Management
primary = uci:get_first( 'argon', 'global', 'primary' )

or

-- stdout of a sh script to a string variable
local bing = sys.exec("/usr/libexec/argon/bing_wallpaper")

Reading from a config file is now difficult to implement. I could rewrite all in shell script and just do system('myscript',0) in ucode to do all the job creating the html page code, but I hope ucode could do something more elegant. Any idea on to implement the previous lua intructions in ucode?

I hope we are doing all this for a larger good. saving 120 KB is not reason enough to create all this mess.

Any resource on where to get more .ut sample files to learn about how to do things from now on?

Thanks

No need to spam all the threads, you have your replies here: Latest Luci changes affecting a lot of legacy 3rd party themes - #6 by stangri

1 Like

Thanks for a good job. I tried and have some thoughts.

  1. Size is 6kb + 58kB libucode which is great. High level scrips are more space effective than compiled C program so here this actually can help to save space. Also it's saves resources when we don't need to compile and build luci for every architecture.

  2. The syntax is inspired by JS but not fully compatible.
    E.g. for in actually works as JS for of. This can lead to a sever mistakes if a developer will not notice the difference.

  3. Global functions like push(arr) instead of JS arr.push(). json as JS JSON.parse() and printf('%J') as JSON.serialize(). This all makes previous knowledge of JS stdlib useless.
    Maybe we can change this to be in line with JS? Then my IDE can work correctly and show autocomplete.
    For those functions that not exists in JS or behaves differently we can create a pseudo package ucode that import explicitly. We can make a JS stub file with function signatures. Or even implement them so the script can be run and tested on nodejs.

  4. File is extension *.uc so when I open a file in IDE I don't have a syntax highlighting. In my IntelliJ I associated the uc files with EcmaScript6 and everything looks fine. But still when I open a file in Notepad/mcedit I see just gray text.

  5. Previously I was able to use an IDE plugin for Lua but now left with plain text. Ok, I associated uc files with JS but that still not complete. We can't use the whole JS ecosystem with libraries and tools like ESLint and testing frameworks. I can't simply start Luci on Ubuntu for development and I need to compile and install the ucode first.

  6. The uc files use tab instead of spaces. I understand that for client side JS files historically is used tab but here we anyway created new uc files so we can start using the most modern JS code style https://standardjs.com/ e.g. 2 spaces. The IntelliJ IDE supports it out of the box so this will help to contribute for newcommers.

So my basic proposition is to make a next step and use JS. We can rewrite the Luci uc files to js and use the old ucode syntax for uc files but a strict JS for js files. Fix the interoperability problems like for in inconsistency.

Another one big concern is the engine itself. Currently we have a small bus factor but also it may have some undiscovered vulnerabilities.
Maybe instead we can reuse some other JS engine?
The default is NodeJS but Dino or a new Bun can be compiled with minimal dependencies.
There is also low.js which is a minimized NodeJS.

Micro-controllers and IoT devices also use JS:

  • https://github.com/cesanta/mjs "Any valid mJS code is a valid ES6 code. Any valid ES6 code is not necessarily a valid mJS code."
  • JerryScript, a JavaScript engine for small devices developed by Samsung
  • Espruino, a JavaScript engine for microcontrollers

Related articles:

BTW Even if just announce the ucode as Yet another JS engine this will attract many JS enthusiasts. So if make a package for Debian/Ubuntu this will make more users, testers and contributors to the ucode. For example GNOME has own JS engine https://gjs.guide/guides/ and they may be interested in something more lightweight.