Fw4 won't start after upgrade to 2022-04-21-fc83d462-1

pronichkin · April 27, 2022, 4:39pm

I have Linksys WRT1900AC v1. It was running OpenWRT 21.02.1 until recently. Last weekend I upgraded it to 22.03.0-rc1. It kept running well.

Yesterday I decided to update some packages. I know it's generally not recommended, but I was curious.

Here's the list of the packages I updated.

iwinfo - 2022-03-01-90bfbb9a-1 - 2022-04-24-a479b9b0-1
luci-app-statistics - git-22.002.35118-8572ae0 - git-22.115.68420-46794ea
libiwinfo-lua - 2022-03-01-90bfbb9a-1 - 2022-04-24-a479b9b0-1
ucode-mod-ubus - 2022-02-11-a29bad94-1 - 2022-04-13-e14b0993-1
ucode-mod-uci - 2022-02-11-a29bad94-1 - 2022-04-13-e14b0993-1
firewall4 - 2022-02-12-53caa1a7-1 - 2022-04-21-fc83d462-1
ucode-mod-fs - 2022-02-11-a29bad94-1 - 2022-04-13-e14b0993-1
libiwinfo-data - 2022-03-01-90bfbb9a-1 - 2022-04-24-a479b9b0-1
ucode - 2022-02-11-a29bad94-1 - 2022-04-13-e14b0993-1
libiwinfo20210430 - 2022-03-01-90bfbb9a-1 - 2022-04-24-a479b9b0-1

After that update, firewall4 refuses to start anymore. Originally, it showed the following error.

root@pronichkin-router:~# fw4 start
Error loading shared library libucode.so.20220322: No such file or directory (needed by /usr/bin/utpl)
Error relocating /usr/bin/utpl: uc_vm_init: symbol not found
Error relocating /usr/bin/utpl: uc_vm_invoke: symbol not found
Error relocating /usr/bin/utpl: ucv_from_json: symbol not found
Error relocating /usr/bin/utpl: ucv_object_delete: symbol not found
Error relocating /usr/bin/utpl: uc_vm_trace_set: symbol not found
Error relocating /usr/bin/utpl: uc_vm_scope_get: symbol not found
Error relocating /usr/bin/utpl: uc_source_new_buffer: symbol not found
Error relocating /usr/bin/utpl: ucv_object_add: symbol not found
Error relocating /usr/bin/utpl: ucv_string_new: symbol not found
Error relocating /usr/bin/utpl: ucv_get: symbol not found
Error relocating /usr/bin/utpl: ucv_object_get: symbol not found
Error relocating /usr/bin/utpl: uc_vm_free: symbol not found
Error relocating /usr/bin/utpl: ucv_int64_get: symbol not found
Error relocating /usr/bin/utpl: uc_compile: symbol not found
Error relocating /usr/bin/utpl: ucv_put: symbol not found
Error relocating /usr/bin/utpl: ucv_array_push: symbol not found
Error relocating /usr/bin/utpl: uc_program_write: symbol not found
Error relocating /usr/bin/utpl: uc_source_new_file: symbol not found
Error relocating /usr/bin/utpl: ucv_array_new: symbol not found
Error relocating /usr/bin/utpl: uc_stdlib_load: symbol not found
Error relocating /usr/bin/utpl: uc_vm_execute: symbol not found
root@pronichkin-router:~#

My system has /usr/lib/libucode.so.20220206 but not /usr/lib/libucode.so.20220322. So, just for testing, I decided to copy the former into the later. Unfortunately, it did not solve the problem, but now fw4 gives an entirely different error error. (Which might mean I'm onto something.)

root@pronichkin-router:~# cp /usr/lib/libucode.so.20220206 /usr/lib/libucode.so.20220322
root@pronichkin-router:~# fw4 start
/proc/self/fd/0:1:8-8: Error: syntax error, unexpected '='

/proc/self/fd/0:2:9-9: Error: syntax error, unexpected '='

/proc/self/fd/0:3:10-10: Error: syntax error, unexpected '='

/proc/self/fd/0:5:15-15: Error: syntax error, unexpected '='

/proc/self/fd/0:7:18-18: Error: syntax error, unexpected '='

/proc/self/fd/0:8:18-18: Error: syntax error, unexpected '='

/proc/self/fd/0:9:18-18: Error: syntax error, unexpected '='

/proc/self/fd/0:10:18-18: Error: syntax error, unexpected '='

/proc/self/fd/0:11:18-18: Error: syntax error, unexpected '='

/proc/self/fd/0:12:18-18: Error: syntax error, unexpected '='

root@pronichkin-router:~#

so far, I'm out of ideas. Any suggestions, besides reflashing my router to the original 22.03.0-rc1 firmware and never updating packages again?

hnyman · April 27, 2022, 4:56pm

Why flash again, when you can just reset the settings & add-on packages?
Either "firstboot" from SSH console or Reset from LuCI.
That removes the packages installed by you, and clears the settings.

Make a backup of your settings, reset, install the needed add-ons again, restore backup.

dave14305 · April 27, 2022, 5:42pm

I did this yesterday as well, and once I saw no nft rules in place, I immediately halted the router and rebuilt a current image with Imagebuilder. If I had known the firstboot command, I would have done that instead.

hnyman · April 27, 2022, 6:06pm

Some info in wiki...

dave14305 · April 27, 2022, 6:17pm

Good to know, but I took drastic measures once I realized my shields were down, and had no IPv4 NAT anyway. And since I use a RPi4, it was easy enough to reflash the card with my original image, get back online, and refresh my Imagebuilder with the latest packages.

I suppose I understand now why there is controversy around updating packages.

hnyman · April 27, 2022, 6:26pm

The firstboot/reset requires ROM+overlay approach, which OpenWrt uses for normal routers with flash (like wrt1900ac of OP), but it may not apply to memory card based systems like yours.

dave14305 · April 27, 2022, 6:47pm

Interesting user comment on GH about this ucode update:

github.com/openwrt/openwrt

ucode: update to latest Git HEAD

committed 07:52AM - 25 Apr 22 UTC

jow-

+5 -5

e14b099 syntax: implement support for ES6 template literals 111cf06 vm: stop exe…cuting bytecode on return of nested calls 33f1e0b treewide: move json-c compat shims into internal header file e0e9431 vm: move unhandled exception reporting out of `uc_vm_execute_chunk()` 2b59140 vm: fix callframe double free on unhanded exceptions 7d7e950 main: abort when failing to load a preload library 1032a67 lib: let `json()` accept input objects implementing `read()` method 5ee68d5 fs: implement `fs.readfile()` and `fs.writefile()` df6b861 ci: debian: change path before attempting to invoke Git operations dfaf05a ci: debian: automatically update changelog from Git tag 34f3c45 ci: fix YAML syntax of Debian workflow e956bcf fs: fix off-by-one in fs.dirname() function 6fc4b6c .gitignore: fix overmatching patterns, blacklist cram .venv 7c2e082 build: remove legacy json-c check 77942af build: add polyfills for older libjson-c versions 0b4aaa3 CI: build Debian package f404285 debian: Add package definition a37f654 types: fix escape sequence encoding of high byte values in JSON strings aae5312 Update README.md 8134e25 build: fix symlink install target 87c7296 treewide: replace some leftover "utpl" occurrences, update .gitignore 7d27ad5 build: only stage ucc symlink if compile support is enabled 171402f lib: add date and time related functions 8b5dc60 lib: provide API function to obtain stdlib function implementations eb0d2f1 main: turn ucode into multicall executable 28ee7e1 uloop: add support for tasks 753dea9 CI: build on macOS 668c5c0 lib: add argument position support (`%m$`) to `sprintf()` and `printf()` ab46fdf treewide: remove legacy json-c include directives b8f49b1 tests: 21_regex_literals: generalize syntax error test case fd2e5e7 tests: 16_sort: fix logic flaw exposed on OS X 2c71bf2 tests: run_tests.sh: pass dummy value to `-T` flag 55c4a90 lib: disallow zero padding for %s formats 0d05cb5 tests: run_tests.sh: use greadlink if available 271e520 resolv: make OS X compatible d13c320 fs: avoid Linux specific sys/sysmacros.h include on OS X 33397a3 uloop: use execvp() on OS X bafdc8f lib: add naive sigtimedwait() stub for OS X ada1585 build: consolidate CMakeLists.txt and cover OS X deviations befbb69 include: add OS X compatible endian.h header 49838a8 include: rename include guards to avoid clashes with system headers 91f65de nl80211: add missing attributes and correct some attribute flags b4a1fd5 lib: adjust require(), render() and include() raw mode semantics 4618807 main: rework CLI frontend 73dcd78 lib: fix potential integer underflow on empty render output c402551 vm: fix crash on object literals with non-string computed properties efe8a02 syntax: support add new operators 078d686 ubus: add event support 6c66c83 ubus: refactor error and argument handling 1cb04f9 ubus: add object publishing, notify and subscribe support 0e85974 uloop: clear errno before integer conversion attempts 05bd7ed types: treat resource type prototypes as GC roots a2a26ca lib: introduce uloop binding 6b6d01f vm: release this context on exception in managed method call 1af23a9 tests: fix proto() testcase 4ce69a8 fs: implement access(), mkstemp(), file.flush() and proc.flush() Signed-off-by: Jo-Philipp Wich <jo@mein.io> (backported from commits cef3e6a69c4751189e946cf2ff0e00304f9fff6e, 0400774a1001835b912ecf2097bba1306deba9f8 and c59704334c4a01bc18ab489f86d4ca7d2ea3cb8f)

Running opkg upgrade ... won't install the newer libucode20220322 package. I had to manually install the lib package and force-reinstall those upgradable packages. Was I missing some flags when running opkg upgrade , or are there something wrong in the ucode/Makefile ?

pronichkin · April 27, 2022, 7:03pm

Yup. Saw the comment to GitHub issue I opened. Installing libucode20220322 seemed to solve everything. (I did not even have to force-reinstall other packages, as you're saying.)

I will still consider resetting, given that my router has been in this state for entire night. But at least we have a solution for now. Thanks everyone!

tmomas · April 27, 2022, 9:35pm

If your problem is solved, please consider marking this topic as [Solved]. See How to mark a topic as [Solved] for a short how-to.

jow · April 27, 2022, 9:42pm

Actually ucode is supposed to depend on libucode20220322 (and not the virtual libucode) so that the required update is automatically installed along, but there appear to be some metadata issues with the buildbot generated repositories...

UPDATE:

Fixed with

Once the packages are updated in the repo, an opkg upgrade firewall4 should pull an upgraded ucode which in turn will pull the required upgraded libucode20220322 along.

system · May 7, 2022, 9:43pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.