The future is now: opkg vs apk

[spence]

@efahl

apk query is my new favorite command. It really is pretty cool.

Nice!
I like apk adbdump. With a bit of sed magic, the output is the same basic text format as the extracted opkg lists and /usr/lib/opkg/status with different field names and values when considering it to be a flat file form of a database.

tldr; The rest of this post is just an example of what the output of apk adbdump could be used for as well as the text form of the opkg index files.

Out of curiosity of understanding the structure and uniformity of the format, I was developing an awk script that provided some info on the fields. It works on the opkg lists and apk adbdumps.

Persons reading this might have other uses for a text dump of the apk index database(s).

Run against the combined opkg lists, I thought some stats were interesting:
( some extra info provided for context )

OpenWrt 24.10.0 with opkg

ubus call system board

ubus call system board
{
	"kernel": "6.6.73",
	"hostname": "R4S-wrt",
	"system": "ARMv8 Processor rev 4",
	"model": "FriendlyElec NanoPi R4S",
	"board_name": "friendlyarm,nanopi-r4s",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "24.10.0",
		"revision": "r28427-6df0e3d02a",
		"target": "rockchip/armv8",
		"description": "OpenWrt 24.10.0 r28427-6df0e3d02a",
		"builddate": "1738624177"
	}
}

opkg update
...
cat r4s-opkg-index | pkgFileStats.awk -v output=stats

Field-Name     Count Min Max
-------------- ----- --- ---
Package         6083   1   1
Version         6083   2   2
Depends         6081   3   3
Conflicts         82   4   4
Provides         191   3   5
Alternatives      38   4   5
License         5856   4   6
Section         6083   4   7
URL             1810   5   8
ABIVersion        88   7   9
Status             5   7   9
CPE-ID           829   5  10
Essential          8   7  10
Architecture    6083   5  11
Installed-Size  6083   6  12
Filename        6083   7  13
Size            6083   8  14
SHA256sum       6083   9  15
Description     6083  10  16

Number of fields seen: 19
Number of field-Name sets seen: 42

cat ~/data/code/r4s-opkg-index | ~/data/code/pkgFileStats -v output=table
usage len Package Version Depends Conflicts Provides Alternatives License Section URL ABIVersion Status CPE-ID Essential Architecture Installed-Size Filename Size SHA256sum Description
----- --- ------- ------- ------- --------- -------- ------------ ------- ------- --- ---------- ------ ------ --------- ------------ -------------- -------- ---- --------- -----------
 4175  11 Package Version Depends                                 License Section                                        Architecture Installed-Size Filename Size SHA256sum Description
  781  12 Package Version Depends                                 License Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
  677  13 Package Version Depends                                 License Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
  166  11 Package Version Depends                                         Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
   57  15 Package Version Depends           Provides              License Section URL ABIVersion        CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
   31  10 Package Version Depends                                         Section                                        Architecture Installed-Size Filename Size SHA256sum Description
   29  15 Package Version Depends Conflicts Provides              License Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
   17  12 Package Version Depends           Provides              License Section                                        Architecture Installed-Size Filename Size SHA256sum Description
   16  13 Package Version Depends Conflicts Provides                      Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
   15  14 Package Version Depends                    Alternatives License Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
   14  12 Package Version Depends                                 License Section                       CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
   13  15 Package Version Depends           Provides Alternatives License Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
   11  13 Package Version Depends           Provides              License Section     ABIVersion                         Architecture Installed-Size Filename Size SHA256sum Description
    9  14 Package Version Depends           Provides              License Section URL ABIVersion                         Architecture Installed-Size Filename Size SHA256sum Description
    9  13 Package Version Depends Conflicts                       License Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
    6  14 Package Version Depends           Provides Alternatives License Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
    5  14 Package Version Depends Conflicts                       License Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    5  13 Package Version Depends           Provides              License Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
    4  12 Package Version Depends                                         Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    4  12 Package Version Depends Conflicts Provides                      Section                                        Architecture Installed-Size Filename Size SHA256sum Description
    4  14 Package Version Depends           Provides              License Section URL                   CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    4  12 Package Version Depends Conflicts                       License Section                                        Architecture Installed-Size Filename Size SHA256sum Description
    3  14 Package Version Depends Conflicts Provides              License Section                       CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    3  13 Package Version Depends Conflicts Provides              License Section                                        Architecture Installed-Size Filename Size SHA256sum Description
    2  14 Package Version Depends           Provides              License Section     ABIVersion        CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    2  13 Package Version Depends           Provides                      Section URL ABIVersion                         Architecture Installed-Size Filename Size SHA256sum Description
    2  14 Package Version Depends Conflicts Provides              License Section     ABIVersion                         Architecture Installed-Size Filename Size SHA256sum Description
    2  12 Package Version Depends                    Alternatives License Section                                        Architecture Installed-Size Filename Size SHA256sum Description
    2  12 Package Version Depends Conflicts                               Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
    2  15 Package Version                   Provides              License Section URL ABIVersion Status        Essential Architecture Installed-Size Filename Size SHA256sum Description
    2  14 Package Version Depends                                 License Section URL            Status        Essential Architecture Installed-Size Filename Size SHA256sum Description
    1  13 Package Version Depends                    Alternatives License Section                       CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    1  15 Package Version Depends           Provides              License Section URL                   CPE-ID Essential Architecture Installed-Size Filename Size SHA256sum Description
    1  15 Package Version Depends Conflicts                       License Section URL                   CPE-ID Essential Architecture Installed-Size Filename Size SHA256sum Description
    1  11 Package Version Depends Conflicts                               Section                                        Architecture Installed-Size Filename Size SHA256sum Description
    1  15 Package Version Depends Conflicts Provides              License Section     ABIVersion        CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    1  16 Package Version Depends Conflicts Provides              License Section URL ABIVersion        CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description
    1  14 Package Version Depends Conflicts Provides              License Section URL                                    Architecture Installed-Size Filename Size SHA256sum Description
    1  13 Package Version Depends                                 License Section URL                          Essential Architecture Installed-Size Filename Size SHA256sum Description
    1  13 Package Version Depends           Provides Alternatives License Section                                        Architecture Installed-Size Filename Size SHA256sum Description
    1  16 Package Version Depends           Provides              License Section URL ABIVersion Status        Essential Architecture Installed-Size Filename Size SHA256sum Description
    1  11 Package Version Depends                                         Section                       CPE-ID           Architecture Installed-Size Filename Size SHA256sum Description

It works on the opkg status file for what is installed:

cat /usr/lib/opkg/status | pkgFileStats.awk -v output=stats
Field-Name     Count Min Max
-------------- ----- --- ---
Package          257   1   1
ABIVersion        48   2   2
Version          257   2   3
Depends          255   3   4
Provides          59   4   5
Conflicts          9   4   6
Status           257   4   7
Essential          8   5   7
Architecture     257   5   8
Conffiles         20   6   8
Installed-Time   257   6   9
Alternatives      11   7   9
Auto-Installed   168   7  10

Number of fields seen: 13
Number of field-Name sets seen: 25

.
.
APK on snapshot:

ubus call system board

ubus call system board
{
	"kernel": "6.6.87",
	"hostname": "OWrtSNAPSHOT",
	"system": "11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz",
	"model": "innotek GmbH VirtualBox",
	"board_name": "innotek-gmbh-virtualbox",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "SNAPSHOT",
		"firmware_url": "https://downloads.openwrt.org/",
		"revision": "r29266-8ffb3b2829",
		"target": "x86/64",
		"description": "OpenWrt SNAPSHOT r29266-8ffb3b2829",
		"builddate": "1744991505"
	}
}

Sample run for one of the apk index files exported with apk adbdump <apk-index-file> downloaded with apk update on a 24.10 snapshot build:

apk update
...
apk adbdump /tmp/cache/apk/APKINDEX.af0f1c9d.tar.gz | sed -e 's/^  - /\n/' -e 's/^    //' | grep -v "^#" > /tmp/apkindex_af0f1c9d.txt

cat /tmp/apkindex_af0f1c9d.txt | pkgFileStats.awk -v output=stats
Field-Name        Count Min Max
----------------- ----- --- ---
name                 58   1   1
version              58   2   2
hashes               58   3   3
description          58   4   4
arch                 58   5   5
license              54   6   6
origin               58   6   7
maintainer           58   7   8
url                  56   8   9
installed-size       58   8  10
file-size            58   9  11
provider-priority     3  12  12
depends              58  10  13
provides              3  14  14

Number of fields seen: 14
Number of field-Name sets seen: 5

.
.
cat /tmp/apkindex_af0f1c9d.txt | pkgFileStats.awk -v output=table
usage len name version hashes description arch license origin maintainer url installed-size file-size provider-priority depends provides
----- --- ---- ------- ------ ----------- ---- ------- ------ ---------- --- -------------- --------- ----------------- ------- --------
   50  12 name version hashes description arch license origin maintainer url installed-size file-size                   depends 
    3  11 name version hashes description arch         origin maintainer url installed-size file-size                   depends 
    3  14 name version hashes description arch license origin maintainer url installed-size file-size provider-priority depends provides
    1  10 name version hashes description arch         origin maintainer     installed-size file-size                   depends 
    1  11 name version hashes description arch license origin maintainer     installed-size file-size                   depends 

One thing I found interesting with this info is that only a few records have or are missing particular fields. The awk script will report the record name for those:

Depends missing for:libgcc1 libstdcpp6
Status present:libatomic1 libgcc1 libpthread librt libstdcpp6
Essential present for:libatomic1 libgcc1 libpthread librt libstdcpp6

( as run on the combined opkg lists )

If you read this far, you might wonder what the Min and Max values are in the stats output. They are the min and max field positions for that field name as seen within all the records looked at. It is used to derive the sequences of field names in the table output.

The script is a work in progress with only occasional time spent on it. Being pure awk, it runs on OpenWrt and other posix gnu/linux type systems. It is not ready to share yet.

[efahl]

Oh yeah, apk adbdump is pretty good. That was my go-to for quite a while when inspecting the v3 objects, as it works on the package indexes and the packages themselves, so sort of a Swiss army knife.

Nice to see that things are mostly comparable; some of those missing attributes are strange, and I'm not sure where/when "Essential" came to be an opkg thing.

When I was fixing https://github.com/openwrt/openwrt/pull/19278, though, adbdump's "gimme everything!" was getting in the way, as I only wanted to selectively list certain fields or not (specifically the ABI-version tags) for the packages in question

That led to this simple solution, where if the make variable STRIP_ABI is true, we add the tags field to the output, otherwise we leave it off. The subsequent processor sees, or doesn't, the ABI-version tag, so the output contains either the sanitized or versioned names. Thequery's --fields option made this so easy.

apk query --format json --fields name,version,$(if $(STRIP_ABI),tags) --installed '*' | ...

[sngsnd]

1. there are posts like this Major Change Notice: New Package Manager or this The future is now: opkg vs apk - #9 by efahl and even documentation on how to use apk manager once you have it on your system
2. And, haha, there is ZERO. ZERO guide on how a random person could install an apk manager on his already edge wrt 24 to try using snapshot packages.
3. and guess what. most common idea like

root@OpenWrt:~# opkg install apk-tools
Unknown package 'apk-tools'.
Collected errors:
 * opkg_install_cmd: Cannot install package apk-tools.
root@OpenWrt:~# opkg install apk
Unknown package 'apk'.
Collected errors:
 * opkg_install_cmd: Cannot install package apk.
opkg list |grep -i apk - no output

simply gives nothing.

how do you guys deal with making things clear for everyone, if there is even zero docs on how to install new things.

[skuzzer]

Try apk update then apk add apk-tools etc.

[el_charlie]

@sngsnd

And, haha, there is ZERO. ZERO guide on how a random person could install an apk manager on his already edge wrt 24 to try using snapshot packages.

You don't install an apk package manager, you install a sysupgrade image that includes apk. Currently, you need to install a SNAPSHOT build.

This process is not that complicated becasue you can run owut upgrade --version-to snapshot and it'll deal with the new package manager and it'll generate a sysupgrade image with apk built in and install it right away. If there are some incompatibilities, owut should tell you.

[hnyman]

@sngsnd

ZERO guide on how a random person could install an apk manager on his already edge wrt 24 to try using snapshot packages.

1. there is no APK in OpenWrt 24.10.x. 24.10 still uses opkg.
2. you should not use packages from a different release branch than the firmware itself. Thus, in 24.10 firmware you use opkg to install 24.10 packages. And in main/master snapshot firmware, you use APK to install master packages.