The future is now: opkg vs apk

hnyman · 2024-11-14T14:30:47.910Z

Well, it is good to remember that the apk change has only been done on the bleeding-edge development snapshots. Anything may change there at any time...

The stable OpenWrt release 23.05 and currently also the forthcoming 24.10 are still using opkg.
If you want the almost most current code, just take the 24.10-SNAPSHOT that is the two-weeks ago bleeding edge.

(It is possible that 24.10 will be switched later to apk, and for just that reason the apk is in wider test use in the development version now.)

efahl · 2024-11-14T14:53:37.846Z

hnyman:

If you want the almost most current code, just take the 24.10-SNAPSHOT that is the two-weeks ago bleeding edge.

The only problem is that 24.10 isn't exposed through the ASU system, so it doesn't appear in the Firmware Selector yet... (Or auc/owut/luci app.) Lots of people are dependent on those tools, which is causing a lot of this grief (which could be alleviated, like you say above).

@hauke Got any guesses as to how long before v24.10.0-rc1 is a thing? That should filter automatically into ASU as soon as @aparcar merges https://github.com/openwrt/asu/pull/1066 or something similar that adds the 24.10 branch...

hnyman · 2024-11-14T15:14:52.051Z

efahl:

@hauke Got any guesses as to how long before v24.10.0-rc1 is a thing?

Right now I think that 24.10.0-rc1 might need to be done without apk enabled, yet.
So that 24.10.0-rc1 would offer a recent OpenWrt codebase for those who want pre-compiled images. And when apk seems stabilized enough, it could then be enabled in 24.10, too, likely before rc2.

Ps.

efahl:

The only problem is that 24.10 isn't exposed through the ASU system

We have the normal download images available https://downloads.openwrt.org/releases/24.10-SNAPSHOT/ , as well as packages built.
Yes, there is no automatics for tailored firmware via ASU, yet, but that does not prevent anybody from taking the normal sysupgrade image, or to use the imagebuilder, or to build from sources (like I always do)

efahl · 2024-11-14T15:35:25.147Z

hnyman:

Right now I think that 24.10.0-rc1 might need to be done without apk enabled, yet.

Exactly just a tag on current 24.10-SNAPSHOT, thus reducing the pressure for support here.

And when apk seems stabilized enough, it could then be enabled in 24.10, too, likely before rc2.

Wow, you're pretty optimistic!

hnyman:

We have the normal download images available

The problem there is that the downloads site explicitly hides the links to the release-SNAPSHOT directories for some reason, so discoverability is nigh on impossible.

tboege · 2024-11-15T00:31:23.055Z

Tried snapshot r28072-b902284290 from today (2024-11-15).
apk can install some packages, eg. luci. Some fails eg:

root@OpenWrt:~# apk add speedtestcpp
(1/5) Installing libnghttp2-14 (1.63.0-r1)
Executing libnghttp2-14-1.63.0-r1.post-install
(2/5) Installing libcurl4 (8.10.1-r1)
Executing libcurl4-8.10.1-r1.post-install
(3/5) Installing libstdcpp6 (13.3.0-r4)
 23% ##########################

and then it is just stuck. CTRL-Z +bg and ps w shows
8128 root 1868 S wget -q -T 60 https://downloads.openwrt.org/snapshots/targets/ipq40xx/generic/packages/libstdcpp6-13.3.0-

Another example is

root@OpenWrt:~# apk add luci-proto-wireguard
( 1/13) Installing luci-lib-uqr (24.315.52043~d650688)
Executing luci-lib-uqr-24.315.52043~d650688.post-install
( 2/13) Installing resolveip (2)
Executing resolveip-2.post-install
( 3/13) Installing ip-tiny (6.11.0-r1)
Executing ip-tiny-6.11.0-r1.post-install
add alternative: /sbin/ip -> /usr/libexec/ip-tiny
( 4/13) Installing kmod-crypto-lib-chacha20 (6.6.60-r1)
ERROR: kmod-crypto-lib-chacha20-6.6.60-r1: ADB integrity error
( 5/13) Installing kmod-crypto-lib-poly1305 (6.6.60-r1)
ERROR: kmod-crypto-lib-poly1305-6.6.60-r1: ADB integrity error
( 6/13) Installing kmod-crypto-lib-chacha20poly1305 (6.6.60-r1)
ERROR: kmod-crypto-lib-chacha20poly1305-6.6.60-r1: ADB integrity error
( 7/13) Installing kmod-crypto-kpp (6.6.60-r1)
ERROR: kmod-crypto-kpp-6.6.60-r1: ADB integrity error
( 8/13) Installing kmod-crypto-lib-curve25519 (6.6.60-r1)
ERROR: kmod-crypto-lib-curve25519-6.6.60-r1: ADB integrity error
( 9/13) Installing kmod-udptunnel4 (6.6.60-r1)
ERROR: kmod-udptunnel4-6.6.60-r1: ADB integrity error
(10/13) Installing kmod-udptunnel6 (6.6.60-r1)
ERROR: kmod-udptunnel6-6.6.60-r1: ADB integrity error
(11/13) Installing kmod-wireguard (6.6.60-r1)
ERROR: kmod-wireguard-6.6.60-r1: ADB integrity error
(12/13) Installing wireguard-tools (1.0.20210914-r4)
Executing wireguard-tools-1.0.20210914-r4.post-install
(13/13) Installing luci-proto-wireguard (24.315.52043~d650688)
Executing luci-proto-wireguard-24.315.52043~d650688.post-install
8 errors; 18 MiB in 168 packages
root@OpenWrt:~#

stangri · 2024-11-15T01:02:15.517Z

Following @aparcar instructions I've set up the repo for APK binaries for my packages: https://dev.melmac.net/apk/, if a kind soul on snapshots image with apk bundled in can post instructions on how can a repo be added to a live OpenWrt install and test if the repo linked above works as a repository for OpenWrt, I'd be much obliged.

Some of the apk binaries posted up are noarch, while some are x86-64 only, I wonder if it produces any unwanted results on non-x86 systems? Also, none of the packages above are signed with the OpenWrt key, so I wonder if any other settings on apk-enabled OpenWrt need to be changed.

efahl · 2024-11-15T01:21:40.807Z

I added your repo and apk doesn't like something.

$ cat /etc/apk/repositories
https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb
https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb
https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb
https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb
https://dev.melmac.net/apk/APKINDEX.tar.gz

$ apk update
fetch https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb
fetch https://dev.melmac.net/apk/APKINDEX.tar.gz/x86_64/APKINDEX.tar.gz
WARNING: updating and opening https://dev.melmac.net/apk/APKINDEX.tar.gz: remote server returned error (try 'apk update')
 [https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb]
1 unavailable, 0 stale; 9901 distinct packages available

Not sure what the deal is, as wget works just fine. (And apk update doesn't have an verbose options, so no help to be had there.)

$ wget https://dev.melmac.net/apk/APKINDEX.tar.gz
Downloading 'https://dev.melmac.net/apk/APKINDEX.tar.gz'
Connecting to 104.21.7.165:443
Writing to 'APKINDEX.tar.gz'
APKINDEX.tar.gz      100% |*******************************|  2532   0:00:00 ETA
Download completed (2532 bytes)

Scanning the index with adbdump looks fine, too.

$ apk adbdump APKINDEX.tar.gz | head
#%SCHEMA: 78646E69
# ADB block, size: 4460, compat: 0, ver: 0
packages: # 9 items
  - name: adblock-fast
    version: 1.1.2-r12
    hashes: |
      c782c3358daaadecd8ec74aea361f4ca54c6921f03a00fffd5ba91e5ef34b94f
    description: |
      Fast AdBlocking script to block ad or abuse/malware domains with Dnsmasq, SmartDNS or Unbound. Script supports local/remote list of domains and hosts-files for both block-listing and allow-listing. Please see https://docs.openwrt.melmac.net/adblock-fast/ for more information.
    arch: all

ka2107 · 2024-11-15T01:22:34.313Z

efahl:

Exactly just a tag on current 24.10-SNAPSHOT, thus reducing the pressure for support here.

There's already a ticket requesting option in https://firmware-selector.openwrt.org/ to build 23.05-SNAPSHOT images. In the same ticket I have added a comment requesting option to also build 24.10-SNAPSHOT images.

github.com/openwrt/firmware-selector-openwrt-org

Please allow to build 23.05-SNAPSHOT images

opened 12:02PM - 22 Mar 24 UTC

lukasz1992

Hi, that works but is not exposed to UI.

efahl · 2024-11-15T01:24:02.057Z

Thanks! I was digging around for that exact issue just this morning and couldn't find it.

ka2107 · 2024-11-15T01:28:32.436Z

efahl:

The problem there is that the downloads site explicitly hides the links to the release-SNAPSHOT directories for some reason, so discoverability is nigh on impossible.

Yeah, neither https://downloads.openwrt.org/ nor https://downloads.openwrt.org/releases/ has any links to specific RELEASE-SNAPSHOT URLs.

Who should we contact to get this resolved? OR should we just create a ticket in https://github.com/openwrt/openwrt/issues ?

efahl · 2024-11-15T01:49:25.767Z

@jow appears to be the last one to modify the downloads site, but https://git.openwrt.org/?p=web/downloads-openwrt-org.git;a=summary is a couple years old and I'm quite sure that's not what is actually running there. I have a vague memory that I had found the current site source on gitlab or somewhere 6+ months ago, but I can't locate it now.

ka2107 · 2024-11-15T02:06:09.713Z

efahl:

I have a vague memory that I had found the current site source on gitlab or somewhere 6+ months ago, but I can't locate it now.

This one?

GitLab

OpenWrt / web / downloads-openwrt-org · GitLab

downloads.openwrt.org website

efahl · 2024-11-15T02:35:45.372Z

Got a little farther, ran strace and figured out where the index would be saved (APKINDEX.06edf16b.tar.gz):

writev(2, [{iov_base="fetch https://dev.melmac.net/apk"..., iov_len=55}, {iov_base="\n", iov_len=1}], 2fetch https://dev.melmac.net/apk/x86_64/APKINDEX.tar.gz
) = 56
writev(4, [{iov_base="fetch https://dev.melmac.net/apk"..., iov_len=56}, {iov_base=NULL, iov_len=0}], 2) = 56
openat(6, "APKINDEX.06edf16b.tar.gz.tmp", O_RDWR|O_CREAT|O_TRUNC|O_LARGEFILE|O_CLOEXEC, 0644) = 7
mmap(NULL, 131148, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f570e4ee000
pipe2([8, 9], O_CLOEXEC)                = 0
rt_sigprocmask(SIG_BLOCK, ~[], [], 8)   = 0
pipe2([10, 11], O_CLOEXEC)              = 0
clone(child_stack=0x7fffc0399438, flags=CLONE_VM|CLONE_VFORK|SIGCHLD) = 29405
close(11)                               = 0
read(10, "", 4)                         = 0
close(10)                               = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
close(9)                                = 0
read(8, "", 131072)                     = 0
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=29405, si_uid=0, si_status=8, si_utime=3 /* 0.03 s */, si_stime=0} ---
wait4(29405, [{WIFEXITED(s) && WEXITSTATUS(s) == 8}], 0, NULL) = 29405
close(8)                                = 0
munmap(0x7f570e4ee000, 135168)          = 0
close(7)                                = 0
unlinkat(6, "APKINDEX.06edf16b.tar.gz.tmp", 0) = 0

So, renamed the wget version:

$ mv APKINDEX.tar.gz /var/cache/apk/APKINDEX.06edf16b.tar.gz

Then it "works":

$ apk info adblock-fast
WARNING: opening from cache https://dev.melmac.net/apk/: UNTRUSTED signature
adblock-fast-1.1.2-r10 description:
Fast AdBlocking script to block ad or abuse/malware domains with Dnsmasq, SmartDNS or Unbound. Script supports local/remote list of domains and hosts-files for both block-listing and allow-listing. Please see https://docs.openwrt.melmac.net/adblock-fast/ for more information.

adblock-fast-1.1.2-r10 webpage:
https://github.com/stangri/adblock-fast/

adblock-fast-1.1.2-r10 installed size:
104 KiB

efahl · 2024-11-15T02:38:54.739Z

ka2107:

This one?

Yeah, that's probably it, but it's even older. It does show code that mirrors what we're seeing on the website (Our Nemesis: https://gitlab.com/openwrt/web/downloads-openwrt-org/-/blob/main/dir-index.cgi?ref_type=heads#L395).

evs · 2024-11-15T06:33:19.157Z

Hmm. Just a thought regarding the naming of the topic and how to communicate this.

New doesn't necessarily imply that opkg is immediately gone in snapshot/24.10+ in the title below?

Major Change Notice: New Package Manager Release and security announcements

As of November 2024 for snapshots and future stable release builds (including the upcoming 24.10 series), the package manager in OpenWrt has changed from opkg to apk. This new package manager offers a number of advantages over the older opkg system and is a significant milestone in the development of the OpenWrt platform. The older opkg package manager has been deprecated and is no longer part of OpenWrt. As this represents a fundamental shift in the method for installing and managing packages…

i.e. There's no transition period. IDK what the best wording is. Relies on people reading more than the title =P

"Different package manager for 24.10+, opkg removed" or "opkg replaced with apk for 24.10+". Anyway, current title does get the message accross fine =)

psherman · 2024-11-15T06:36:43.359Z

I could edit it the title to something like "Major Change Notice: New Package Manager and opkg Deprecation"

That should make it more obvious, but not sure if it becomes too long of a title... thoughts?

evs · 2024-11-15T06:40:30.472Z

I have no idea, your guess is as good as mine. Sorry for pointing out a potential issue whilst not providing a good solution =(

It does get a little long haha.

I'm still thinking of a good way to draw users in to read. Opkg going away for next release seems like a good draw but eh.

another edit:
Also of note. At least for me it's now gone from the latest in the release and announcements category, when browsing categories, even though it's pinned.

2024-11-15-181452_1114x182_scrot1114×182 21.1 KB

But I'm guessing this is a function of the way activity is done on Discourse.

psherman · 2024-11-15T07:18:37.480Z

The topic will remain pinned on a per-user basis until that user reads the thread.

evs · 2024-11-15T07:19:22.945Z

Thanks for the clarification!

predators_46 · 2024-11-15T07:33:02.709Z

Is this the final apk that will no longer be replaced with apt?