RFC pkgs.staging.openwrt.org

Hey all,

tl;dr checkout https://pkgs.staging.openwrt.org/

As some may experienced, openwrt.org is slow an one (of many reasons) is that it stores and renders tables with all packages.

Based on feedback the experience is neither convincing nor does it perform well within a DokuWiki.

After asking our fellow friends from Alpine Linux1 respectively postmarketOS2 I got an okay to fork and deploy the tool apkbrowser, which is now pkgbrowser3 since it currently handles OPKG packages.

I’ll refactor and commit the update-script later, essentially it polls our buildbots and whenever a build is completed, the database is updated. For that reason some packages are still missing. With more time missing packages/archs are added.

The tool nor server are tested under load but I fell quite positive it’s more fun than our wiki implementation.

If this turns out to fly, I’ll link it in the wiki, move it to pkgs.openwrt.org and remove all package functionality from the wiki.

Thanks to Carlo Landmeter et al for implementing such a handy and adoptable tool!

Best,
Paul

9 Likes

Good! :smile:

As long as web crawlers can pick up the details and it becomes searchable from Google that’s fantastic.
99% of the time I reach package pages by googling “pkgname openwrt” and it’s usually the top result.

1 Like

This looks great, thank you!

I have used Alpine's package searching UI quite extensively and so would benefit from having OpenWrt's use a similar interface.

It does feel more responsive then the current wiki version.

Outstanding! Looking forward for when it's released, so we could send a curl search page to the curl project so that they could scrape it and fetch and show version of curl available for OpenWrt!

@aparcar

The maintainer drop-down field needs some UTF love to fix some (Asian) scripts. At the bottom it shows:

Maintainer 源 文雨

for packages belonging to:

PKG_MAINTAINER:=源 文雨

Also for:

Maintainer Álvaro Fernández Rojas

which should be:

PKG_MAINTAINER:=Álvaro Fernández Rojas

2 Likes

Much better, much faster, however... Doing a search thus (arbitrary arch choice!) returns multiple identical results:

https://pkgs.staging.openwrt.org/packages?name=curl*&branch=openwrt-23.05&arch=arm_cortex-a9&origin=

With a more broad search the result list becomes quite unwieldy.

1 Like

I think that the PKG_MAINTAINER should contain only ASCII symbols to avoid unexpected problems in unexpected places like the maintainers dropdown.

@aparcar what do you think about this?

I just want to clarify. We need a few features:

  1. Show a package info: name, dependencies, issues, source link
  2. Search for a package

Technically speaking all the packages with their meta are already on the downloads.openwrt.org. So instead of a the apkbrowser frontend written in Python we can make a static pages.

It can be a generated html files but that would be annoying to maintain.
Or just a single html page (SPA) that will render the package meta to HTML.
Extracting the meta from the ipk file is overkill but we already have Packages.gz file which contains description of all packages but still is not that big. Anyway to show dependencies with a description we would need to have their meta too.

Basically the Luci opkg page works exactly like this. Maybe it's even possible to reuse it. Also the Firmware Selector would benefit of this. People who have JS disabled may be disappointed but they are smart enough to use the opkg search.

The search in the apkbrowser is anyway not that good: you need to not forget to specify wildcard * which was confusing for me when I searched for openssh and nothing was shown to me. There is no search by description, category but we have a search by a maintainer which is not really useful.

What do you think about such the package viewer? I can try to make POC of it and if it will be useful it may be placed on the downloads page independently of the apkbrowser.

Please feel free to create a PoC and I'm happy to look at it. I don't intent do create a complex project, that's also the reason why I reused the existing apkbrowser. If you come up with something else, great!

1 Like

That may take a time and I don't want to block the apkbrowser which is useful just today. I'll back here once have something to show.

Is it safe to link to this URL (in luci stuff for example)? Or is it a temporary one?

ping @aparcar

1 Like

Eventually it'd move to pkgs.openwrt.org and it's not fully ready yet, please be patient :slight_smile:

Hi @aparcar - I wonder what your take is on stuff like this?

Problem 1: in some Makefile, the PKG_MAINTAINER field is co-opted for multiple entries, which I think is a good idea. But in these dual maintainers in the luci repo with e.g.

./applications/luci-app-cloudflared/Makefile:PKG_MAINTAINER:=Hilman Maulana <hilman0.0maulana@gmail.com>, Sergey Ponomarev <stokito@gmail.com>
./applications/luci-app-strongswan-swanctl/Makefile:PKG_MAINTAINER:=Nicholas Smith <nicholas@nbembedded.com>, Lukas Voegl <lvoegl@tdt.de>

They don't appear in any searches. But currently e.g. this shows as Hans Dedecker. Which is definitely not a maintainer for this app.

Problem 2:

If I started a search and already selected an author, I cannot "deselect" the Maintainer field (choose nobody) on subsequent searches. Maintainer is "greyed out" in the dropdown. Only way is to manually delete the &maintainer= field from the URL and mash enter to reload.

Hej, thanks for testing. I fixed the former problem, you can now deselect filters. Regarding the problematic maintainers, I'll try to figure out we're things go wrong.

I'm not terribly excited to add extra logic to handle multiple maintainers and would always just show the first. Feel free to give it a try if you finde the time.

1 Like

Works fine. Just with multiple maintainers, it's OK to show all in the maintainers field, but filter only on the one (as would seem logical). Anyway, if extra logic isn't too complex...

A couple of other 'could be bugs'?

Search for package *splash* - maintainer == None. Click on maintainer None: No matching packages found...

Search for package *splash*, click through to luci-app-splash. Maintainer == John Crispin. Some fallback default? Makefile does not contain anybody specific.

Search for package *adblock*, maintainer == Stan Grishin. Click on maintainer. 7 pages. Click on 6 or 7. Nothing found. Pagination then shows up to 10.

Is there some built in amount for entries per page? Perhaps the pagination value needs revealing as a filter.