# Package pages question

**URL:** <https://forum.openwrt.org/t/package-pages-question/113300>\
**Category:** Talk about Documentation\
**Created:** [November 29, 2021, 7:50pm UTC](https://forum.openwrt.org/t/package-pages-question/113300 "2021-11-29T19:50:19Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![stangri](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stangri/32/58_2.png) [@stangri](https://forum.openwrt.org/u/stangri)\
**Post date:** [November 29, 2021, 7:50pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/1 "2021-11-29T19:50:19Z")

</div>

Hello @tmomas

The package pages, like this one: [https://openwrt.org/packages/pkgdata/curl](https://openwrt.org/packages/pkgdata/curl) list the OpenWrt version/package version from the current release branch (21.02 ATM), however if you follow the links to the architectures they lead to the snapshot packages.

Would it be possible to include a section called Binaries, which would be the same list as Architectures but pointing to the actual IPKs? I've been talking to curl people (their download page links for OpenWrt are hilariously outdated) about having the OpenWrt links up-to-date and they use HTML scraper so having a new section called Binaries would be the easiest way for them.

Thanks!

---

<div class="post-metadata">

**Author:** ![tmomas](https://forum.openwrt.org/letter_avatar/tmomas/32/5_5575768a8748004e209b776fc1b2916d.png) [@tmomas](https://forum.openwrt.org/u/tmomas)\
**Post date:** [November 29, 2021, 8:11pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/2 "2021-11-29T20:11:07Z")

</div>

> [@stangri](#):
>
> if you follow the links to the architectures they lead to the snapshot packages.

Can you give an example for that?

---

<div class="post-metadata">

**Author:** ![stangri](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stangri/32/58_2.png) [@stangri](https://forum.openwrt.org/u/stangri)\
**Post date:** [November 29, 2021, 8:29pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/3 "2021-11-29T20:29:20Z")

</div>

Thanks for your prompt reply.

If you click any architecture listed for the package, you get to (for example) [https://openwrt.org/docs/techref/instructionset/aarch64\_cortex-a53](https://openwrt.org/docs/techref/instructionset/aarch64_cortex-a53) which at the very top has links to the snapshots paclages.

---

<div class="post-metadata">

**Author:** ![tmomas](https://forum.openwrt.org/letter_avatar/tmomas/32/5_5575768a8748004e209b776fc1b2916d.png) [@tmomas](https://forum.openwrt.org/u/tmomas)\
**Post date:** [November 29, 2021, 8:57pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/4 "2021-11-29T20:57:12Z")

</div>

OK, now I see. Every other architecture I clicked before was on 18.06, therefore I was asking 🙂

> [@stangri](#):
>
> Would it be possible to include a section called Binaries, which would be the same list as Architectures but pointing to the actual IPKs?

Do I understand this correctly: When you click on aarch64\_cortex-a53, you want to directly download the curl ipk for this architecture?

 ![grafik](https://forum.openwrt.org/uploads/default/original/3X/f/3/f389b213ab5778905fd1f60952ce08fc70ad8cf6.png)

---

<div class="post-metadata">

**Author:** ![stangri](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stangri/32/58_2.png) [@stangri](https://forum.openwrt.org/u/stangri)\
**Post date:** [November 29, 2021, 9:27pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/5 "2021-11-29T21:27:39Z")

</div>

> [@tmomas](#):
>
> Do I understand this correctly: When you click on aarch64\_cortex-a53, you want to directly download the curl ipk for this architecture?

I thought if you could add Binaries section and in that section when I click on aarch64\_cortex-a53 I want to download the curl ipk for this architecture.

If Architectures sections serves no purpose currently then maybe it can be renamed/changed.

---

<div class="post-metadata">

**Author:** ![tmomas](https://forum.openwrt.org/letter_avatar/tmomas/32/5_5575768a8748004e209b776fc1b2916d.png) [@tmomas](https://forum.openwrt.org/u/tmomas)\
**Post date:** [November 29, 2021, 11:45pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/6 "2021-11-29T23:45:36Z")

</div>

In principle this should be doable, if @bobafetthotmail has the links to the packages available.

Questions which come to my mind:

- Are there really that many people manually downloading packages (instead of just installing them via opkg)? Just asking, since I have no idea how often this is done.
- Should we really invest time in the current package table (which is painfully slow), or rather think about a more performant solution?

---

<div class="post-metadata">

**Author:** ![bobafetthotmail](https://forum.openwrt.org/user_avatar/forum.openwrt.org/bobafetthotmail/32/39_2.png) [@bobafetthotmail](https://forum.openwrt.org/u/bobafetthotmail)\
**Post date:** [November 30, 2021, 9:48am UTC](https://forum.openwrt.org/t/package-pages-question/113300/7 "2021-11-30T09:48:05Z")

</div>

> [@stangri](#):
>
> Would it be possible to include a section called Binaries, which would be the same list as Architectures but pointing to the actual IPKs?

it is possible, yes, it would require some modification to the script that indexes the packages.

> [@](#):
>
> I've been talking to curl people (their download page links for OpenWrt are hilariously outdated) about having the OpenWrt links up-to-date and they use HTML scrubber so having a new section called Binaries would be the easiest way for them.

why curl people need direct access to OpenWrt package binaries? Can they just tell people to **opkg install curl** on OpenWrt?

and if they are scraping HTML why not scraping the download server landing page [https://downloads.openwrt.org/](https://downloads.openwrt.org/) to find whatever is latest version and the list of targets, then you can assemble the download link for the package (again in the download server that you can also reach by just clicking around if you are not a scraper bot limited to text)

I mean, I'm already scraping from that site and from manifest files anyway, it makes little sense to scrape to create a html that is scraped again by someone else that could just scrape the same source instead.

Also people that want to download packages manually should go there.

EDIT: My intent here is not "brushing off" anyone, it's just that the OpenWrt wiki is not meant for machine interaction, if you need interfaces for machine interaction there are better ways (that I also use to create the package lists) so if that is required just ask for that directly and we can explain how or if it's missing we can create it. For example in the past I sent a PR to the OpenWrt build system to keep a "verbose" version of the package manifest text file in the download server that is used only for my script to read/parse information. for example this [https://downloads.openwrt.org/releases/21.02.1/targets/x86/64/packages/Packages.manifest](https://downloads.openwrt.org/releases/21.02.1/targets/x86/64/packages/Packages.manifest)

Also I think **aparcar** has a more modern system based on json and python or something, which is probably better suited for more modern tools. [https://github.com/aparcar/openwrt-json](https://github.com/aparcar/openwrt-json)

---

<div class="post-metadata">

**Author:** ![stangri](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stangri/32/58_2.png) [@stangri](https://forum.openwrt.org/u/stangri)\
**Post date:** [November 30, 2021, 3:57pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/8 "2021-11-30T15:57:16Z")

</div>

> [@bobafetthotmail](#):
>
> why curl people need direct access to OpenWrt package binaries? Can they just tell people to **opkg install curl** on OpenWrt?

Because that's how their download page is designed: [https://curl.se/download.html](https://curl.se/download.html). We can have it listed outdated versions as things are now, not listed there at all or try to provide a way for them to scrape/list supported architectures/packages. I offered shell script to manipulate their DB directly, they prefer to scrape some of our pages instead.

> [@bobafetthotmail](#):
>
> and if they are scraping HTML why not scraping the download server landing page [https://downloads.openwrt.org/](https://downloads.openwrt.org/) to find whatever is latest version and the list of targets, then you can assemble the download link for the package (again in the download server that you can also reach by just clicking around if you are not a scraper bot limited to text)

Because my understanding is that that's not how their scraper works.

If it's too much trouble to add a Binaries section, they can scrape the package page, the problem is that it lists release OpenWrt/package version however the links lead to the snapshots packages which can be a different version.

---

<div class="post-metadata">

**Author:** ![bobafetthotmail](https://forum.openwrt.org/user_avatar/forum.openwrt.org/bobafetthotmail/32/39_2.png) [@bobafetthotmail](https://forum.openwrt.org/u/bobafetthotmail)\
**Post date:** [December 1, 2021, 2:16pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/9 "2021-12-01T14:16:28Z")

</div>

> [@stangri](#):
>
> Because my understanding is that that's not how their scraper works.

I find it hard to believe that people making Curl of all things can't write a couple one-liner shell commands that wget/curl the page and grep/awk the raw HTML text in a shell script.

> [@](#):
>
> If it's too much trouble to add a Binaries section,

I can create a page with the information they need, even if I really think they should be able to do this themselves.

Not in [https://openwrt.org/packages/pkgdata/curl](https://openwrt.org/packages/pkgdata/curl) but in a new article. As said also by Tmomas, that page is part of package table infrastructure and unless we want to add package download links to every single package (and we really don't want to, as it will degrade performance of the package table significantly), that page will be left as-is.

This new page will be updated automatically by a dedicated script that runs on its own (still on the wiki server) and will do this only for Curl package, at least for now.

So please tell me what you want exactly on the page or even just create a stub of it in the wiki and I'll hook it up to a script this weekend.

---

<div class="post-metadata">

**Author:** ![stangri](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stangri/32/58_2.png) [@stangri](https://forum.openwrt.org/u/stangri)\
**Post date:** [December 2, 2021, 12:46am UTC](https://forum.openwrt.org/t/package-pages-question/113300/10 "2021-12-02T00:46:11Z")

</div>

> [@bobafetthotmail](#):
>
> I find it hard to believe that people making Curl of all things can't write a couple one-liner shell commands that wget/curl the page and grep/awk the raw HTML text in a shell script.

I don't think it's that they can't, it's just that they probably wouldn't for a single distro that they were comfortable to list very outdated links for a while for.

> [@bobafetthotmail](#):
>
> I can create a page with the information they need, even if I really think they should be able to do this themselves.
> 
> Not in [https://openwrt.org/packages/pkgdata/curl](https://openwrt.org/packages/pkgdata/curl) but in a new article.

I see. If it's too computationally expensive to append the existing page, I think we'll have to live with the fact that sometimes the version numbers would be out of sync.

---

<div class="post-metadata">

**Author:** ![bobafetthotmail](https://forum.openwrt.org/user_avatar/forum.openwrt.org/bobafetthotmail/32/39_2.png) [@bobafetthotmail](https://forum.openwrt.org/u/bobafetthotmail)\
**Post date:** [December 2, 2021, 11:28am UTC](https://forum.openwrt.org/t/package-pages-question/113300/11 "2021-12-02T11:28:41Z")

</div>

> [@stangri](#):
>
> I don't think it's that they can't, it's just that they probably wouldn't for a single distro that they were comfortable to list very outdated links for a while for.

I was just pointing out that I'm fully aware that what you wrote is not the real reason

> [@](#):
>
> I see. If it's too computationally expensive to append the existing page

No, that's not the problem.  
The existing page is part of a larger system written by a script and backed by a database used in a somewhat debatable way by a dokuwiki plugin to create and show the package table. [https://openwrt.org/packages/table/start](https://openwrt.org/packages/table/start)

I can create a new wiki page with all data you want on it. It is not "computationally intensive".

I'm not going to add stuff to existing pages used by another system that is slow and bloated already unless it is really necessary, and imho it's not the case for this request.

---

<div class="post-metadata">

**Author:** ![stangri](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stangri/32/58_2.png) [@stangri](https://forum.openwrt.org/u/stangri)\
**Post date:** [December 3, 2021, 7:57pm UTC](https://forum.openwrt.org/t/package-pages-question/113300/12 "2021-12-03T19:57:50Z")

</div>

> [@bobafetthotmail](#):
>
> I was just pointing out that I'm fully aware that what you wrote is not the real reason

I don't like the implications of that statement, so I'm going to conclude my participation in this thread. Everything I've had to say on the subject has been stated in the OP.

---

<div class="post-metadata">

**Author:** ![bobafetthotmail](https://forum.openwrt.org/user_avatar/forum.openwrt.org/bobafetthotmail/32/39_2.png) [@bobafetthotmail](https://forum.openwrt.org/u/bobafetthotmail)\
**Post date:** [December 5, 2021, 11:28am UTC](https://forum.openwrt.org/t/package-pages-question/113300/13 "2021-12-05T11:28:56Z")

</div>

> [@stangri](#):
>
> I don't like the implications of that statement,

I don't like wasting my free time for for people that don't care. Curl people have clearly shown they don't care enough to do their part.

---

<div class="post-metadata">

**Author:** ![system](https://forum.openwrt.org/uploads/default/original/3X/7/4/740520c71f00039bbf001045c682f0ecb25e905a.png) [@system](https://forum.openwrt.org/u/system)\
**Post date:** [December 15, 2021, 11:29am UTC](https://forum.openwrt.org/t/package-pages-question/113300/14 "2021-12-15T11:29:39Z")

</div>

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.
