Python, numpy and or other libraries not installing

I am trying to add python functionality to my OpenWRT box, but installing python packages seems to fail all the time. And I cannot figure it out why. I have tried installing everything under the sun that is even remotely related to getting numpy in my box, but it throws just a bunch of errors:

root@OpenWrt:~# pip3 install numpy --no-binary :all:
Collecting numpy
  Using cached numpy-1.21.0.zip (10.3 MB)
  Installing build dependencies ... error
WARNING: Discarding https://files.pythonhosted.org/packages/66/03/818876390c7ff4484d5a05398a618cfdaf0a2b9abb3a7c7ccd59fe181008/numpy-1.21.0.zip#sha256=e80fe25cba41c124d04c662f33f6364909b985f2eb5998aaa5ae4b9587242cce (from https://pypi.org/simple/numpy/) (requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 /tmp/pip-standalone-pip-6h1qf1lv/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-i82anqdb/overlay --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'packaging==20.5; platform_machine=='"'"'arm64'"'"'' 'setuptools<49.2.0' wheel==0.36.2 'Cython>=0.29.21,<3.0' Check the logs for full command output.
  Using cached numpy-1.20.3.zip (7.8 MB)
  Installing build dependencies ... error
WARNING: Discarding https://files.pythonhosted.org/packages/f3/1f/fe9459e39335e7d0e372b5e5dcd60f4381d3d1b42f0b9c8222102ff29ded/numpy-1.20.3.zip#sha256=e55185e51b18d788e49fe8305fd73ef4470596b33fc2c1ceb304566b99c71a69 (from https://pypi.org/simple/numpy/) (requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 /tmp/pip-standalone-pip-i_1ms2rf/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-ducmhmws/overlay --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools<49.2.0' 'wheel<=0.35.1' 'Cython>=0.29.21,<3.0' Check the logs for full command output.
  Using cached numpy-1.20.2.zip (7.8 MB)
  Installing build dependencies ... error
WARNING: Discarding https://files.pythonhosted.org/packages/82/a8/1e0f86ae3f13f7ce260e9f782764c16559917f24382c74edfb52149897de/numpy-1.20.2.zip#sha256=878922bf5ad7550aa044aa9301d417e2d3ae50f0f577de92051d739ac6096cee (from https://pypi.org/simple/numpy/) (requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 /tmp/pip-standalone-pip-y3hxj5ln/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-228sn8g6/overlay --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools<49.2.0' 'wheel<=0.35.1' 'Cython>=0.29.21,<3.0' Check the logs for full command output.
  Downloading numpy-1.20.1.zip (7.8 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 7.8 MB 8.7 MB/s
  Installing build dependencies ... error
WARNING: Discarding https://files.pythonhosted.org/packages/d2/48/f445be426ccd9b2fb64155ac6730c7212358882e589cd3717477d739d9ff/numpy-1.20.1.zip#sha256=3bc63486a870294683980d76ec1e3efc786295ae00128f9ea38e2c6e74d5a60a (from https://pypi.org/simple/numpy/) (requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 /tmp/pip-standalone-pip-wf7v7j36/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-9q2qcexu/overlay --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools<49.2.0' 'wheel<=0.35.1' 'Cython>=0.29.21,<3.0' Check the logs for full command output.
  Downloading numpy-1.20.0.zip (8.0 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 8.0 MB 5.0 MB/s
  Installing build dependencies ... error
WARNING: Discarding https://files.pythonhosted.org/packages/c3/97/fd507e48f8c7cab73a9f002e52e15983b5636b4ac6cf69b83ae240324b44/numpy-1.20.0.zip#sha256=3d8233c03f116d068d5365fed4477f2947c7229582dad81e5953088989294cec (from https://pypi.org/simple/numpy/) (requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 /tmp/pip-standalone-pip-iklo2821/__env_pip__.zip/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-134vwo05/overlay --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools<49.2.0' 'wheel<=0.35.1' 'Cython>=0.29.21,<3.0' Check the logs for full command output.
  Downloading numpy-1.19.5.zip (7.3 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 7.3 MB 9.1 MB/s
  Installing build dependencies ... error
...

I have pip (21.1.2), setuptools (57.0.0), wheel (0.36.2), cython (0.29.23), which should satisfy the requirements (?)

I also run with --no-binary :all:, because without it, the installation crashes faster when trying to build wheel.

Is this a doomed project to begin with?

Looks like there are no available packages for your architecture in the repo.
I suggest you running Alpine Linux container in Docker and installing Python + numpy inside it

I have no idea about that, but is there no way of getting a simple python installment on the device? Is there a way of creating numpy somehow for the OpenWRT? How about opencv? I see I can install it with opkg, but for python it isn't working.

You can use LXC container also…
With Debian or any other linux system !
Look for LXC howto at https://forum.openwrt.org/t/openwrt-arm64-quick-lxc-howto-guide-lms-in-debian-system-in-lxc-container

What is the platform (cpu) ?
Have you installed gcc and other necessary build tools ?

Platform is x86_64 PCEngines APU 2, I have installed gcc, yes. Now I'm looking into having a virtual machine debian to build stuff for it. I'll look into the containerlink you gave me.

Strange that it do not work on OpenWrt directly…
Strange also that you get no logs nor errors details…

As it's been mentioned above, your problem is that there are no packages available for your platform.
It has to be compiled for your platform which requires gcc, python-dev and several dependencies that your package needs.

I've been on this path before and IMO running a lightweight Linux distro like Alpine Linux in Docker is the easiest way.

I cannot find a docker for openwrt, and in previous link for LXC container tutorial I get error

root@OpenWrt:~# opkg install cgroupfs-mount cgroup-tools
Unknown package 'cgroupfs-mount'.
Package cgroup-tools (0.41-1) installed in root is up to date.
Collected errors:
 * opkg_install_cmd: Cannot install package cgroupfs-mount.

Is there a need to compile openwrt somehow differently than default to get everything working?

Do you use a 21.02-rc ?

using pip on OpenWrt usually blows up like that when it's trying to download libraries that are not pure python.

Check if someone has an openwrt package for that that was compiled with OpenWrt build system.

For example, numpy OpenWrt package exists, https://github.com/openwrt/packages/blob/master/lang/python/numpy/Makefile so install that.

1 Like

LXC works fine (and out of the box) in 21.02-RC...
DOCKER do not :

root@ultra:~# /usr/share/docker/check-config.sh 
info: reading kernel config from /proc/config.gz ...

Generally Necessary:
- cgroup hierarchy: properly mounted [/sys/fs/cgroup]
- CONFIG_NAMESPACES: enabled
- CONFIG_NET_NS: enabled
- CONFIG_PID_NS: enabled
- CONFIG_IPC_NS: enabled
- CONFIG_UTS_NS: enabled
- CONFIG_CGROUPS: enabled
- CONFIG_CGROUP_CPUACCT: enabled
- CONFIG_CGROUP_DEVICE: missing
- CONFIG_CGROUP_FREEZER: missing
- CONFIG_CGROUP_SCHED: enabled
- CONFIG_CPUSETS: enabled
- CONFIG_MEMCG: enabled
- CONFIG_KEYS: enabled
- CONFIG_VETH: enabled (as module)
- CONFIG_BRIDGE: enabled
- CONFIG_BRIDGE_NETFILTER: enabled (as module)
- CONFIG_IP_NF_FILTER: enabled (as module)
- CONFIG_IP_NF_TARGET_MASQUERADE: missing
- CONFIG_NETFILTER_XT_MATCH_ADDRTYPE: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_CONNTRACK: enabled (as module)
- CONFIG_NETFILTER_XT_MATCH_IPVS: enabled (as module)
- CONFIG_NETFILTER_XT_MARK: enabled (as module)
- CONFIG_IP_NF_NAT: enabled (as module)
- CONFIG_NF_NAT: enabled (as module)
- CONFIG_POSIX_MQUEUE: enabled

Optional Features:
- CONFIG_USER_NS: enabled
- CONFIG_SECCOMP: enabled
- CONFIG_CGROUP_PIDS: enabled
- CONFIG_MEMCG_SWAP: enabled
- CONFIG_MEMCG_SWAP_ENABLED: missing
    (cgroup swap accounting is currently not enabled, you can enable it by setting boot option "swapaccount=1")
- CONFIG_BLK_CGROUP: enabled
- CONFIG_BLK_DEV_THROTTLING: enabled
- CONFIG_CGROUP_PERF: missing
- CONFIG_CGROUP_HUGETLB: missing
- CONFIG_NET_CLS_CGROUP: missing
- CONFIG_CGROUP_NET_PRIO: missing
- CONFIG_CFS_BANDWIDTH: enabled
- CONFIG_FAIR_GROUP_SCHED: enabled
- CONFIG_RT_GROUP_SCHED: enabled
- CONFIG_IP_NF_TARGET_REDIRECT: missing
- CONFIG_IP_VS: enabled (as module)
- CONFIG_IP_VS_NFCT: enabled
- CONFIG_IP_VS_PROTO_TCP: enabled
- CONFIG_IP_VS_PROTO_UDP: enabled
- CONFIG_IP_VS_RR: enabled (as module)
- CONFIG_EXT4_FS: enabled
- CONFIG_EXT4_FS_POSIX_ACL: missing
- CONFIG_EXT4_FS_SECURITY: missing
    enable these ext4 configs if you are using ext3 or ext4 as backing filesystem
- Network Drivers:
  - "overlay":
    - CONFIG_VXLAN: enabled (as module)
    - CONFIG_BRIDGE_VLAN_FILTERING: enabled
      Optional (for encrypted networks):
      - CONFIG_CRYPTO: enabled
      - CONFIG_CRYPTO_AEAD: enabled
      - CONFIG_CRYPTO_GCM: enabled (as module)
      - CONFIG_CRYPTO_SEQIV: enabled (as module)
      - CONFIG_CRYPTO_GHASH: enabled (as module)
      - CONFIG_XFRM: enabled
      - CONFIG_XFRM_USER: enabled (as module)
      - CONFIG_XFRM_ALGO: enabled (as module)
      - CONFIG_INET_ESP: enabled (as module)
  - "ipvlan":
    - CONFIG_IPVLAN: missing
  - "macvlan":
    - CONFIG_MACVLAN: enabled (as module)
    - CONFIG_DUMMY: enabled (as module)
  - "ftp,tftp client in container":
    - CONFIG_NF_NAT_FTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_FTP: enabled (as module)
    - CONFIG_NF_NAT_TFTP: enabled (as module)
    - CONFIG_NF_CONNTRACK_TFTP: enabled (as module)
- Storage Drivers:
  - "aufs":
    - CONFIG_AUFS_FS: missing
  - "btrfs":
    - CONFIG_BTRFS_FS: enabled (as module)
    - CONFIG_BTRFS_FS_POSIX_ACL: missing
  - "devicemapper":
    - CONFIG_BLK_DEV_DM: enabled (as module)
    - CONFIG_DM_THIN_PROVISIONING: missing
  - "overlay":
    - CONFIG_OVERLAY_FS: enabled
  - "zfs":
    - /dev/zfs: missing
    - zfs command: missing
    - zpool command: missing

Limits:
- /proc/sys/kernel/keys/root_maxkeys: 1000000
Sat Jun 26 11:14:54 2021 daemon.err dockerd[17263]: failed to start daemon: Devices cgroup isn't mounted
Sat Jun 26 11:15:42 2021 daemon.err modprobe: failed to find a module named aufs
Sat Jun 26 11:15:42 2021 daemon.err dockerd[17429]: time="2021-06-26T09:15:42.258204662Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
Sat Jun 26 11:15:42 2021 daemon.err dockerd[17429]: time="2021-06-26T09:15:42.260466216Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
Sat Jun 26 11:15:42 2021 daemon.err dockerd[17429]: time="2021-06-26T09:15:42.371194445Z" level=warning msg="Your kernel does not support swap memory limit"
Sat Jun 26 11:15:42 2021 daemon.err dockerd[17429]: time="2021-06-26T09:15:42.371525970Z" level=warning msg="Your kernel does not support cgroup blkio weight"
Sat Jun 26 11:15:42 2021 daemon.err dockerd[17429]: time="2021-06-26T09:15:42.371593651Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Sat Jun 26 11:15:43 2021 daemon.err dockerd[17429]: failed to start daemon: Devices cgroup isn't mounted

only CGROUPFS v2 are supported in OpenWrt !

Please read the topic well

Well, I have currently been able to run 19.07.7, and I am having trouble compiling the 21 version (actually, I have never been able to compile any linux distribution, there is always some error somewhere, even if done according to guides and tutorials).

I will try the LXC method, after I get a working system up.

why compiling a snapshot !?
It is available from OpenWrt...
look at wiki or with firmware select.
https://firmware-selector.openwrt.org/

for python alone... lxc/docker is likely overkill and deboostrap/chroot is more than enough... (and much simpler)

Because I cannot find a version for my device. I am also very noob when it comes linux stuff, let alone embedded stuff, so I have 99% no idea what I'm doing, just trying my best to follow instructions.

may be this one will works :
https://downloads.openwrt.org/releases/21.02.0-rc3/targets/x86/64/

I cannot use any of those, as I cannot flash the memory card. I need a sysupgrade file, and the latest for PC Engines APU series seems to be 19.07.7