Difficulties with installing SciPy on OpenWrt

Installing and Using OpenWrt
1

I'm attempting to run a Python application off my OpenWRT device (most recent version for a Pi 3B) and one of the required modules is SciPy. Naturally, I try installing it via pip3 install scipy. I get the following:


root@OpenWrt:~# pip3 install scipy
Collecting scipy
  Downloading scipy-1.6.0.tar.gz (27.3 MB)
     |████████████████████████████████| 27.3 MB 5.1 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error

WARNING: Discarding https://files.pythonhosted.org/packages/16/48/ff7026d26dfd92520f00b109333e22c05a235f0c9115a5a2d7679cdf39ef/scipy-1.6.0.tar.gz#sha256=cb6dc9f82dfd95f6b9032a8d7ea70efeeb15d5b5fd6ed4e8537bb3c673580566 (from https://pypi.org/simple/scipy/) (requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp0zgfl0_3 Check the logs for full command output.

  Downloading scipy-1.5.4.tar.gz (25.2 MB)
     |████████████████████████████████| 25.2 MB 5.4 MB/s
  Installing build dependencies ... error

WARNING: Discarding https://files.pythonhosted.org/packages/aa/d5/dd06fe0e274e579e1dff21aa021219c039df40e39709fabe559faed072a5/scipy-1.5.4.tar.gz#sha256=4a453d5e5689de62e5d38edf40af3f17560bfd63c9c5bd228c18c1f99afa155b (from https://pypi.org/simple/scipy/) (requires-python:>=3.6). Command errored out with exit status 1: /usr/bin/python3 /usr/lib/python3.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-jv_894mx/overlay --no-warn-script-location -v --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- wheel setuptools 'Cython>=0.29.18' 'numpy==1.14.5; python_version=='"'"'3.6'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.14.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system!='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.6'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.16.0; python_version=='"'"'3.7'"'"' and platform_system=='"'"'AIX'"'"'' 'numpy==1.17.3; python_version>='"'"'3.8'"'"' and platform_system=='"'"'AIX'"'"'' 'pybind11>=2.4.3' Check the logs for full command output.

  Downloading scipy-1.5.3.tar.gz (25.2 MB)
     |████████████████████████████████| 25.2 MB 4.5 MB/s
ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

Note that there very much is space left on the device:

root@OpenWrt:~# df -k

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root             61360156  11560792  49782980  19% /
tmpfs                   472640     98872    373768  21% /tmp
/dev/mmcblk0p1           20422     14884      5538  73% /boot
tmpf                      512         0       512   0% /dev

For other packages I've either been able to install via pip3 or had to install from source via Github. Installing from source for SciPy has equally failed:

root@OpenWrt:~/scipy# python3 setup.py install

Note: for reliable uninstall behaviour and dependency installation
and uninstallation, please use pip instead of using
`setup.py install`:
  - `pip install .`       (from a git repo or downloaded source release)
  - `pip install scipy`   (last SciPy release on PyPI)
 
Cythonizing sources
Running scipy/linalg/_generate_pyx.py
Running scipy/special/_generate_pyx.py
scipy/linalg/_generate_pyx.py: all files up-to-date
scipy/special/_generate_pyx.py: all files up-to-date
scipy/_lib/_ccallback_c.pyx has not changed
scipy/_lib/_test_deprecation_call.pyx has not changed
scipy/_lib/_test_deprecation_def.pyx has not changed
scipy/_lib/messagestream.pyx has not changed
scipy/fftpack/convolve.pyx has not changed
scipy/cluster/_optimal_leaf_ordering.pyx has not changed
scipy/cluster/_vq.pyx has not changed
scipy/interpolate/interpnd.pyx has not changed
scipy/interpolate/_bspl.pyx has not changed
scipy/interpolate/_ppoly.pyx has not changed
scipy/cluster/_hierarchy.pyx has not changed
scipy/io/matlab/mio5_utils.pyx has not changed
scipy/io/matlab/mio_utils.pyx has not changed
scipy/io/matlab/streams.pyx has not changed
scipy/linalg/_decomp_update.pyx.in has not changed
scipy/linalg/_matfuncs_sqrtm_triu.pyx has not changed
scipy/linalg/_solve_toeplitz.pyx has not changed
scipy/linalg/cython_blas.pyx has not changed
scipy/linalg/cython_lapack.pyx has not changed
scipy/ndimage/src/_cytest.pyx has not changed
scipy/ndimage/src/_ni_label.pyx has not changed
scipy/optimize/_bglu_dense.pyx has not changed
scipy/optimize/_group_columns.pyx has not changed
scipy/optimize/_highs/cython/src/constants.pyx has not changed
scipy/optimize/_highs/cython/src/highs_wrapper.pyx has not changed
scipy/optimize/_highs/cython/src/mpswriter.pyx has not changed
scipy/optimize/_lsq/givens_elimination.pyx has not changed
scipy/optimize/_trlib/_trlib.pyx has not changed
scipy/optimize/cython_optimize/_zeros.pyx.in has not changed
scipy/signal/_max_len_seq_inner.pyx has not changed
scipy/signal/_peak_finding_utils.pyx has not changed
scipy/signal/_sosfilt.pyx has not changed
scipy/signal/_spectral.pyx has not changed
scipy/signal/_upfirdn_apply.pyx has not changed
scipy/sparse/_csparsetools.pyx.in has not changed
scipy/sparse/csgraph/_flow.pyx has not changed
scipy/sparse/csgraph/_matching.pyx has not changed
scipy/sparse/csgraph/_min_spanning_tree.pyx has not changed
scipy/sparse/csgraph/_reordering.pyx has not changed
scipy/sparse/csgraph/_shortest_path.pyx has not changed
scipy/sparse/csgraph/_tools.pyx has not changed
scipy/sparse/csgraph/_traversal.pyx has not changed
scipy/spatial/_hausdorff.pyx has not changed
scipy/spatial/_voronoi.pyx has not changed
scipy/spatial/ckdtree.pyx has not changed
scipy/spatial/qhull.pyx has not changed
scipy/spatial/transform/rotation.pyx has not changed
scipy/special/_comb.pyx has not changed
scipy/special/_ellip_harm_2.pyx has not changed
scipy/special/_test_round.pyx has not changed
scipy/special/_ufuncs.pyx has not changed
scipy/special/_ufuncs_cxx.pyx has not changed
scipy/special/cython_special.pyx has not changed
scipy/stats/_sobol.pyx has not changed
scipy/stats/_stats.pyx has not changed
Running from SciPy source directory.
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
  libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

openblas_lapack_info:
  libraries openblas not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

openblas_clapack_info:
  libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

flame_info:
  libraries flame not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

atlas_3_10_threads_info:
Setting PTATLAS=ATLAS
  libraries lapack_atlas not found in /usr/local/lib
  libraries tatlas,tatlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/lib64
  libraries tatlas,tatlas not found in /usr/lib64
  libraries lapack_atlas not found in /usr/lib
  libraries tatlas,tatlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
  NOT AVAILABLE

atlas_3_10_info:
  libraries lapack_atlas not found in /usr/local/lib
  libraries satlas,satlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/lib64
  libraries satlas,satlas not found in /usr/lib64
  libraries lapack_atlas not found in /usr/lib
  libraries satlas,satlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_3_10_info'>
  NOT AVAILABLE

atlas_threads_info:
Setting PTATLAS=ATLAS
  libraries lapack_atlas not found in /usr/local/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/lib64
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
  libraries lapack_atlas not found in /usr/lib
  libraries ptf77blas,ptcblas,atlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_threads_info'>
  NOT AVAILABLE

atlas_info:
  libraries lapack_atlas not found in /usr/local/lib
  libraries f77blas,cblas,atlas not found in /usr/local/lib
  libraries lapack_atlas not found in /usr/lib64
  libraries f77blas,cblas,atlas not found in /usr/lib64
  libraries lapack_atlas not found in /usr/lib
  libraries f77blas,cblas,atlas not found in /usr/lib
<class 'numpy.distutils.system_info.atlas_info'>
  NOT AVAILABLE

lapack_info:
  libraries lapack not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

/usr/lib/python3.7/site-packages/numpy/distutils/system_info.py:1849: UserWarning:
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  return getattr(self, '_calc_info_{}'.format(name))()
lapack_src_info:
  NOT AVAILABLE

/usr/lib/python3.7/site-packages/numpy/distutils/system_info.py:1849: UserWarning:
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  return getattr(self, '_calc_info_{}'.format(name))()
  NOT AVAILABLE

Traceback (most recent call last):
  File "setup.py", line 605, in <module>
    setup_package()
  File "setup.py", line 601, in setup_package
    setup(**metadata)
  File "/usr/lib/python3.7/site-packages/numpy/distutils/core.py", line 135, in setup
    config = configuration()
  File "setup.py", line 516, in configuration
    raise NotFoundError(msg)

numpy.distutils.system_info.NotFoundError: No BLAS/LAPACK libraries found.
To build Scipy from sources, BLAS & LAPACK libraries need to be installed.
See site.cfg.example in the Scipy source directory and
https://docs.scipy.org/doc/scipy/reference/building/index.html for details.

Given that neither solution seems to work, what should I be doing here? Why isn't it installing via pip3? What are BLAS and LAPACK libraries? Is it even possible?

1 Like
2

follow these directions

also take note of the other error output, maybe post the error log mentioned

(requires-python:>=3.7). Command errored out with exit status 1: /usr/bin/python3 
/usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py 
prepare_metadata_for_build_wheel /tmp/tmp0zgfl0_3 Check the logs for full command output.
3

RE the first part, because OpenWRT doesn't appear to have BLAS and LAPACK libraries, I have to install those from source too.

EDIT: OpenBLAS installs from source successfully, though I'm not sure it's finding it.

Whereas for LAPACK, make gives:

 make -C INSTALL run

make[1]: Entering directory '/home/user/lapack/INSTALL'

gfortran -O2 -frecursive  -c -o lsame.o lsame.f

make[1]: gfortran: Command not found

make[1]: *** [<builtin>: lsame.o] Error 127

make[1]: Leaving directory '/home/user/lapack/INSTALL'

make: *** [Makefile:57: lapack_install] Error 2

This is particularly a problem, given that gfortran is neither an OpenWRT package nor available from source. It's available as a Debian package, but for OpenWRT that's not an option and its source code states that it is part of gcc, which is an OpenWRT package and one I have installed.

RE the Python logs, I cannot find where the hell they're being kept, if at all. I'd post them if I knew. It's installed via opkg but I haven't found anything that helps me locate where the logs are.

1 Like
4

https://forum.openwrt.org/search?q=openwrt%20gcc%20python%20error

#> gcc -v
gcc version 7.4.0 (OpenWrt GCC 7.4.0)
#> cat /tmp/pip-log.txt  | wc -l
110638

don't know how to compile Fortran code on platform 'posix'
5

That's the versions I've got, though cat /tmp/pip-log.txt | wc -l gave me 59939 if that matters. Guess I gotta install GCC from source too.

EDIT: Or rather, I've tried installing GCC from source, but it fails on the ./configure stage with the following result:



root@OpenWrt:/home/user/gcc-10.2.0# ./configure

checking build system type... aarch64-unknown-linux-musl

checking host system type... aarch64-unknown-linux-musl

checking target system type... aarch64-unknown-linux-musl

checking for a BSD-compatible install... ./install-sh -c

checking whether ln works... yes

checking whether ln -s works... yes

checking for a sed that does not truncate output... /bin/sed

checking for gawk... no

checking for mawk... no

checking for nawk... no

checking for awk... awk

checking for libatomic support... yes

checking for libitm support... yes

checking for libsanitizer support... yes

checking for libvtv support... yes

checking for libhsail-rt support... no

checking for libphobos support... yes

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking for g++... g++

checking whether we are using the GNU C++ compiler... yes

checking whether g++ accepts -g... yes

checking whether g++ accepts -static-libstdc++ -static-libgcc... no

checking for gnatbind... no

checking for gnatmake... no

checking whether compiler driver understands Ada... no

checking how to compare bootstrapped objects... tail -c +17 $$f1 > tmp-foo1; tail -c +17 $$f2 > tmp-foo2; cmp tmp-foo1 tmp-foo2

checking for objdir... .libs

checking for the correct version of gmp.h... no

configure: error: Building GCC requires GMP 4.2+, MPFR 3.1.0+ and MPC 0.8.0+.

Try the --with-gmp, --with-mpfr and/or --with-mpc options to specify

their locations.  Source code for these libraries can be found at

their respective hosting sites as well as at

https://gcc.gnu.org/pub/gcc/infrastructure/.  See also

http://gcc.gnu.org/install/prerequisites.html for additional info.  If

you obtained GMP, MPFR and/or MPC from a vendor distribution package,

make sure that you have installed both the libraries and the header

files.  They may be located in separate packages.

I note that GMP is a opkg package that I have installed, whereas MPFR and MPC are not; however, simply being installed on the system by the package manager does not prevent GCC from being unable to find it.

1 Like