Using the SDK to include Python libraries in the firmware


After far too much difficulty in trying to install Scipy directly to OpenWRT (Difficulties with installing SciPy on OpenWRT), I found that apparently what I could be doing instead is using the SDK or the Image Builder to include it in the firmware.

Specifically, what I want to do is include my Python program with all associated dependencies. And the dependencies for those dependencies. I know that the hardware can cope - it's a Raspberry Pi 3B - but since the firmware doesn't want to co-operate it seems I gotta write my own.

I've never used the SDK or Image Builder before. Will either of them do what I want them to do, and if so how would I do so?

if using the build tree, this is part of
make menuconfig

if using the imagebuilder, you include package names in the command

guides for both:

I set up the Image Builder via the first guide and the make menuconfig stage has given me the config menu, as below:

but I can't seem to find where I'd make it add the Python modules I want it to - e.g. a search for "pip" gives only results for pipes or ipip. Can the GUI be used to include custom packages/libraries/etc or is that something I'll need to do using the second guide?

this has everything you need to know

pick the right package carefully, as there is massive size differences

I've included the full-fat Python module and managed to write some firmware.

But what I'm trying to include is a Python library i.e. a package installed by the pip command as opposed to opkg. It's not on the opkg repository, so it's not covered by the ./scripts/feeds command given by that Python link you provided.

So what should I be using instead?

Is pip not available after installing the main python package?

at least by using python to invoke pip

python -m pip .....

pip3 is an available command.

However, the Python library (SciPy) I'm trying to install using pip fails to install (as per the previous question I asked, linked in the OP), so I'm instead trying to include it in the firmware to get around that problem entirely. That's what I've been trying to do the entire time.

Is that something that can be done?

EDIT: I should also mention that I tried including Node as well as Python in the build, and that caused the build to fail due to a lack of space - given that I know that I've sufficient memory space on the card, is that an adjustable parameter?

need to see the output...

also try

python3 -m pip

If you mean the output of building it with Node, first I get this:

make -r world: build failed. Please re-run make with -j1 V=s or V=sc for a higher verbosity level to see what's going on
make: *** [/home/ian/Documents/openwrt/include/ world] Error 1

and then with those parameters, at the end I get:

Creating filesystem with parameters:
    Size: 109051904
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 6656
    Inode size: 256
    Journal blocks: 0
    Label: rootfs
    Blocks: 26624
    Block groups: 1
    Reserved blocks: 0
    Reserved block group size: 7
error: make_directory: failed to allocate inode

make[5]: *** [/home/ian/Documents/openwrt/include/ /home/ian/Documents/openwrt/build_dir/target-aarch64_cortex-a53_musl/linux-bcm27xx_bcm2710/root.ext4] Error 1
make[5]: Leaving directory '/home/ian/Documents/openwrt/target/linux/bcm27xx/image'
make[4]: *** [Makefile:31: install] Error 2
make[4]: Leaving directory '/home/ian/Documents/openwrt/target/linux/bcm27xx'
make[3]: *** [Makefile:11: install] Error 2
make[3]: Leaving directory '/home/ian/Documents/openwrt/target/linux'
time: target/linux/install#35.24#5.28#39.86
    ERROR: target/linux failed to build.
make[2]: *** [target/Makefile:25: target/linux/install] Error 1
make[2]: Leaving directory '/home/ian/Documents/openwrt'
make[1]: *** [target/Makefile:19: /home/ian/Documents/openwrt/staging_dir/target-aarch64_cortex-a53_musl/stamp/.target_install] Error 2
make[1]: Leaving directory '/home/ian/Documents/openwrt'
make: *** [/home/ian/Documents/openwrt/include/ world] Error 2

If you mean the output of pip3 install scipy or python3 -m pip install scipy, that's all given in here:

first try to update pip and similar tools

if that fails, you can download wheels and install them manually

I've checked, and whilst there are Scipy version compatible with Python 3.7, none of the wheels are supported by the OpenWRT platform, so it fails.

Installing it from source archives, either by using pip3 installs scipy-1.6.1.tar.gz or by using the application in the unpacked Scipy directory, has the effects as given in the link in the previous reply.

EDIT: One potential solution that occurs is to build Scipy the Python3 module as a .ipk package. Is doing so possible (I note that there are opkg packages such as python3-flask, so it must have been at some point) and if so, how?