Packaging Python script

[svet]

I have a service written in Python 3 that I'd like to turn into a package - to be deployed to devices via opkg. Can anyone point me to a guide for how to do this, or some example code?

I checked out some related discussion at Finding / compiling Python packages for OpenWrt and Compiling python packages recompiles python where the authors have done this, but those threads unfortunately don't share the earlier steps of the process.

By the way I have the build toolchain set up, and am able to build the stock firmware/packages. Just looking for a "best practice" example of how to package my own Python code.

Thank you!

[bobafetthotmail]

It depends, is that just a python script? Because if it is a script you just package it like if it was a shell script, see watchcat https://github.com/openwrt/packages/blob/master/utils/watchcat/Makefile
and then add the python libraries you need in the dependency list (you can see a dependency list in the other example below)

If you are packaging a more complex python application that was set up to be packaged/installed with pip you can use docker-compose as an example. https://github.com/openwrt/packages/blob/master/utils/docker-compose/Makefile

[svet]

Thanks @bobafetthotmail for the quick response!

The code I'm trying to port (https://github.com/ammpio/ammp-edge) is fairly involved - though in the process of porting to OpenWRT I also anticipate doing some refactoring and making some simplifications. So my thinking was to start with a simple "hello world" Python package and build up from there. But I take your point that packaging a simple script would look quite different to packaging a bigger application.

Looks like the docker-compose example is a good starting point in that regard. Am I right in thinking that the "magic" happens in the last two lines of that Makefile, i.e.

$(eval $(call Py3Package,docker-compose))
$(eval $(call BuildPackage,docker-compose))

as well as

PYPI_NAME:=docker-compose

at the top, which basically tells the build system to pull a package from PyPI and build it?

I guess if I adopt the mindset of "converting a Python package into an opkg package" then it mostly makes sense. And I can see how the Readme at https://github.com/openwrt/packages/blob/master/lang/python/README.md comes into play.

Anyway, that gives me enough to chew on for now... Thanks again for the pointers!

[svet]

Well that actually turned out to be a bit more of an adventure than I was bargaining for .

Some pointers for anyone else wanting to do this:

1. Being a bit of an OpenWRT noob, I fell into the pitfall of assuming that I can just slot a Makefile somewhere in the OpenWRT tree and build it. Going through the "build your own Hello World application" guide was pretty instrumental in clarifying the necessity of setting up a custom package feed, etc.
2. The Makefile for a Python package that's introduced via a custom feed looks subtly different to the Makefile of a package that is built within the regular OpenWRT feeds/tree. Some of this is covered here: https://github.com/openwrt/packages/tree/master/lang/python#using-python-in-externalother-package-feeds ; following that guide more or less led me to the right place. Unfortunately there are a fair few examples out there (e.g. this) that look wildly different.
3. The build system is somewhat magical (source thread ), and if you don't get the magic to work with you, it can fail you in obscure ways.

With respect to the last point, I found that if I didn't specify VARIANT:=python3 in the package definition in the Makefile, the result was invariably

make[3]: *** No targets specified and no makefile found.  Stop.

which isn't terribly helpful. I've also not been able to find any documentation on VARIANT, but that might be poor googling on my part.

Anyway, it seems to all be working reasonably sanely now. If anyone's interested - or keen to give me feedback - here are two example Makefiles. The first is for a package from PyPI:

# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk

PKG_NAME:=python3-paho-mqtt
PKG_VERSION:=1.5.1
PKG_RELEASE:=1

PYPI_NAME:=paho-mqtt
PKG_HASH:=9feb068e822be7b3a116324e01fb6028eb1d66412bf98595ae72698965cb1cae

PKG_MAINTAINER:=Josef Schlehofer <josef.schlehofer@nic.cz>, Alexandru Ardelean <ardeleanalex@gmail.com>
PKG_LICENSE:=EPL-1.0 Eclipse Distribution License v1.0
PKG_LICENSE_FILES:=epl-v10 edl-v10

include $(TOPDIR)/feeds/packages/lang/python/pypi.mk
include $(INCLUDE_DIR)/package.mk
include $(TOPDIR)/feeds/packages/lang/python/python3-package.mk

define Package/python3-paho-mqtt
  SECTION:=lang
  CATEGORY:=Languages
  SUBMENU:=Python
  TITLE:=MQTT version 5.0/3.1.1 client class
  URL:=http://eclipse.org/paho
  DEPENDS:=+python3-light +python3-idna
  VARIANT:=python3
endef

define Package/python3-paho-mqtt/description
  MQTT version 3.1/3.1.1/5.0 client class
endef

$(eval $(call Py3Package,python3-paho-mqtt))
$(eval $(call BuildPackage,python3-paho-mqtt))

And this is for a package from a personal git repo:

# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk

PKG_NAME:=python-openwrt-test
PKG_VERSION:=0.1
PKG_RELEASE:=1

PKG_MAINTAINER:=Svet Bajlekov <tomatoman@gmail.com>
PKG_LICENSE:=APACHE-2.0
PKG_LICENSE_FILES:=LICENSE

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://github.com/svet-b/python-openwrt-test.git
PKG_SOURCE_VERSION:=a6fc8c118aa57adf296ec1f25b9b71d7176ee321

include $(INCLUDE_DIR)/package.mk
include $(TOPDIR)/feeds/packages/lang/python/python3-package.mk

define Package/python-openwrt-test
  SECTION:=examples
  CATEGORY:=Examples
  TITLE:=Python OpenWRT test
  URL:=https://github.com/svet-b/python-openwrt-test
  DEPENDS:= \
    +python3-light \
    +python3-paho-mqtt
  VARIANT:=python3
endef

define Package/python-openwrt-test/description
  An example Python script that subscribes to the localhost MQTT broker and prints messages
endef

$(eval $(call Py3Package,python-openwrt-test))
$(eval $(call BuildPackage,python-openwrt-test))

In this case PKG_SOURCE_VERSION needs to be set to the hash of the desired git commit.