OpenWrt Forum Archive

Topic: Python 3.2 and OpenWRT

The content of this topic has been archived between 28 Apr 2018 and 5 May 2018. Unfortunately there are posts – most likely complete pages – missing.

linux718 wrote:

I followed those direction while building barrier breaker.  I get the following failure:
Package python3 is missing dependencies for the following libraries: libdb-4.7.so

I have libdb47 and libdb47xx both selected and built.  I simply selected python3 (and python3-mini) in make menuconfig and got that error. 

What do I do?

I have never tried a complete build of all OpenWrt packages, so sometimes the python3 build system pulls in dependencies I have not thought about.

I packaged dbm.ndbm which should fix this problem, and lzma which I also discovered.

(Last edited by Cybjit on 21 Jun 2013, 23:15)

Hello!

I have some concerns regarding the Makefile used to compile python. Recently, I decided to check the Cython tool to implement a external module. I noticed that the Makefile disables the shared libs compilation.
I commented the --disable-shared option but normal compilation does not goes as expected.

Can you give me some help here? smile

Thanks in advanced!

Ok. I noticed that removing the --disable-shared is not enough. It is also necessary to enable it by --enable-shared command.

But now I'm having a different problem.

ln -sf python3.3m /home/openwrt/Desktop/OpenWRT-XEN/openwrt/build_dir/target-i386_uClibc-0.9.33.2/Python-3.3.2/ipkg-x86/python3-mini/usr/bin/python3
find /home/openwrt/Desktop/OpenWRT-XEN/openwrt/build_dir/target-i386_uClibc-0.9.33.2/Python-3.3.2/ipkg-x86/python3-mini -name 'CVS' -o -name '.svn' -o -name '.#*' -o -name '*~'| xargs -r rm -rf
Package python3-mini is missing dependencies for the following libraries:
libpython3.3m.so.1.0
make[3]: *** [/home/openwrt/Desktop/OpenWRT-XEN/openwrt/bin/x86/packages/python3-mini_3.3.2-1_x86.ipk] Error 1
make[3]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt/feeds/python3/lang/python3'
make[2]: *** [package/feeds/python3/python3/compile] Error 2
make[2]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt'
make[1]: *** [/home/openwrt/Desktop/OpenWRT-XEN/openwrt/staging_dir/target-i386_uClibc-0.9.33.2/stamp/.package_compile] Error 2
make[1]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt'
make: *** [world] Error 2
[openwrt@ArchLinux32Bits openwrt]$ 

It seems i'm missing the libpython3.3m.so.1.0 for python-mini and I don't know how, in the Makefile, to add it to the list of dependencies.

(Last edited by Claymore on 6 Sep 2013, 11:48)

Claymore wrote:

Ok. I noticed that removing the --disable-shared is not enough. It is also necessary to enable it by --enable-shared command.

But now I'm having a different problem.

ln -sf python3.3m /home/openwrt/Desktop/OpenWRT-XEN/openwrt/build_dir/target-i386_uClibc-0.9.33.2/Python-3.3.2/ipkg-x86/python3-mini/usr/bin/python3
find /home/openwrt/Desktop/OpenWRT-XEN/openwrt/build_dir/target-i386_uClibc-0.9.33.2/Python-3.3.2/ipkg-x86/python3-mini -name 'CVS' -o -name '.svn' -o -name '.#*' -o -name '*~'| xargs -r rm -rf
Package python3-mini is missing dependencies for the following libraries:
libpython3.3m.so.1.0
make[3]: *** [/home/openwrt/Desktop/OpenWRT-XEN/openwrt/bin/x86/packages/python3-mini_3.3.2-1_x86.ipk] Error 1
make[3]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt/feeds/python3/lang/python3'
make[2]: *** [package/feeds/python3/python3/compile] Error 2
make[2]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt'
make[1]: *** [/home/openwrt/Desktop/OpenWRT-XEN/openwrt/staging_dir/target-i386_uClibc-0.9.33.2/stamp/.package_compile] Error 2
make[1]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt'
make: *** [world] Error 2
[openwrt@ArchLinux32Bits openwrt]$ 

It seems i'm missing the libpython3.3m.so.1.0 for python-mini and I don't know how, in the Makefile, to add it to the list of dependencies.

python3-mini is the base package, so if it depends on libpython3.3m.so.1.0 you should add it to the filespec.
add a - line to the PyPackage/python3/filespec and a + line to PyPackage/python3-mini/filespec

Look at post #20 at Python 3.2 and OpenWrt
Builds perfectly here :-)

oliver@oliver-desktop:~/wndr3700$ (cd bin/ar71xx/packages/ && ll -h python3*)
-rw-r--r-- 1 oliver oliver 1.8M Sep  7 21:01 python3_3.3.2-1_ar71xx.ipk
-rw-r--r-- 1 oliver oliver 1.6M Sep  7 21:00 python3-mini_3.3.2-1_ar71xx.ipk
oliver@oliver-desktop:~/wndr3700$

(Last edited by written_direcon on 7 Sep 2013, 20:05)

Cybjit wrote:
Claymore wrote:

Ok. I noticed that removing the --disable-shared is not enough. It is also necessary to enable it by --enable-shared command.

But now I'm having a different problem.

ln -sf python3.3m /home/openwrt/Desktop/OpenWRT-XEN/openwrt/build_dir/target-i386_uClibc-0.9.33.2/Python-3.3.2/ipkg-x86/python3-mini/usr/bin/python3
find /home/openwrt/Desktop/OpenWRT-XEN/openwrt/build_dir/target-i386_uClibc-0.9.33.2/Python-3.3.2/ipkg-x86/python3-mini -name 'CVS' -o -name '.svn' -o -name '.#*' -o -name '*~'| xargs -r rm -rf
Package python3-mini is missing dependencies for the following libraries:
libpython3.3m.so.1.0
make[3]: *** [/home/openwrt/Desktop/OpenWRT-XEN/openwrt/bin/x86/packages/python3-mini_3.3.2-1_x86.ipk] Error 1
make[3]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt/feeds/python3/lang/python3'
make[2]: *** [package/feeds/python3/python3/compile] Error 2
make[2]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt'
make[1]: *** [/home/openwrt/Desktop/OpenWRT-XEN/openwrt/staging_dir/target-i386_uClibc-0.9.33.2/stamp/.package_compile] Error 2
make[1]: Leaving directory `/home/openwrt/Desktop/OpenWRT-XEN/openwrt'
make: *** [world] Error 2
[openwrt@ArchLinux32Bits openwrt]$ 

It seems i'm missing the libpython3.3m.so.1.0 for python-mini and I don't know how, in the Makefile, to add it to the list of dependencies.

python3-mini is the base package, so if it depends on libpython3.3m.so.1.0 you should add it to the filespec.
add a - line to the PyPackage/python3/filespec and a + line to PyPackage/python3-mini/filespec

Thanks! It worked perfectly! smile

@written_directon
I was trying to compile Python 3.3 with Shared libs (*.so). Cybjit Makefile compiles python 3.3 with static libs (.a).
The reason for me wanting to do this, is because I need to write some Python extensions and if I do it with static libs, the compiler will add the full lib to the extension. this makes the extension module go from a mere 50KB to 2MB.

Anything else, cybjit Makefile works perfectly!

(Last edited by Claymore on 8 Sep 2013, 16:47)

Cybjit wrote:

I actually already have a repository with 3.3, that should be easier to integrate with the standard build. I have just not posted about how to use it yet.

Thanks for Python 3.3, I just built 3.3.2 for my OpenWRT router.

What about python3-readline though? Does your old patch still work (from OpenWRT bug #8102 at https://dev.openwrt.org/ticket/8102) and do you think you could include it in this too?

Edit: well, since the abovementioned patch is actually to libreadline and I have a patched libreadline installed, all I needed to do for python3-readline was to remove the @BROKEN from the Makefile and go. Works perfectly!

Ideally, I'd like to have IPython3 package as well...

Edit: looks like IPython would create an ipython3 script just fine if its setup.py is run with python3. Only problem is, that requires setuptools and I need to install that first...

(Last edited by amrs on 24 Sep 2013, 11:57)

Claymore wrote:

Thanks! It worked perfectly! smile

@written_directon
I was trying to compile Python 3.3 with Shared libs (*.so). Cybjit Makefile compiles python 3.3 with static libs (.a).
The reason for me wanting to do this, is because I need to write some Python extensions and if I do it with static libs, the compiler will add the full lib to the extension. this makes the extension module go from a mere 50KB to 2MB.

Anything else, cybjit Makefile works perfectly!

smile

I briefly tried to enable shared libs, but all the C modules from the standard library failed to build. If you solved that problem it would be nice if you published your modifications somewhere.

amrs wrote:

Thanks for Python 3.3, I just built 3.3.2 for my OpenWRT router.

What about python3-readline though? Does your old patch still work (from OpenWRT bug #8102 at https://dev.openwrt.org/ticket/8102) and do you think you could include it in this too?

Edit: well, since the abovementioned patch is actually to libreadline and I have a patched libreadline installed, all I needed to do for python3-readline was to remove the @BROKEN from the Makefile and go. Works perfectly!

Ideally, I'd like to have IPython3 package as well...

Edit: looks like IPython would create an ipython3 script just fine if its setup.py is run with python3. Only problem is, that requires setuptools and I need to install that first...

Well, the patch need to be added to trunk, so there is nothing I can do from packages really. But as you discovered it still applies. If one removes @BROKEN from python3-readline without the patch it still segfaults.

Setuptools (and pip) would be useful to have on the router. I have had some Makefiles lying around, might update and push them today.

Testing installing IPython3 with pip on the router seem to proceed, but the 2to3 phase is really slow. (10-20 files a minute, 560 files total). So I can see why one would want a ready made package.

Edit: pushed python3-setuptools and python3-pip. IPython3 install seems to have succeeded after 30 min install time.

(Last edited by Cybjit on 24 Sep 2013, 19:34)

@Cybjit

I need following python3 packages:

- python3-imaging-library
- python3-curl
- python3-crypto
- python3-django

Can you add those four peaces to your git repository please?

(Last edited by written_direcon on 8 Oct 2013, 21:24)

@Cybjit
I was able to compile with shared libs but then there where required libs that where not simply there, after installing python3 in the openwrt machine. Weird things were happening like, import time did not work because the required shared lib was not there.

After that, I lost focus because other things came up at work. What I was trying to do was implementing some Python Extensions using Cython. The reason for shared libs was because I didn't want a extention to be over 2 megas just because it had to have the entire python module inside.

Anyway, thanks for the setup-tools smile

PS: Sorry for taking this long to respond. I didn't notice you had answered because of my filter in the Gmail box. Only noticed now because I updated the python repo in my local OpenWRT repo and saw that there were modifications.

(Last edited by Claymore on 11 Oct 2013, 16:05)

Claymore wrote:

Python 3.4 is out! It brings some goodies that I am eager to try out smile

This is probably my favorite
http://docs.python.org/3.4/library/asyncio.html

I have now updated the python3 package to 3.4.0.

The regression tests for the asyncio are mostly OK, but 6 out of 788 test failed.
All failed with the same AssertionError.

FAIL: test_create_server_ssl_verify_failed (test.test_asyncio.test_events.EPollEventLoopTests)
FAIL: test_create_server_ssl_verify_failed (test.test_asyncio.test_events.PollEventLoopTests)
FAIL: test_create_server_ssl_verify_failed (test.test_asyncio.test_events.SelectEventLoopTests)
FAIL: test_create_unix_server_ssl_verify_failed (test.test_asyncio.test_events.EPollEventLoopTests)
FAIL: test_create_unix_server_ssl_verify_failed (test.test_asyncio.test_events.PollEventLoopTests)
FAIL: test_create_unix_server_ssl_verify_failed (test.test_asyncio.test_events.SelectEventLoopTests)

AssertionError: "certificate verify failed " does not match "[SSL: CERTIFICATE_VERIFY_FAILED] unknown error (_ssl.c:598)"

written_direcon wrote:

@Cybjit

I need following python3 packages:

- python3-imaging-library
- python3-curl
- python3-crypto
- python3-django

Can you add those four peaces to your git repository please?

I have held out for a long time creating Makefiles for extra modules, since it would take a lot of time to keep up to date.

But a few months ago I created a Makefile builder from PyPI metadata.

So now these should have Makefiles generated, along with a bunch of others. Almost no testing have been done of them however.

python3-crypto
python3-dateutil
python3-django
python3-flask
python3-httplib2
python3-jinja2
python3-lxml
python3-openssl
python3-pil
python3-pip
python3-postgresql
python3-psycopg2
python3-pycurl
python3-pymongo
python3-pymysql
python3-requests
python3-serial
python3-setuptools
python3-simplejson
python3-six
python3-sqlalchemy
python3-tz
python3-urllib3
python3-werkzeug
python3-yaml
ipython3

(Last edited by Cybjit on 23 Mar 2014, 23:01)

Thanks Cybjit for your effort in this!
I would really like that the OpenWRT dev's would add these makefiles to the official tree.
I will keep an eye on the problems you pointed.

Hello again, Cybjit!
I'm having trouble in compiling the packets crypto, dateutil  and serial.

The same error ocurred for all three packets

cp: cannot stat '/home/openwrt/Desktop/OpenWRT/OpenWRT-12.09/build_dir/target-i386_uClibc-0.9.33.2/pyserial-2.7//ipkg-install/usr/lib/python3.4/site-packages/*': No such file or directory
Makefile:69: recipe for target '/home/openwrt/Desktop/OpenWRT/OpenWRT-12.09/staging_dir/target-i386_uClibc-0.9.33.2/root-x86/stamp/.python3-serial_installed' failed

Packets openssl, setuptools and simplejson compiled and installed well.

Claymore wrote:

Hello again, Cybjit!
I'm having trouble in compiling the packets crypto, dateutil  and serial.

The same error ocurred for all three packets

cp: cannot stat '/home/openwrt/Desktop/OpenWRT/OpenWRT-12.09/build_dir/target-i386_uClibc-0.9.33.2/pyserial-2.7//ipkg-install/usr/lib/python3.4/site-packages/*': No such file or directory
Makefile:69: recipe for target '/home/openwrt/Desktop/OpenWRT/OpenWRT-12.09/staging_dir/target-i386_uClibc-0.9.33.2/root-x86/stamp/.python3-serial_installed' failed

Packets openssl, setuptools and simplejson compiled and installed well.

This is probably a result of the variables in python3-package.mk changing, while the packages Makefiles remain unchanged.
In that case the build system does not trigger the needed rebuild.
I have now made changes to all package Makefiles, which should fix this

(Last edited by Cybjit on 6 Jun 2014, 20:25)

Once again, thanks for your effort Cybjit. smile

thank you so much, Cybjit.

I am able to compile all the 3.4 packages.

Hi,

Could you help me?

I'm trying to build Python3.4 but I get this error when compile python-mini

checking for python interpreter for cross build... python3.4: error while loading shared libraries: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
python3: error while loading shared libraries: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
configure: error: python3.4 interpreter not found


Thanks!

roger_ wrote:

Someone should start a pull request for this on GitHub and get it added: https://github.com/openwrt/packages

You can do it :-)

I had to disable python3-lzma, since liblzma in no longer available.


ayasystems wrote:

checking for python interpreter for cross build... python3.4: error while loading shared libraries: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
python3: error while loading shared libraries: libpython3.4m.so.1.0: cannot open shared object file: No such file or directory
configure: error: python3.4 interpreter not found

Probably just me being lazy with the versioning of the package, cleaning and rebuilding should resolve it.

Hello, can someone point me to a guide how to install python3 on Barrier Breaker ?

I recently updated my OpenWRT git trunk and noticed python3 has been added to the packages_github feed. What I don't know is if this addition can cause some problems in compiling the python3 by cybjit. At any rate, after my local OpenWRT git trunk has been updated to include the python3, the python3 by cybjit is no longer cleanly compilable and gives error messages as shown below. Sorry, the forum doesn't allow to attach the 600+ KB log file. I hope cybjit and/or anyone will be able to fix this problem. Thank you.

/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_EchoChar':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1136:18: error: dereferencing pointer to incomplete type
     if (self->win->_flags & _ISPAD)
                  ^
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_NoOutRefresh':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1659:22: error: dereferencing pointer to incomplete type
         if (self->win->_flags & _ISPAD)
                      ^
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_Refresh':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1834:22: error: dereferencing pointer to incomplete type
         if (self->win->_flags & _ISPAD)
                      ^
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_SubWin':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1901:18: error: dereferencing pointer to incomplete type
     if (self->win->_flags & _ISPAD)
                  ^
...

/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_EchoChar':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1136:18: error: dereferencing pointer to incomplete type
     if (self->win->_flags & _ISPAD)
                  ^
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_NoOutRefresh':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1659:22: error: dereferencing pointer to incomplete type
         if (self->win->_flags & _ISPAD)
                      ^
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_Refresh':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1834:22: error: dereferencing pointer to incomplete type
         if (self->win->_flags & _ISPAD)
                      ^
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c: In function 'PyCursesWindow_SubWin':
/opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Modules/_cursesmodule.c:1901:18: error: dereferencing pointer to incomplete type
     if (self->win->_flags & _ISPAD)
                  ^

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_curses_panel         _tkinter              nis                
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_curses                                                        

running build_scripts
copying /opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Tools/scripts/pydoc3 -> build/scripts-3.4
copying /opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Tools/scripts/idle3 -> build/scripts-3.4
copying /opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Tools/scripts/2to3 -> build/scripts-3.4
copying /opt/openwrt-git-trunk/build_dir/target-arm_xscale_uClibc-0.9.33.2_eabi/Python-3.4.2rc1/Tools/scripts/pyvenv -> build/scripts-3.4
Cybjit wrote:
Claymore wrote:

Hello Again!

Python 3.3 was released a week ago and with it, the new OpenWRT version
I would like to ask you, if you could please, update the package script to support the lates version?  It would be a great help.
Thanks for your effort!

I actually already have a repository with 3.3, that should be easier to integrate with the standard build. I have just not posted about how to use it yet.


Make sure git is installed.
In buildroot dir create feeds.conf if it does not exist

[ ! -e feeds.conf ] && cp feeds.conf.default feeds.conf

Add the repository to feeds.conf

echo src-git python3 git://gitorious.org/openwrt-cybjit/python3.git >> feeds.conf

Update and install the feed

./scripts/feeds update python3
./scripts/feeds install python3

Select the python3 packages in menuconfig, and build.

Is your git repository still being maintained or has it been moved to other git server? The reason I asked is because I have been getting an error message (fatal: The remote end hung up unexpectedly) for the past two weeks when trying to update my local trunk.