Chapter 3 - PIP, wheels and C-code
A simple example is the module cffi. This module as a pure-python dependency
and it has it's own C language component.
The easiest way to see what is needed is to download cffi - this will also download it's dependencies.
(pve) michael@x066:[/home/michael]pip download cffi
Collecting cffi
Downloading
https://files.pythonhosted.org/packages/0d/aa/c5ac2f337d9a10ee95d160d47beb8d9400e1b2a46bb94990a0409fe6d133/cffi-1.13.1.tar.gz
(460kB)
|################################| 460kB 3.7MB/s
Saved ./cffi-1.13.1.tar.gz
Collecting pycparser
Downloading
https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
(158kB)
|################################| 163kB 7.5MB/s
Saved ./pycparser-2.19.tar.gz
Successfully downloaded cffi pycparser
From "pure-Python" sources to wheel
From memory, pycparser is pure-python - so rather than "pip install" it, start with creating a wheel.
Unlike six, in "Chapter 2" the file we have is a gzipped tar file - so we shall actually build and
package as a wheel - without actually installing it!
(pve) michael@x066:[/home/michael]pip wheel pycparser
Collecting pycparser
Using cached https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
Building wheels for collected packages: pycparser
Building wheel for pycparser (setup.py) ... done
Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=111029 sha256=76cb4dad9444e8fa538b99b8d5fea237631c47db33efdeb1ed1cf976b779dd6d
Stored in directory: /home/michael
Successfully built pycparser
(pve) michael@x066:[/home/michael]pip list
Package Version
---------- -------
pip 19.3.1
setuptools 41.4.0
six 1.12.0
wheel 0.33.6
TaDa - wheel built, but not installed.
How about installing our freshly built wheel?
Before proceding we need to set some virtual-env config settings
PIP has a command to set certain environment variables. The file for this chapter is located
at ${HOME}/pve/pip.conf. If the file does not exist pip will create it. In any case, we are going to
set some variables.
(pve) $ pip config --site set install.find-links "."
(pve) $ pip config --site set install.no-index true
(pve) $ cat /home/michael/pve/pip.conf
[install]
find-links = .
no-index = true
Now when we use the pip 'install' command pip will only look at what is available in "."
Install or pycparser wheel
(pve) $ pip install pycparser
Looking in links: .
Processing ./pycparser-2.19-py2.py3-none-any.whl
Installing collected packages: pycparser
Successfully installed pycparser-2.19
And now cffi - tar to wheel.
Since this is a 64-bit build of Python we need to prepare the environment,
i.e., tell the compiler, we need 64-bit mode
Note: the C compiler will need to be installed! Not covered here.
(pve) $ export OBJECT_MODE=64
Again, we will create a wheel of cffi - this takes some time, and potentially,
lots of screen lines... when things go wrong.
For now - this is what I saw!
(pve) $ pip wheel cffi
Collecting cffi
Using cached https://files.pythonhosted.org/packages/0d/aa/c5ac2f337d9a10ee95d160d47beb8d9400e1b2a46bb94990a0409fe6d133/cffi-1.13.1.tar.gz
Collecting pycparser
Using cached https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
Building wheels for collected packages: cffi, pycparser
Building wheel for cffi (setup.py) ... done
Created wheel for cffi: filename=cffi-1.13.1-cp36-cp36m-AIX_6106_1043_64.whl size=200520 sha256=4384c81bc9474c3b8823474f06cb114e6d6cdba6b536e393750a258570dceca2
Stored in directory: /home/michael
Building wheel for pycparser (setup.py) ... done
Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=111029 sha256=f7949694b22ddc28a9d7524945ffc0f902577635452ace2a71725f21ade460d9
Stored in directory: /home/michael
Successfully built cffi pycparser
From wheel (with C-code) to installed
(pve) $ pip install cffi
Looking in links: .
Processing ./cffi-1.13.1-cp36-cp36m-AIX_6106_1043_64.whl
Requirement already satisfied: pycparser in ./pve/lib/python3.6/site-packages (from cffi) (2.19)
Installing collected packages: cffi
Successfully installed cffi-1.13.1
And this includes the the shared library _cffi_backend.so
(pve) michael@x066:[/home/michael]find pve -name \*.so | grep cffi
pve/lib/python3.6/site-packages/_cffi_backend.so
(pve) michael@x066:[/home/michael]dump -H pve/lib/python3.6/site-packages/_cffi_backend.so
pve/lib/python3.6/site-packages/_cffi_backend.so:
***Loader Section***
Loader Header Information
VERSION# #SYMtableENT #RELOCent LENidSTR
0x00000001 0x000000c4 0x00000516 0x0000004f
#IMPfilID OFFidSTR LENstrTBL OFFstrTBL
0x00000004 0x000063f8 0x00000e85 0x00006447
***Import File Strings***
INDEX PATH BASE MEMBER
0 /usr/vac/lib:/usr/lib:/lib
1 libpthreads.a shr_xpg5_64.o
2 libc.a shr_64.o
And my wheels look like:
(pve) michael@x066:[/home/michael]ls -l *whl
-rw-r--r-- 1 michael staff 200520 Oct 31 09:46 cffi-1.13.1-cp36-cp36m-AIX_6106_1043_64.whl
-rw-r--r-- 1 michael staff 111029 Oct 31 09:46 pycparser-2.19-py2.py3-none-any.whl
-rw-r--r-- 1 michael staff 10586 Oct 31 09:00 six-1.12.0-py2.py3-none-any.whl
Looking forward
- I hope to hear (via forums) that you see added utility for adequately tagged wheels on AIX
- I also hope to have enough time to make some additional improvements in the PIP and/or WHEEL projects so that AIX support is "mature".
|