Chapter 3 - PIP, wheels and C-code

From "pure-Python" sources to wheel

From memory, pycparser is pure-python - so rather than "pip install" it, start with creating a wheel.
Unlike six, in "Chapter 2" the file we have is a gzipped tar file - so we shall actually build and
package as a wheel - without actually installing it!

(pve) michael@x066:[/home/michael]pip wheel pycparser
Collecting pycparser
  Using cached https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
Building wheels for collected packages: pycparser
  Building wheel for pycparser (setup.py) ... done
  Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=111029 sha256=76cb4dad9444e8fa538b99b8d5fea237631c47db33efdeb1ed1cf976b779dd6d
  Stored in directory: /home/michael
Successfully built pycparser

(pve) michael@x066:[/home/michael]pip list
Package    Version
---------- -------
pip        19.3.1
setuptools 41.4.0
six        1.12.0
wheel      0.33.6

TaDa - wheel built, but not installed.

How about installing our freshly built wheel?

Before proceding we need to set some virtual-env config settings

PIP has a command to set certain environment variables. The file for this chapter is located
at ${HOME}/pve/pip.conf. If the file does not exist pip will create it. In any case, we are going to
set some variables.

(pve) $ pip config --site set install.find-links "."
(pve) $ pip config --site set install.no-index true

(pve) $ cat /home/michael/pve/pip.conf
[install]
find-links = .
no-index = true

Now when we use the pip 'install' command pip will only look at what is available in "."

Install or pycparser wheel

(pve) $ pip install pycparser
Looking in links: .
Processing ./pycparser-2.19-py2.py3-none-any.whl
Installing collected packages: pycparser
Successfully installed pycparser-2.19

And now cffi - tar to wheel.

Since this is a 64-bit build of Python we need to prepare the environment,
i.e., tell the compiler, we need 64-bit mode

Note: the C compiler will need to be installed! Not covered here.
(pve) $ export OBJECT_MODE=64

Again, we will create a wheel of cffi - this takes some time, and potentially,
lots of screen lines... when things go wrong.

For now - this is what I saw!

(pve) $ pip wheel cffi
Collecting cffi
  Using cached https://files.pythonhosted.org/packages/0d/aa/c5ac2f337d9a10ee95d160d47beb8d9400e1b2a46bb94990a0409fe6d133/cffi-1.13.1.tar.gz
Collecting pycparser
  Using cached https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz
Building wheels for collected packages: cffi, pycparser
  Building wheel for cffi (setup.py) ... done
  Created wheel for cffi: filename=cffi-1.13.1-cp36-cp36m-AIX_6106_1043_64.whl size=200520 sha256=4384c81bc9474c3b8823474f06cb114e6d6cdba6b536e393750a258570dceca2
  Stored in directory: /home/michael
  Building wheel for pycparser (setup.py) ... done
  Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=111029 sha256=f7949694b22ddc28a9d7524945ffc0f902577635452ace2a71725f21ade460d9
  Stored in directory: /home/michael
Successfully built cffi pycparser

From wheel (with C-code) to installed

(pve) $ pip install cffi
Looking in links: .
Processing ./cffi-1.13.1-cp36-cp36m-AIX_6106_1043_64.whl
Requirement already satisfied: pycparser in ./pve/lib/python3.6/site-packages (from cffi) (2.19)
Installing collected packages: cffi
Successfully installed cffi-1.13.1

And this includes the the shared library _cffi_backend.so

(pve) michael@x066:[/home/michael]find pve -name \*.so | grep cffi
pve/lib/python3.6/site-packages/_cffi_backend.so
(pve) michael@x066:[/home/michael]dump -H pve/lib/python3.6/site-packages/_cffi_backend.so

pve/lib/python3.6/site-packages/_cffi_backend.so:

                        ***Loader Section***
                      Loader Header Information
VERSION#         #SYMtableENT     #RELOCent        LENidSTR
0x00000001       0x000000c4       0x00000516       0x0000004f

#IMPfilID        OFFidSTR         LENstrTBL        OFFstrTBL
0x00000004       0x000063f8       0x00000e85       0x00006447


                        ***Import File Strings***
INDEX  PATH                          BASE                MEMBER
0      /usr/vac/lib:/usr/lib:/lib
1                                    libpthreads.a       shr_xpg5_64.o
2                                    libc.a              shr_64.o

And my wheels look like:

(pve) michael@x066:[/home/michael]ls -l *whl
-rw-r--r--    1 michael  staff        200520 Oct 31 09:46 cffi-1.13.1-cp36-cp36m-AIX_6106_1043_64.whl
-rw-r--r--    1 michael  staff        111029 Oct 31 09:46 pycparser-2.19-py2.py3-none-any.whl
-rw-r--r--    1 michael  staff         10586 Oct 31 09:00 six-1.12.0-py2.py3-none-any.whl

Looking forward

• I hope to hear (via forums) that you see added utility for adequately tagged wheels on AIX
• I also hope to have enough time to make some additional improvements in the PIP and/or WHEEL projects so that AIX support is "mature".