Details
Description
Goal
We want to improve the Qt For Python build process to provide 3 separate wheel packages for shiboken2, shiboken2-generator and PySide2.
Nice to have (future)
We would also like some additional nice to have things or at least pave the way towards implementing them in the future:
1) Separate builds for each of the of the PySide2 modules. Which will
2) Ease building external modules based on Qt (think non-essential QtCharts, external qwt library) and
3) Allow separate wheels for each module (smaller wheels, use what you need, helps upstream distros)
Proposal
My proposal is to add a --build-type option (among others) to setup.py. This build type will represent:
1) A monolithic build (all which is the default) - the same one we had so far, which builds shiboken2, pyside2, tools.
2) shiboken2 - builds only the shiboken2 module
3) shiboken2-generator - builds only the shiboken2 generator
4) pyside2 - builds only pyside2, needs info where to find shiboken2 module and generator
5) possibly tools - builds pyiside2-tools
In the future extend this to allow:
5) module-subset - allow building just one module or a list of pyside2 modules, needs info where to find the shiboken generator and already built pyside2 modules.
e.g. first build core, gui. Then build only network. Then build only declarative. This will also allow separate wheel generation. And separate wheels for modules like qtcharts, or qtwebengine.
6) possibly external-module - to ease building of custom modules based on qt (think qwt).
Implementation
My idea is to use a multi-step setup.py script which calls itself with different options.
An example of a top-level setup.py invocation which calls additional setup.py invocations would look like:
python setup.py bdist_wheel --build-type=all --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
python setup.py bdist_wheel --internal-build-type=shiboken --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
python setup.py bdist_wheel --internal-build-type=pyside2 --shiboken-path=path_to_previous_built_shiboken --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
The -build-type of the top-level build can be "all", "shiboken2", "shiboken2-generator", "pyside2". The top level setup.py then calls itself with the same usual arguments, except build-type is replaced with internal-build-type which actually triggers the whole build. We might also need -shiboken-path to specify where the previous invocation of shiboken was installed.
If no --build-type is provided, "all" is implied to keep backwards compatibility with previous build commands.
In the future we might need also a --built-modules=/path for supporting separate module building.
General steps
To implement this we will need to:
- Refactor and add certain functionality and options to setup.py to accommodate the new build strategy
- Separate the copying / installing of shiboken files from the PySide2 ones (currently it's a single step)
- Take care of rpaths / PATH in regards to new location of shiboken libraries
- Provide pyside2_config.py and shiboken2_config.py as part of the wheels (based on the scriptable application example ones) to allow end-users easier use of the components.
- Other things I haven't thought about
For future per-module builds we would also need to:
- Change some of the cmake file logic
- Figure out how to pass dependency information for already pre-built modules
- Other things
As the end result of the initial work, we will have 3 separate wheels for shiboken2, shiboken2-generator and pyside2.
Attachments
Issue Links
- relates to
-
PYSIDE-662 Consider extracting shiboken from the generated PySide2 wheel package
- Closed