Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-749

Modularize / decouple the shiboken / PySide2 build process

    XMLWordPrintable

Details

    • User Story
    • Resolution: Done
    • P2: Important
    • 5.12.0
    • 5.11.0
    • PySide, Shiboken
    • None
    • 43fe3494a9d902034896e3afa7b5158c77163be0

    Description

      Goal

      We want to improve the Qt For Python build process to provide 3 separate wheel packages for shiboken2, shiboken2-generator and PySide2.

      Nice to have (future)

      We would also like some additional nice to have things or at least pave the way towards implementing them in the future:
      1) Separate builds for each of the of the PySide2 modules. Which will
      2) Ease building external modules based on Qt (think non-essential QtCharts, external qwt library) and
      3) Allow separate wheels for each module (smaller wheels, use what you need, helps upstream distros)

      Proposal

      My proposal is to add a --build-type option (among others) to setup.py. This build type will represent:
      1) A monolithic build (all which is the default) - the same one we had so far, which builds shiboken2, pyside2, tools.
      2) shiboken2 - builds only the shiboken2 module
      3) shiboken2-generator - builds only the shiboken2 generator
      4) pyside2 - builds only pyside2, needs info where to find shiboken2 module and generator
      5) possibly tools - builds pyiside2-tools

      In the future extend this to allow:
      5) module-subset - allow building just one module or a list of pyside2 modules, needs info where to find the shiboken generator and already built pyside2 modules.
      e.g. first build core, gui. Then build only network. Then build only declarative. This will also allow separate wheel generation. And separate wheels for modules like qtcharts, or qtwebengine.
      6) possibly external-module - to ease building of custom modules based on qt (think qwt).

      Implementation

      My idea is to use a multi-step setup.py script which calls itself with different options.
      An example of a top-level setup.py invocation which calls additional setup.py invocations would look like:

      python setup.py bdist_wheel --build-type=all --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
      
      • python setup.py bdist_wheel --internal-build-type=shiboken --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
        
      • python setup.py bdist_wheel --internal-build-type=pyside2 --shiboken-path=path_to_previous_built_shiboken --qmake=/home/alex/qt/bin --jobs=8 --ignore-git --debug --limited-api=yes
        

      The -build-type of the top-level build can be "all", "shiboken2", "shiboken2-generator", "pyside2". The top level setup.py then calls itself with the same usual arguments, except build-type  is replaced with internal-build-type which actually triggers the whole build. We might also need -shiboken-path to specify where the previous invocation of shiboken was installed.

      If no --build-type is provided, "all" is implied to keep backwards compatibility with previous build commands.

      In the future we might need also a --built-modules=/path for supporting separate module building.

      General steps

      To implement this we will need to:

      1. Refactor and add certain functionality and options to setup.py to accommodate the new build strategy
      2. Separate the copying / installing of shiboken files from the PySide2 ones (currently it's a single step)
      3. Take care of rpaths / PATH in regards to new location of shiboken libraries
      4. Provide pyside2_config.py and shiboken2_config.py as part of the wheels (based on the scriptable application example ones) to allow end-users easier use of the components.
      5. Other things I haven't thought about

      For future per-module builds we would also need to:

      1. Change some of the cmake file logic
      2. Figure out how to pass dependency information for already pre-built modules
      3. Other things

      As the end result of the initial work, we will have 3 separate wheels for shiboken2, shiboken2-generator and pyside2.

       

      Attachments

        Issue Links

          Activity

            People

              alexandru.croitor Alexandru Croitor
              alexandru.croitor Alexandru Croitor
              Alex Blasche Alex Blasche
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 2 weeks
                  2w
                  Remaining:
                  Remaining Estimate - 2 weeks
                  2w
                  Logged:
                  Time Spent - Not Specified
                  Not Specified