WebAssembly: dynamic linking and dlopen

What is the benefit? Why is this valuable?

Issue: Dynamic linking could provide smaller downloads by only downloading the modules that are needed. Also needed for binary QML and shared libraries that have been requested by several customers.

What are common use cases?

Solution: Qt WebAssembly provides dynamic linking support to allow optimising download sizes and allows shared libraries.

Technical information

External Documentation

WebAssembly supports modules and dynamic linking. These are low-level APIs

Emscripten supports dynamic linking with compiler options and a run-time API.

Dynamic Linking for Qt

Emscripten supports dividing programs into one main module and several side modules, where the main module contains all system libraries. This is controlled by setting -s MAIN_MODULE=1/-s SIDE_MODULE=1 on the linker line.

There are two choices for the main module:

• Qt Core
• The application binary

The main module then needs to link/include all needed system libraries. It's possible to set EMCC_FORCE_STDLIBS=1 (on the environment) to enable "kitchen sink" mode, but this option is currently broken.

Setting MAIN_MODULE=1 makes emscripten not generate wasm fetching code: qtloader.js should either implement Module.readBinary or fetch the main wasm module and set Module.wasmBinary. (Going one step further we should look into compiling the wasm file from qtloader.js as well in order to enable streaming compiles. This is done by setting Module.instantiateWasm)

Side modules are loaded by setting Module.dynamicLibraries to an array of wasm file paths (not binaries!), which emscripten will fetch and compile. The module list can either be created manually or be provided by the build system (we'll aim to support both).

dlopen for Qt

Emscripten supports dlopen()ning side modules. The dlopen implementation expects to find the side module wasm binary on the filesystem: dlopen("foo.wasm") reads "/foo.wasm" and so on.

The easiest way to handle this is to preload the wasm files onto the (default-enabled) MEMFS file system in qtloader.js. All modules must then be specified at load time. This runs somewhat counter to the dynamic nature of dlopen. There are however other possible issues that prevents true dlopen-any-file:

• dlopen is synchronous API, which is hard/impossible to implement on the main thread
• The callers of dlopen may expect it to return in a reasonable time (and not wait for a download)

Prototypes and implementation

Minimal emscripten prototype: emscripten_dynamic

Build Qt libs as wasm modules WIP: https://codereview.qt-project.org/230044

Load libQtCore.wasm prototype: https://github.com/msorvig/qt-webassembly-examples/tree/master/core_dynamic

Qt Implementation Tasks

• EM_ASM is not supported for side modules; figure out what to do.
   ** Check if the EM_JS works: https://github.com/kripken/emscripten/commit/5f697f9349ea3662ee8f38ce439218ee4d22b5a0
• Set MAIN_MODULE/SIDE_MODULE as needed
• Add qtloader.js API and implementation for loading the main module / dynamic side modules / dlopen side modules
   ** Prototype: https://github.com/msorvig/qt-webassembly-examples/tree/master/emscripten_dynamic
• See if we can get the Qt library and plugin list from qmake
• Port/Test QLibrary and QPlugin(Loader)