29 Commits

Author SHA1 Message Date
Piotr Wiercinski
caa0aa3fb4 wasm: Fix Brotli compression in qtwasmserver.py
There is not compress() function in brotli.Compressor API.
Use process().
Use requirements.txt instead of Pipfile.

Change-Id: I55a0263f16f36bcb4b96e443f85925b7d5dd15af
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2025-03-26 19:35:48 +01:00
Juha Vuolle
fcada7c5c3 qtwasmserver to serve assets from the provided path parameter
qtwasmserver accepts a positional path argument which tells where to
serve the assets from. The argument wasn't actually used, and this
resulted in always using the cwd.

In addition add a check for the path directory existence. Otherwise
this becomes only visible as a 404 Not Found error.

Amends: 156e5c8b690d01ad3043d2163168c4ea3608a046

Pick-to: 6.9
Fixes: QTBUG-134393
Change-Id: Iacfafe8a2fb2409169b09a17dbc9ffed0ad16fdf
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2025-03-07 13:02:47 +02:00
Piotr Wierciński
010ed17884 wasm: Add helper for pluging preloads
Dynamic linking on WebAssembly involves preloading .so libraries
during startup of application. Normally, those plugin preload
lists are generating manually by user, which can be tedious.
Add a bash script which demonstrates how to call python
programs to generate preload lists.

Pick-to: 6.9
Change-Id: I0a9869ad0d26606f8b33af2c38248cec3088dd0d
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2025-01-13 18:23:10 +00:00
Alexandru Croitor
9804741d98 Revert "wasm: Generate plugin preloads for dynamic linking at install step"
This reverts commit f0f89d3c1946fa819c01b28d757ba0a5074dd163.

Reason for revert: Breaks qttools. Installs without consent in user projects.

Fixes: QTBUG-130717
Change-Id: I2a856cc89cc144f9d69c75a396ba9b8fa4fba482
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2024-10-31 17:27:05 +00:00
Morten Sørvig
156e5c8b69 wasm: update qtwasmserver.py
Bulk update to qtwasmserver 0.1.4 (as distributed on pypi)

Removes the usage of the deprecated/removed wrap_socket function.

Fixes: QTBUG-128938
Change-Id: I03262c799b34f2114150949f4a660c50317d44be
Reviewed-by: Lorn Potter <lorn.potter@qt.io>
Reviewed-by: Piotr Wierciński <piotr.wiercinski@qt.io>
2024-10-23 17:46:00 +02:00
Piotr Wierciński
f0f89d3c19 wasm: Generate plugin preloads for dynamic linking at install step
Dynamic linking on WebAssembly involves preloading .so libraries
during startup of application. Normally, those plugin preload
lists are generating manually by user.
Automate this process as part of installation step.

Change-Id: I364ebdb170f9fac53da241c96f601613352972d8
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2024-10-21 18:34:57 +02:00
Piotr Wiercinski
a6e7274704 wasm: make qtloader.js use FS.createPreloadedFile when preloading
Currently qtloader.js fetches and copies the files manually. By doing
so we are missing some preproccessing by Emscripten preload plugins.
Use Emscripten API to preload files, so preload plugin for .so can
download, compile and resolve dependencies of imported shared libraries.

This makes looking for dependencies in preload_qml_import.py no longer
needed. Remove redundant code.

Fixes: QTBUG-121817
Change-Id: Idd35f25d5f54123910f813a636407eea23e157cb
Reviewed-by: Piotr Wierciński <piotr.wiercinski@qt.io>
2024-03-14 14:54:53 +00:00
Lucie Gérard
f9136fd0db Correct wasm util file
License is set to that of the other files in the directory

Pick-to: 6.7
Task-number: QTBUG-121787
Change-Id: I9922feb2a520a549ab9cc1d82a85417202ad83c3
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
2024-03-07 16:10:02 +01:00
Piotr Wiercinski
382ee7e200 wasm: Parse wasm binaries to get dependencies
Currently we are searching binaries for string ending with ".so" in
order to find shared library dependencies. This is not the cleanest
solution.
Parse wasm binaries instead, and extract the information in structured
way.

Fixes: QTBUG-121833
Change-Id: Id0efb07af4efe7641ed4fd66f76c395722478f95
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2024-02-23 20:41:29 +01:00
Morten Sørvig
3424234579 wasm: make preload_qml_imports take a source path
This was running qmlimportscanner on ".", which in some
cases would recurse down the qt/ symlink in the app
build directory (used with dynamic linking on wasm),
and find all of the imports from the Qt installation.

Make it take a path instead. Users can then provide
a path to the QML sources which will be passed to
qmlimportscanner

Change-Id: Ib5175e5dc1d26875c42f5a3e286314b7d602c9fe

Reviewed-by: Piotr Wierciński <piotr.wiercinski@qt.io>
Reviewed-by: Lorn Potter <lorn.potter@gmail.com>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2024-02-20 17:25:25 +01:00
Lucie Gérard
657e0ec343 Correct util file to reduce exceptions
Pick-to: 6.7
Task-number: QTBUG-121787
Change-Id: I538ecece16c6daeef4394e29a1581c52bd4d9d02
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
2024-02-20 16:22:28 +01:00
Piotr Wierciński
d21bfee667 wasm: Fix duplication of .so files in preload_qml_imports.py
The naive approach of "find_dependencies()" function, sometimes finds a
given plugin itself as its own dependency, which results in duplication
and wrong paths.
Remove the plugin from its own dependencies list if needed.

Task-number: QTBUG-121833
Change-Id: I4cf30233cfc291d360f69158bd12bd36024832be
Reviewed-by: Lorn Potter <lorn.potter@gmail.com>
2024-02-13 13:55:01 +01:00
Piotr Wierciński
4f168621d2 wasm: Fix test runner for asynchronous tests
Test runner was not properly handling tests which
return the control back to browser event loop.
It was treating such tests as if they exited with
code 0, marking them as succesfull even if they were
eventually failing or hanging.
This commit adds a callback to TestCase so the runner
is notified when a test truly has finished.
As a side effect, two tests need to be disabled for now
as they are failing for wasm, which was not properly
detected previously.

Change-Id: I0eb9383e5bb9cd660431c18747b9e94413629d1e
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
2023-10-26 13:43:39 +02:00
Mikolaj Boc
592494e2e5 Use module.exports to get the entry function name in WASM test lib
Obtaining entry function name with exports = {} is about to get removed
(in emscripten 3.1.44). Use the only alternative, which is to specify
the module = {} object that gets the 'exports' variable. Oddly enough,
it gets assigned the only export name, even though the name is plural.

Change-Id: Idcda29bfcaed2d0a923a8d39af078359abc73f7d
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2023-08-02 15:43:45 +02:00
Morten Sørvig
be7b748b7e wasm: add shared library preload scripts
Add scripts which generates Qt plugins and QML imports
preload lists, for use with the "preload" functionality
form qtloader.js.

The preload lists downlad plugins and imports from $QTDIR/
to /qt/ at application load time, where $QTDIR is configurable
using the qt.qtdir qtloader configuration property (set
to "qt" by default).

Sample directory structure:
  app.html
  app.js
  qtloader.js
  qt_plugins.json     [generated]
  qt_qml_imports.json [generated]
  qt -> /path/to/qt   [symlink]

The json files are generated by the scripts in this commit.
app.html configures qtloader.js to use the json files
as preload lists, which instructs it to preload from
"qt", which again is a symlink to or a copy of the Qt
installation.

Pick-to: 6.6
Task-number: QTBUG-63925
Change-Id: I53bd197f22057dbb70e9a9bee43b9d9b969aa072
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2023-07-03 19:02:48 +02:00
Mikolaj Boc
1f6cac0da9 Make WASM export names different across modules
The export name is now ${TARGET_NAME}Entry. This can also be overridden
by using QT_WASM_EXPORT_NAME, both in CMake and qmake

Change-Id: I59c97ae6e22f0b2720716e9d7eff7b6b13d37ab5
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2023-06-20 09:08:03 +02:00
Mikolaj Boc
d552558170 Allow filtering of test functions to run in WASM testrunner
Just proxy any non-standard arg to the WASM module and let it use
the arg as a filter

Change-Id: Ia77a30ca872497425017edb8fa8961b6d687ca68
Reviewed-by: Michal Klocek <michal.klocek@qt.io>
2023-01-30 13:55:58 +01:00
Mikolaj Boc
025659a18c wasm testrunner: Join output into batches
Sending each line in separate POST requests seems to clog emrun.
Join the output in 300ms intervals so that the number of requests is
limited to 3,(3) per second.

Fixes: QTBUG-109827
Change-Id: I099e8cc14d9d162c54b7040b62297d1070b234c3
Reviewed-by: Michal Klocek <michal.klocek@qt.io>
2023-01-24 08:34:42 +01:00
Mikolaj Boc
8745a73c4f Remove the instance.ENV assignment in js test runner
The assignment crashes with 3.1.25 as instance.ENV is now a read-only
property. This used to be assigned to an empty object, so it was
useless anyway.

Change-Id: I47d4cd831052de514c372adb3f2202d8a0a6e06f
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2023-01-12 17:14:11 +00:00
Mikolaj Boc
b9887d51c3 Provide visual output in page in WASM test runner
There will now be a visual output in page if the qvisualoutput query
parameter is supplied. This simplifies debugging.

The main html resource has been renamed test_batch.html to reflect
the name of the actual test unit, not functionality.

Change-Id: Ib6cd4712de9c47cfcc5f670e7b34f998858f99b7
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-10-05 00:36:41 +02:00
Mikolaj Boc
bafbffb033 Provide the wasm module correctly to the instantiateWasm callback
The second parameter to the onDone callback in istantiateWasm was
missing in qwasmjsrunner.js, which prevented the wasm module from
working on the threaded qt build.

Change-Id: I5d1be7a2e0d8043112f304b4d2530acdaae7b398
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-10-01 00:31:17 +00:00
Mikolaj Boc
7dbbe0a222 Adapt the js batched test runner to emrun interface
This makes the js batched test runner cooperate with emrun.
The runner sends back the output and exit messages to emrun to inform it
about the test execution state.
The code is based on emrun_postjs.js from the emsdk.

Change-Id: I758f2c185797f4000810eb4314423eebc1c5d457
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-09-12 22:28:10 +02:00
Mikolaj Boc
9e05c9ad86 Add error message & format selection capability to batched test runner
Change-Id: I6686bf951204672c3542148e85e59e14e83e73c4
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-09-01 17:29:30 +02:00
Mikolaj Boc
3c9a12934e Implement new features in qt-wasmtestrunner
The script is now able to:
- kill a test that is not responding after certain timeout
- multicast output to files/streams (tee-like, but not only on posix)
- forward requested format (xml/txt/etc.) to the test executable
- run a batched test from a batch test package
- interop with the js batched test runner

Change-Id: Ia189d78a078e11b9efd25865c5a0ddc6a62d9b85
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-08-31 12:47:40 +02:00
Mikolaj Boc
ad1980cd43 Create a driver for running batched tests on WASM
A driver application has been prepared in js for running batched tests.
There is a convenient public API defined for reading the current test
status & subscribing to changes thereof.
The solution is modular - the module qwasmjsruntime can be used for any
wasm instantiation, e.g. in the next iteration of qtloader.

Change-Id: I00df88188c46a42f86d431285ca96d60d89b3f05
Pick-to: 6.4
Reviewed-by: David Skoland <david.skoland@qt.io>
2022-08-24 19:08:58 +02:00
David Skoland
fd4f218c1e Expose the qtloader object globally
When testing, we need to query the state of the Qt application,
so change the scope of qtloader from inside the init function
to global scope.

Additionally, adjust the test script accordingly to query and use this
state to make good decisions on how to terminate.

Change-Id: I6264ba20843716eb87340b160680617b718f6bd9
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-06-03 05:30:46 +02:00
Lucie Gérard
05fc3aef53 Use SPDX license identifiers
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.

Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-05-16 16:37:38 +02:00
David Skoland
b01ee1d44c Add WASM testrunner
Add a python script that allows us to run wasm tests in CI, along with
the necessary cmake logic to install the script and execute tests
accordingly.

Change-Id: I93b95c115538c4e27b2b833405acab8162be2a8a
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
2022-03-31 16:50:25 +02:00
Morten Johan Sørvig
d3646d2b05 Say hello to qtwasmserver
Development web server for web applications. Supports
http and https. Sets COOP and COEP headers.

The web server script supports certificate generation using
mkcert (github.com/FiloSottile/mkcert). Briefly, mkcert
supports generating server certificates for the current
local ip address(es), using a certificate authority
which can be installed on devices where the app should
run.

The COOP and COEP headers are required to enable the
SharedArrayBuffer API, which is required for Emscripten’s
pthreads implementation.

The server serves the current directory on localhost
by default. Use the “-a” argument to bind to an additional
address, or "--all” to bind to all IPv4 addresses found on
local network interfaces. Change the directory by passing
it as a positional argument.

Task-number: QTBUG-79087
Change-Id: Id0cba649e42af53ed8106e336a77e78398bcf901
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: David Skoland <david.skoland@qt.io>
2021-12-06 16:36:09 +01:00