Note this change has no impact on it own on current integrations,
as it barely adds tests compilation.
It is required for qtwebengine on WoA story.
Do not run some tests:
(required for qt5 patch introducing woa test node)
* cmake test as host tools are provided by mingw,
but tests are run natively by WoA platform.
* dbus as this does not run natively on Woa anyway
due to missing dbus service.
Change-Id: Ifdb3e1b76a7feb906a9031815984c716618e3882
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Running many tests on the VxWorks in the emulator without
restarting it between can cause exhaustion of some
resources and cause an RTP spawn fail in some tests.
This change will restart the emulator in such cases, which
allows the test to try again on fresh boot.
Pick-to: 6.9
Change-Id: I5ad715f243867d43b055271d6bc61ca4ccb3089b
Reviewed-by: Simo Fält <simo.falt@qt.io>
Reviewed-by: Michał Łoś <michal.los@siili.com>
Cloning the git sources will:
- allow embedding the git sha1 into the SBOM generated by the build
system
- ease development process in CI VMs when investigating issues.
This will slightly slow down the CI build phase by a minute or two.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTQAINFRA-6520
Change-Id: If554ed0fedca26ff77d04752288ee8497bc4b9ce
Reviewed-by: Toni Saario <toni.saario@qt.io>
This includes:
- turning VERIFY_SOURCE_SBOM ON
- adding exception to the licenseRule.json files
- correcting the licensing given via REUSE.toml files
- renaming license files not located in LICENSES folder.
They need to be named LICENSE. to be ignored by reuse and
excluded from the source SBOM. The name are updated in the
corresponding qt_attribution.json
A lot of files are skipped during the license test,
but all are present in the source SBOM.
This is why correction are needed before turning the
source SBOM check on.
[ChangeLog][Third-Party Code] Renaming the license files with prefix
LICENSE. to have them ignored by reuse tool.
Task-number: QTBUG-131434
Pick-to: 6.9
Change-Id: Iab517215bb10a17357d2d2436bba8d3af76e5cd1
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The license check is only called if VERIFY_SOURCE_SBOM is set to ON
in the module_config.yaml. This way, a module can turn on the check
only when ready.
The license check is done via a perl script. Since no perl is
available for Windows on ARM64, VERIFY_SOURCE_SBOM is set to OFF
in this case.
Task-number: QTBUG-131434
Pick-to: 6.9
Change-Id: I183b7726928bbc925940330d39779a4ac2307254
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Axivion analysis for qt5 is currently ran only for linux and
qtactiveqt started to fail because it now uploads aftifacts also
for linux and is not skipped anymore. Adding condition to prevent
running Axvion analysis for qtactiveqt.
Task-number: QTQAINFRA-6752
Change-Id: I2ce141a1b7c6de387493651c4429edac65e49506
Reviewed-by: Toni Saario <toni.saario@qt.io>
Unless a module overrides it by setting the environment variable
COIN_CTEST_FORCE_IGNORE_EXIT_CODE.
As this change trickles through submodule updates we'll mark modules
as COIN_CTEST_FORCE_IGNORE_EXIT_CODE if needed.
Change-Id: Ia37d6e4c4dee41a3dd7446fa23ab96593dab9402
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If COIN_CTEST_IGNORE_EXIT_CODE is in effect and ctest aborts for some
reason (for example due to the workitem being cancelled) the ctest exit
code will be ignored, as expected, but we will not have a LastTest.log
to copy, which will fail the workitem unexpectedly.
We now match the ignoreExitCode of the LastTest.log copy with the ctest
run.
Pick-to: 6.9
Change-Id: I9df0f863a42dd4cf25cee1694e85cb32058a4e5b
Reviewed-by: Toni Saario <toni.saario@qt.io>
Add crash handling to avoid waiting until 10min timeout on each crash.
Force restart on each crash by killing emulator.
Make health check more robust to allow it to work even when emulator is
down. This is done via wait in separate process which cannot be done
with normal timeout command, as input to pipe that is not being read
by emulator will block forever.
Pick-to: 6.9
Change-Id: I86c3c86f936cc96d57b38983da48d1d73162399d
Reviewed-by: Simo Fält <simo.falt@qt.io>
Increases to around 16MB with the format change.
Required by some tests.
Pick-to: 6.9
Change-Id: Ia41436b4269220f84271b614ea6f2b96ca605c32
Reviewed-by: Simo Fält <simo.falt@qt.io>
Startup the correct emulator for each arch.
Add a health check for emulator during test start. If emulator
does not respond, restart it. Occurs in declarative tests where
tests hang and crash. After the crashes the pipes also need to be
reset, only emulator reset is not sufficient.
Read variables from WIND_CC_SYSROOT which is the VSB path, so
correct VSB is used.
Change-Id: If918ee45d4fa3a452c484a48fe9507f12f788e86
Reviewed-by: Simo Fält <simo.falt@qt.io>
This reverts commit 8d5e7d50d8dbf1ad79bd8ff9f6ef6028eba481c9.
Reason for revert: Arm packages should be sign-able now.
Pick-to: 6.8
Change-Id: I6a896bfd84927da9c15647d207a6ad5024a27949
Reviewed-by: Jani Heikkinen <jani.heikkinen@qt.io>
Backstory.
The current approach for building documentation in the CI is that we
build a static qdoc in a Jenkins job, manually provision it in
qt5.git, and then build qt documentation using that somewhat
old qdoc version.
Updating the provisioned qdoc is a very slow process, because changes
first need to be merged into qttools, then a Jenkins job needs to be
manually run, then provisioning needs to be updated, and sometimes it
also depends on a full qt5.git submodule update or has to wait for
other unrelated changes to qt5.git.
The new approach.
This change adds CI instructions to build a static qdoc (and
other relevant tools necessary for documentation generation)
using the latest qttools/dev/HEAD sha1 in a specific host platform's
qtbase build instructions, and then uses those tools for documentation
generation and doc warning checks in each repo's TestDocs CI test
phase.
To enable these new instructions, the host platform and the doc
checking platform need to be tagged with the 'DocsGenerationV2'
feature.
The built tools are cached in a separate archive using a new Coin
feature, which is then extracted during each repo's TestDocs phase.
Pros and cons.
The benefit of this new approach is that we will always use a mostly
up-to-date qdoc from qttools/dev/HEAD when generating documentation,
making the documentation team's life easier.
Specifically, once the doc tools are built in qtbase with the latest
qttools/dev/HEAD sha1 that was available during that integration, the
resulting built tools are then used for doc checks in dependent repos.
A newer qttools/dev/HEAD sha1 will only be used for a particular
repo once the qtbase dependencies.yaml sha1 is updated, or when doing
qt5.git changes that cause a rebuild of qtbase.
Note that while new tools are built due to dependency updates, the
used qttools sha1 is not recorded in any way in any of the
dependencies.yaml files.
qtbase doc checks will always use the latest qttools/dev/HEAD sha1 at
the integration time, because the tools will always be built from
scratch.
The downside of the new approach is that, by default, the doc
generation will not use a pinned version of qdoc anymore,
but rather follow the flow described above. This has a chance of
introducing failures which are unrelated to integrating changes.
This should happen rarely because the doc team usually tries to fix
doc warnings before changing qdoc's code.
If temporary pinning of qttools or qt5.git is required when testing the
docs of a specific repo, it can be achieved as follows:
- set the QT_CI_BUILD_REPO_DOC_TOOLS env var to "1" in the repo
coin/module_config.yaml file build instructions, to ensure the doc
tools are built in that repo, rather than reuse the ones built in
qtbase
- set the QT_CI_DOC_TOOLS_PIN_GIT_REF env var to the desired qttools
sha1 or other git ref, in the same build instructions phase
- optionally set the QT_CI_DOC_TOOLS_TOP_LEVEL_PIN_GIT_REF env var to
the desired qt5.git sha1 or other git ref, in the same build
instructions phase, if a different version of qt5.git is required
- optionally set the QT_CI_DOC_TOOLS_USE_CI_TOP_LEVEL_BRANCH env var
to ON, in case if the module branch that the CI job computes should
be used as the qt5.git branch
- set the QT_CI_FETCH_REPO_DOC_TOOLS env var to "1" in the repo test
instructions, to ensure the just built doc tools are fetched instead
of the qtbase-built ones
Test running this new approach will allow us to collect some feedback
on how often breakages happen, and how much easier it makes the
documentation team's work, or how much harder it makes the life of
regular integrations.
The abundant amount of optional pinning options should be enough to
avoid any permanent integration dead locks.
Pick-to: 6.8
Task-number: QTBUG-128730
Change-Id: I8606cb3076036a4a0ec652d0fa74d270e8f5dfdf
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Add a new coin_test_docs_common.yaml file that contains the common
instructions for configuring a documentation warning build.
Pick-to: 6.8
Task-number: QTBUG-128730
Change-Id: Ifc86570f3c98e64364f3c2c2bb71f45eaaee0c19
Reviewed-by: Toni Saario <toni.saario@qt.io>
This makes tqtc/ branches to use the public branches.
Amends 2a0e89981a52633c497f62bad0c7d26c466493cb
Pick-to: 6.8
Task-number: QTBUG-124453
Task-number: QTBUG-125211
Task-number: QTBUG-125569
Task-number: QTQAINFRA-3935
Change-Id: I1098165c3122befabc503d38c46d476920d835a2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Simo Fält <simo.falt@qt.io>
Coin can now dynamically refer to the current branch of the repo being
integrated in the module instructions.
Change the license checking instructions to use the current branch
of the qtqa repo, rather than the dev branch.
This makes the behavior consistent with what everyone expects, instead
of using a hardcoded qtqa branch for all stable branches.
Amends 2a0e89981a52633c497f62bad0c7d26c466493cb
Pick-to: 6.8
Task-number: COIN-573
Task-number: QTQAINFRA-3935
Task-number: QTQAINFRA-3878
Task-number: QTQAINFRA-4975
Change-Id: I6575c2868f3a7170570803f620feb5723fbbe6df
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The Qt default has changed to always generate an SBOM for all
platforms in the CI, so we always want to pass the location of the
python interpreter meant for SBOM processing to configure.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I1042c97b9133ca37e42eb3efd1689ec1f9f9ab31
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
It was removed in a previous change and does nothing now.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I7a84c243c08a65e445cbd657eb281b37e68f1893
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
If the tests fail we still want to upload the full CTest log.
Change-Id: Ib1c95d2b0b3498b05e335aeeb487190a747d162e
Reviewed-by: Toni Saario <toni.saario@qt.io>
This environment variable might be set by some party (not quite sure
what, maybe the host OS), and when androidtestrunner passes QT_* vars
it might interfere with the input tests.
Pick-to: 6.8
Change-Id: I2a88c78535d6e3a809e8fcee8f8d752f91044a8c
Reviewed-by: Toni Saario <toni.saario@qt.io>
Now that the ctest is not run as verbose the logging is significantly
decreased and VxWorks will hit timeout on some tests.
There are plenty of other timeouts that will be hit instead
before the total timeout specified for the command e.g.
- Ctest timeout 25min default per test case.
- Testlib timeout per test function 10min default.
Change-Id: Ic02e812d4b059609553d2ac8d44ec91f358d76c5
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
This stops confusing the 'reuse' tool when trying to generate a source
SBOM. The tool considers all files in the source dir as source files,
and because we placed the build dir inside the source dir, all build
files were considered source files. This caused errors saying the
files don't have copyright headers.
Change the build directory for cross-compilation jobs to be outside of
the source directory, similar how we do it for the native builds.
Augments ae62d908a5945900b11451d694e0c784e635b987
Pick-to: 6.8
Task-number: QTBUG-82820
Task-number: QTBUG-96513
Task-number: QTBUG-122899
Task-number: QTBUG-124453
Task-number: QTBUG-125211
Change-Id: Ie82a6c55aa5326f7e9cc5c0e09342f3405e78d17
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
We don't build a host Qt during a cross-compilation build for a few
years now. Instead we reuse the prebuilt host from the platform
dependencies. This was left over to allow gradual porting to the new
way of using platform dependencies, which is now long done.
Remove the code to ease maintenance.
Amends 684fea131433dcbb6d24feaf261404d1d9cf69b4
Pick-to: 6.8
Change-Id: I9988da6f33f9d9b98d304316317d39c3dfb7c633
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
It's mostly redundant, as ctest already prints timing information
for each individual test:
1/1 Test #572: tst_qwindowcontainer ............. Passed 0.71 sec
And we track test timing though the JUnit and Qt XML files we produce
and ship to our Grafana dashboard.
Change-Id: I374d0b2201efabccec18da6358e3859b02702c27
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
The test output of passing tests is most of the time unnecessary
noise when investigating a test failure in a failing test. With
the size of the qtbase log being 25MB uncompressed, working with
these logs locally or in a (mobile) browser is also painful.
We now disable verbose test output in ctest, and only output test
output if the test fails. We still run our tests with -o foo.txt,txt
and these files are uploaded to coin's artifact storage, so the
full logs are available if closer inspections are needed. We
also store CMake's LastTest.log, which include verbose output
of all non-testlib tests.
The verboseness of the coin config can also be enabled via the
VerboseCTestOutput feature, either for specific configs, or
on demand when scheduling a check via coin.
Fixes: QTQAINFRA-6592
Fixes: QTBUG-109441
Change-Id: I5cdfb8839190b1598f12b7f9ea1d9a242a32a790
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Daniel Smith <daniel.smith@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
The ASAN configuration runs on OpenSUSE and has the variable
CMAKE_MIN_SUPPORTED_BIN_PATH=/opt/cmake-3.16.8/bin
because in the non-ASAN configuration it prepends this to PATH to test
functionality with old CMake.
In the ASAN configuration the above variable is still set, but it's not
prepended to PATH, so the latest CMake is used. We now detect that
properly and enable junit XML summary files.
Change-Id: I75f7308b9332434853cbe2a9878f7b4c192710e7
Reviewed-by: Toni Saario <toni.saario@qt.io>
Match to Windows target OS and different target arch, which tells that
it is a cross-compilation.
Pick-to: 6.8
Change-Id: Iba2d087cda501cd2622a1bd8a8732100b920956b
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
So that we don't rely on building/running Qt auto tests just to
build the docs.
Change-Id: If3939717c57e8d4074afe0d611a1d04e65da7d36
Reviewed-by: Toni Saario <toni.saario@qt.io>
Some tests like Qtlanguageserver do some building during tests and
require access to build tools.
Change-Id: I3a8e4fba6e59bd974794edc157a33ffec8dd6cb3
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Some build configurations might have their own instructions for
the test work item, where it doesn't make sense to run the Qt
auto tests.
We can't use DisableTests for this, as that is hard-coded in coin
to skip the entire test work item.
Nor can we use DoNotBuildTests, as the test config might not be
building anything (depends on another build config, that does build
the tests).
Nor can we use DoNotRunTests, as that is hard-coded in coin
to skip the entire work item. We should probably fix that,
but until then let's use a dedicated feature to selectively
skip running the tests while still creating the test work item.
By using this feature we can for example let the documentation
warning configuration build docs and check for new warnings,
without also running Qt auto tests in the same test run.
Change-Id: I108bece654c811e98ba06e07e4e7b06f6d84c8f4
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
This allows to run tests larger in size where the test transfer
becomes an issue.
Change-Id: Ic8647f9c88233f4fcff15c350540e00dfb642b47
Reviewed-by: Simo Fält <simo.falt@qt.io>
This can be used to configure extra cmake args for the documentation
build in coin.
Pick-to: 6.8
Change-Id: I11417bee9630aebe0bbc118cd843b1536101da71
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
When starting QNX emulator this makes VM to try to publickey first.
This avoids VM from trying to use gssapi-with-mic authentication,
which in some cases hangs indefinitely causing a timeout.
Pick-to: 6.8 6.7 6.5
Change-Id: I499c668a910851d97ce8de803a1064dcd7b0b74d
Reviewed-by: Jukka Jokiniva <jukka.jokiniva@qt.io>
Add yaml instructions to run the license checker from the qtqa dev
branch, when the LicenseCheckV2 feature is enabled.
This is intended to replace the old legacy instructions that are
hard coded into Coin, that are activated via the 'LicenseCheck' and
'UseLegacyInstructions' features.
We can't reuse the old name and remove the 'UseLegacyInstructions'
feature, because Coin expects license checking instructions to be
added to a separate 'LicenseCheck' phase in the yaml file,
and doesn't allow doing it in the 'Test' phase.
A platform config should enable both the LicenseCheckV2 and TestOnly
features, to ensure the license checking doesn't wait for the build to
finish before running.
The instructions are added to
coin_module_test_template_v3.yaml and
coin_module_test_template_v2.yaml
so that we don't have to add them manually to all repo's
module_config.yaml file, but rather reuse the existing yaml files.
We add the instructions to the v2 file as well, because not all repos
have been ported to v3, like qtmqtt, which is not ported on purpose,
because tests don't pass there currently.
Note that so far, all license checking was done using the qtqa master
branch, not the dev branch. The dev branch currently has more changes
than master (many changes were not cherry-picked). That means that the
check might fail in some repos, and thus have to be fixed.
Pick-to: 6.8
Task-number: QTBUG-124453
Task-number: QTBUG-125211
Task-number: QTBUG-125569
Task-number: QTQAINFRA-3935
Change-Id: I715b99853aa041cb036d6b61b071746f010565d0
Reviewed-by: Toni Saario <toni.saario@qt.io>
Keeping DESTDIR set to "{{.InstallRoot}}" after installation has
potential to break other unintended CMake file(INSTALL) calls that
take DESTDIR into account.
This already happened for the CMake macOS POST_BUILD deployment API,
which accidentally installed into the install root, when it shouldn't
have.
Unset the DESTDIR env var after we install Qt.
We already do that in the cross-compiling Coin instructions, so this
brings uniformity to the host builds as well.
Pick-to: 6.8
Task-number: QTBUG-90820
Task-number: QTBUG-96232
Change-Id: I6aca31e36c67c8d4b293efd746c37a42ea9ca834
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This will enable generation and linting of source SBOMs only if the
repository source root contains a REUSE.toml file. Otherwise the steps
will be skipped.
Pick-to: 6.8
Task-number: QTBUG-122899
Task-number: QTBUG-125211
Change-Id: I87ea9aad7fb4f15ec7fa9d00072c81cddbe7ea2c
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Dividing qtbase into essentials and add-ons modules in analysis.
Task-number: QTQAINFRA-5965
Change-Id: I9b923939ced35ee72cb62fb8d62e486f38a8d709
Reviewed-by: Jukka Jokiniva <jukka.jokiniva@qt.io>
If the VerifySBOM feature is enabled, pass additional options to
CMake configuration:
- runs syntactic verification of the SBOM
- so it converts the generated SBOM files into JSON files (and
installs them). This does additional useful validations.
- runs the NTIA SBOM verifier
- shows some of the content in the SBOM in a more user-friendly table
format via the sbom2doc python app
- runs sbomaudit to show things like packages without a license
expression, doesn't exit with errors if issues are found.
To ensure the tables are wide enough in the log on the CI, we set
an explicit value for COLUMNS env var, which is used by sbom2doc
to determine the table size.
To ensure the sbom2doc and sbomaudit python applications are found,
we supply additional locations where they can be found, via the
env vars that coin python provisioning sets.
We also make sure to pass the found application paths when executing
the python apps, because they might not be in PATH by default.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I0baef8b9c949209b15ab304e1e840b4dcdf5a61c
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The use case is where blocking test building is wanted but they cannot
be uploaded due to being static or they would simply be unused.
Pick-to: 6.7 6.8
Change-Id: I32ad20fea2ec032fb1f1a9b86025b34f192e0ed2
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The instructions will only run if a coin platform config has the
'GenerateSBOM' feature.
The -sbom configure option is not sticky across repos,
similar to -headersclean, so that if a user wants to build another
qt repo against a prebuilt qt, it won't try to generate the sbom
automatically.
That's why we append -sbom to all the repos configure args, and not
just the qtbase one.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: Ice91cbc430826838119ce3085cbd52c0909f2746
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
The last 30 days shows that these configs are stable, so we can
make them significant:
http://testresults.qt.io/grafana/goto/jBYi1JsSR?orgId=1
The tst_qheif::writeImage test has been blacklisted separately.
Pick-to: 6.8
Change-Id: Ibe7981bce2db1ec90781d3ea33fe7d4541990aea
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
By selectively setting the COIN_CTEST_IGNORE_EXIT_CODE we can allow
modules to gradually become significant, without touching qt5.git
Change-Id: Ie5b6e683cd755b1cf46886bb30803a56885a11a6
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
This deduplicates some of the coin instruction files, and opens up the
possibility for other instructions to determine whether a test run
is significant or not.
The only module still using the coin_module_test_template_v2 template
is the QtMQTT module, so the behavior of ignore the test run has been
kept.
The static library opt out was a remnant from the cmake porting, and
can be removed.
Change-Id: I0f9fcbbd3a5a18bd9c8e0f545b0f0656e03bf388
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
