Blacklisting items on the baseline server would avoid any mismatch
failures for such items. However, they would still be rendered. That
is undesirable if the rendering itself has unwanted side effects for
such items, e.g. crashing the test executable.
Fix by adding new functionality in the baseline testing framework to
allow checking blacklisting status ahead of time, and add a new macro
QBASELINE_SKIP_IF_BLACKLISTED to do just that.
Add usage of that macro to the QPainter baseline test.
Pick-to: 6.7 6.5 6.2
Change-Id: I35f6df8cff2c6cb985c25ab5470cd42b53d44940
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
If a commit adds new baseline test items, they would not be reported
to the server when doing a LanceBot test on that commit, and so not be
visible in the report either. This commit fixes the client side of
that issue; the server is already updated.
Change-Id: I60c2958def5e7b54ddc2789cf5727edcbb1a90e1
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
Reviewed-by: Jonas Karlsson <jonas.karlsson@qt.io>
According to QUIP-18 [1], all tests file should be
LicenseRef-Qt-Commercial OR GPL-3.0-only
[1]: https://contribute.qt-project.org/quips/18
Pick-to: 6.7
Task-number: QTBUG-121787
Change-Id: I9657df5d660820e56c96d511ea49d321c54682e8
Reviewed-by: Christian Ehrlicher <ch.ehrlicher@gmx.de>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Remove some kludgy, redundant and never used functionality for setting
project and test case names, as it also hindered overriding those
properties at runtime.
Pick-to: 6.3 6.2
Change-Id: Ibef7d7d0cb5fc1e462752f2ba2db76cc088dbd48
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Adds a function for setting the baseline Project name, which selects
the top level server directory and hence config settings.
Also adds a function that lets the client override the server config
setting for the project image keys (ItemPathKeys). This is the list of
properties that will be used to determine which clients are "the
same", i.e. which baseline to compare against. Overriding that is
handy since it allows experimentation and customization of clients
without involving changes in the project config on the baseline
server.
Pick-to: 6.3 6.2
Change-Id: Id3998356494a9a2cb71c009b43593d3dc1b6963a
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Merge in various minor changes and fixes that have been done to the
branched copy in qtquick3d, and clean out some outdated code.
Mostly just coding style fixes and cleanups, but also:
- adds -keeprunning command line parameter, intended for tests that by
default exits early if it seems the platform is too unstable
(e.g. crashing) for a meaningful testrun.
- Changes behaviour for fuzzy matches, from SKIP to PASS. The (mis)use
of QSKIP was done to force log output; now the output is just
printed by qInfo() instead.
Pick-to: 6.2
Change-Id: I46e77a94cc5b1980ac420086c2ae88dc9b84ef12
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
In preparation for addition of new baseline tests, establish a new
test category, "baseline". This is similar to the category
"benchmarks" in that it contains tests that use the QTest framework,
but conceptually are not unit tests, in contrast to those under auto/.
Move the existing QPainter baseline test, tst_lancelot, into this new
category, and rename it accordingly.
Baseline tests use the QBaselineTest extension to QTest. Move that
extension too into the tests/baseline directory, allowing the clean
out of the baselineserver directory.
Pick-to: 6.2
Change-Id: I1b527f5867c953b1d22be73798fcf7d1494712ea
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Eirik Aavitsland <eirik.aavitsland@qt.io>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
