Required for porting away from QLatin1Char/QLatin1String in scope of
QTBUG-98434.
Change-Id: I3debae5f481037958bfb65caabca97a0d4681eb9
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
In the same ways as for blacklisting. In the process, move reporting
of an unrecognised tag outside the global-data loop, as the tag might
be, or start with, a global data tag; and also report the global data
alongside the test-specific data. Indent the lines reporting tags, as
this makes the structure of the lists more evident. Converted the tag
argument to QLatin1String() to facilitate searching and matching.
Updated documentation, including expanding on the option of specifying
the data tags on the command-line. Drive-by: fixed a link in the
documentation, removed a snippet that was nowhere used.
[ChangeLog][QtTest] When specifying what tests to run on a QtTest
command-line, it is now possible to include a global data tag in the
same way as a function-specific data tag or in combination with it.
Thus if the test reports itself as tst_Class::function(global:local),
command-line option function:global:local will select that specific
test and function:global will run all data-rows for the function on
the given global data tag.
Fixes: QTBUG-102269
Change-Id: I7c86d6026933b05f7994bfc3663a2ea6a47f5f38
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Separated from other clean-ups to avoid inanity 'bot complaints.
Change-Id: Ia33f87da224dba7c50bbf14beb642825dc46fe90
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Now that QL1S has its own arg(), we can use that directly instead of
going via QString before calling arg(). Only works in most cases, as
the QL1S::arg() overload set is more limited.
Change-Id: Ie4c566c5998538b1b8a953b1ba4481e0f1663bf7
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Previously the blacklisting file format only worked with slot:data or
plain slot names for the items to blacklist. However, tests with
global data report themselves with the global data-row tag in the same
way as function-specific ones do; and tests which have both join the
two as slot(global:data) in the test output name, so the reader is apt
to mistake global:data for a data tag. In any case, it is potentially
desirable to be able to blacklist a function with either or both of
global and local data-row tags specified. Add support for that and
remove a blacklisting that was only needed due to the lack of this
support.
For now, make the new parameter to checkBlackLists() optional, so
that qtdeclarative's qmltest framework can adapt to this change.
Fixes: QTBUG-100870
Change-Id: I9125811ebdab75d3fb462ba8b60561f003426502
Reviewed-by: Pasi Petäjäjärvi <pasi.petajajarvi@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
By changing it to unique_ptr.
Pick-to: 6.2 6.3
Change-Id: I91abb69445b537d4c95983ae735341882352b29d
Reviewed-by: Marc Mutz <marc.mutz@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
We don't currently have a use for the extra parameters, but Samuel
Mira's fix to ensure we don't miss fatal signals on Android needs to
be able to call the prior handler, which may need these parameters.
In the process, check returns from sigaction() and use its nullptr
semantics to query first when detecting whether to restore the default
when done, rather than setting and then restoring if it turns out
we've been replaced in the meantime.
Task-number: QTBUG-97652
Pick-to: 6.2 6.3
Change-Id: If30a0db35946c3dceb409ae3f41aa437149472e6
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
If the CSR_ALLOW_UNRESTRICTED_FS bit of System Integrity Protection (SIP)
is enabled lldb will fail to print a valid stack trace when launched from
the crashed process, and might also resulting in hanging or crashing the
parent process (Terminal e.g.):
https://github.com/llvm/llvm-project/issues/53254
We detect this situation and avoid printing a stack trace if so.
Pick-to: 6.3 6.2
Change-Id: Iad8cab5fcdc545d810ca4d4e985aefc0988d0234
Reviewed-by: Timur Pocheptsov <timur.pocheptsov@qt.io>
Added tests for repeated skips and failures (from within void lambdas,
to simulate skips and failures from within event handlers). These
exhibit yet more ways to count more than one outcome for a test. The
new QTest::failOnWarning() can also provoke more than one failure from
a single test, and several existing selftests exhibited various ways
for the Totals line's counts to add up to more than the number of
actual tests run.
Fixed counting so that only the first decisive incident is counted.
Tests can still report later failure or skipping, but only the first
is counted.
Added a currentTestState in qtestlog.cpp, by which it keeps track of
whether the test has resolved to a result, and clearCurrentTestState()
by which other code can reset that at the end of each test. This
brought to light various places where test-end clean-up was not being
handled - due to failure or skipping in a *_data() method or init, or
a skip in cleanup.
Fixes: QTBUG-95661
Change-Id: I5d24a37a53d3db225fa602649d8aad8f5ed6c1ad
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Invert a condition to turn a nested if (that was missing one of its
layers of braces) into an else-if chain, split some long strings to
limit line length.
Change-Id: I10d90487a09affe981aa11c3588281aeb3666df5
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
When running a test using -callgrind, we recurse into a child process,
run under valgrind, to which we pass -callgrindchild; and we only want
the output from the child process, since the parent won't actually be
running any tests. We also won't be using the global data table for
the test in the parent process. So bypass the set-up and tear-down of
both logging and the global data table in the parent process.
Prior to commit 3ee6d8d336db2d9d15818b234ce16531ea0cdd48, these parts
of the set-up and tear-down were skipped in the callgrind parent
process, but that refactoring split qExec() up into qInit(), qRun()
and qCleanup() to enable QtQuick to recombine them to implement an
equivalent of qExec(), calling qRun() once for each built-in style. It
needs these pieces of set-up to happen in qInit(), and of tear-down in
qCleanup(), to avoid repeating them for each style. Leave a comment in
qExec() that might help future readers to understand why it's done the
way it is.
Change-Id: Ieaca9a125c713b8fcf8dec8f9be0c024a798d504
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Extract Method QTest::qCaught() to take the string handling out of the
header. This should help a bit in speeding up compilation of large
unit test files (provided they use QVERIFY_THROWS_EXCEPTION), although
I have no data to support that.
Since we changed the error message, update the selftest accordingly.
Change-Id: Id4a3c8c34d5df8d0c7a861106d269097f4a6de5c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
This solves the long-standing problem of not being able to easily
fail a test when a certain warning is output.
[ChangeLog][QtTest] Added QTest::failOnWarning. When called in a test
function, any warning that matches the given pattern will cause a test
failure. The test will continue execution when a failure is added.
All patterns are cleared at the end of each test function.
Fixes: QTBUG-70029
Change-Id: I5763f8d4acf1cee8178be43a503619fbfb0f4f36
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The stack needs to be at least SIGSTKSZ, which isn't constexpr, so we
can't allocate it at compile time. However, we can resize(SIGSTKSZ) a
QVarLengthArray that's probably big enough anyway. At the same time,
increase the compile-time size to 32k, to match what our Catch2
harness for the self-test uses.
Change-Id: I3a34ece73901dd402672cd6fe4da66923f1932c8
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Shrinks the const array by one zero-terminator and makes the code
accessing it read more gracefully.
Change-Id: I4034116a83ff3cd05ea0feb0ce8a4340c54a9faa
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
If we fail to install our own action, the contents of oldact are noise
anyway, so we can't use them to decide anything; and, in any case, we
didn't replace the prior action, so have no duty to restore it.
In the process, restructure and comment the code to make it easier to
follow what's going on and why. Package a conditional in a lambda to
make the #if-ery less problematic. This should also make it easier to
hack any other complications into the condition, should they arise.
Change-Id: I712335ee27f68a8594dc2fe6441a91f686967da2
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Previously, the loggers would leak if the application failed to call
stopLogging(). Now they are owned by the global static which will delete
them in that case.
Also, since we have to adapt loggerCount() to the fact that std::vector
uses size_t, recognize that we only ever want to know whether the number
of loggers is 0. Change the method to only provide that information
rather than the actual number.
Change-Id: Ieb2e185048d573ec7f36373ad49bb2a0ca391ce3
Reviewed-by: Mitch Curtis <mitch.curtis@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The one place that actually called it with file and line number was
internal to QtTest, so could call QtTestLog::warn().
Change-Id: I8be10fc81996cc3de4d2eea3ecedc59671ffe759
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Calls to QTestResult::setCurrentTestData() are handled via an RAII
class, QTestDataSetter, that ensures this final call passing nullptr
was redundant.
Change-Id: I7788e3ce898e5628d5590ef9d34182f05e85fe50
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Use \note instead of \b{Note:}, don't refer to functions as macros,
and don't lie. The data-table set-up functions can delegate their
calls to addColumn(), addRow() and newRow() to functions not called
directly by the test framework.
Change-Id: I3bef22cf1a45df91dc822ac04f07e7d15053825f
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
The format was renamed in 27db9e458cef512fca3a6b5c9ebbcda7a8172428.
Pick-to: 6.2
Change-Id: I53975c7467d8768dc9dc9ac2d89c42eefa12e22f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Before this change, next() was the only way to advance the iterator,
whether the caller was ultimately interested in just the filePath()
(good) or not (bad luck, had to call .fileInfo()).
Add a new function, nextFileInfo(), with returns fileInfo() instead.
Incidentally, the returned object has already been constructed as part
of advance()ing the iterator, so the new function is faster than
next() even if the result is ignored, because we're not calculating a
QString result the caller may not be interested in.
Use the new function around the code.
Fix a couple of cases of next(); fileInfo().filePath() (just use
next()'s return value) as a drive-by.
[ChangeLog][QtCore][QDirIterator] Added nextFileInfo(), which is like
next(), but returns fileInfo() instead of filePath().
Change-Id: I601220575961169b44139fc55b9eae6c3197afb4
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Take two. f20edffcc8089fe573e03bd5e092dafc9b0ce568 only disabled the
backtrace preamble.
Pick-to: 6.2
Change-Id: I972d007af59d13ec6eb11da44fc2b5e25c36e46e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
It now uses QtPrivate::condition_variable, it pulls in the correct
header for that, so #include <condition_variable> is no longer needed.
Separate opening braces of function bodies onto next line.
Change-Id: I08f721c4d52756932bb9409e34e51dcbb3eda104
Reviewed-by: Marc Mutz <marc.mutz@kdab.com>
char16_t, as returned from QStringView::utf16(), is guaranteed to be
unsigned, so there's no need to reinterpret_cast to ushort.
Change-Id: I7432dd26b4814c61fe70cc33ed307097ef46ce4e
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
MetaMethods is a std::vector, so we should avoid copying it. Add
strategic std::move()s and pass to the TestFunctions ctor by value,
not cref.
Change-Id: Iaa2879ae427b9603a7abaab0e3015556d9c247fc
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Previously, it went direct to QTestResults::addFailure() without going
via the checking for expected failure. Add QTestResults::fail() to
take care of this checking, as for verify() and compare().
Tidied up the code implementing expected failure and QFAIL(), while I
was about it. Adjusted an existing test to verify that expecting a
QFAIL() works, by using QFAIL() instead of QVERIFY(false).
Remove the QVERIFY(false) whose comment brought this to my attention.
[ChangeLog][QtTestLib][QFAIL] QEXPECT_FAIL() now correctly anticipates
a subsequent QFAIL(). Previously QFAIL() counted as a fail regardless.
Change-Id: Icc28cf70e5ff3006363791ea03aa01f2f591eb71
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Clazy warns about the fromUtf8() call with a constant argument, for
every use of the macro, so hide the stuff behind a compiler firewall.
Also fix the format injection error by using QLatin1String::arg()
instead of QString::arg() chaining.
Change-Id: I4bb4d4af56443540efc0c38c75819aa152a441fc
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Output object name and class in QCOMPARE(). This should help
to debug flaky QWidget tests that for example check on focusWidget().
[ChangeLog][QtTestLib] QCOMPARE() now reports QObject * values by class and objectName().
Task-number: QTBUG-64446
Change-Id: Ife04e89bba04fc78d077c8f0f07af17a17c9cf8c
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
The trick we use by spawning lldb to print the backtrace doesn't work
for some reason, and just results in:
Process 91619 stopped
* thread #1, stop reason = signal SIGSTOP
frame #0: 0xffffffffffffffff
Target 0: (No executable module.) stopped.
Architecture set to: .
(lldb) bt all
* thread #1, stop reason = signal SIGSTOP
frame #0: 0xffffffffffffffff
thread #2
frame #0: 0xffffffffffffffff
thread #3
frame #0: 0xffffffffffffffff
thread #4
frame #0: 0xffffffffffffffff
Debugging the stopped process manually with lldb works fine.
Change-Id: If5582803a11f1c5c84c31b42d1ae52bdd245bbbb
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
After 1ed8a7bff503aacf55f7f880ddaad461ea15e5e1,
volatile needs to be specified for toString(), else it is not
used by compare_ptr_helper(). Add an overload.
Pick-to: 6.1 5.15
Change-Id: I3c335f324df346233623272d1014c8360ca33160
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Not 100% sure if this is a bug in tsan, but turning expected into an
atomic variable will avoid the warnings.
Change-Id: I6d6a5d58f90258d201ae3880473228638de0a215
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Andrei Golubev <andrei.golubev@qt.io>
Allow passing command line arguments such as -AppleLocale, that override
user preferences for the test process.
Change-Id: I9e58e91fcb01a36f9d6c64ef52369308be5e95b5
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Other affected rows have also been fixed．
Change-Id: Ie0a32f724bd2e40e7bfacfaa43a78190b58e4a21
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Add a qCompare() overload for qsizetype, preventing it
falling back to the slow code path pre-formatting expected/actual.
toString() should use the correct format from the already present
int-types (see macro TO_STRING_IMPL).
Complements 94aa350621e8a5c4ad3b438c10fc1c0a9ed3bc8a.
Task-number: QTBUG-38890
Change-Id: Ieb8cea7de086141a2c80f93b4c1be01572be96df
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
The usual behavior for env variables is that any nonzero
value means "enable", so do the same for a couple of QTestLib
env vars. While at it: document them.
Change-Id: I854285df5d7af5be771f9d6532ddec5d8a6f8987
Pick-to: 6.0
Reviewed-by: Paul Wicking <paul.wicking@qt.io>
These look like leftovers (API flaws).
Construction of QFileInfo from QString (or similar) should be not
implicit, as QFileInfo construction is expensive (might hit the file
system), and this may have users overlook APIs (for instance build a
QFileInfo out of QDirIterator::next(), instead of using ::fileInfo();
using QDir::entryList instead of entryInfoList; etc.).
Leave an opt-out mechanism to ease porting.
Fix a handful of usages around qtbase, with at least a couple of them
likely to be actual "sloppy" code.
[ChangeLog][Potentially Source-Incompatible Changes][QFileInfo] Most
QFileInfo constructors are now explicit. The
QT_IMPLICIT_QFILEINFO_CONSTRUCTION macro is provided to keep old code
working.
Change-Id: Ic580e6316e67edbc840aa0c60d98c7aaabaf1af6
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Ensures that the backtrace is interleaved with the header and
footer that we print ourselves.
Change-Id: I728f4a05be31e345687cbb5fefe49f76dbe8ae36
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
As per ### Qt6 comment. Also rename the LibraryLocation enum
to LibraryPath.
Change-Id: I556025a19c5bcdf2ff52598eaba32269522d4128
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
This change partially reverts change
a0e0b51001edfc1c7aea113c472ce995efa833fd.
Replacing the QTest specific sleep function with QThread::msleep()
was not a good idea. The reason is that QThread::msleep() will force
the thread to sleep to x mseconds, even if a signal woke the thread
in the meantime.
This would cause qWaitFor() to not call processEvents(), in some cases,
leading to flakyness and test failures in tests that rely on timing,
such as the animation tests in Qt Qml.
Change-Id: I0ad132cdf32be5813b2e73552d772251fe1d7f89
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Simply use QThread::msleep() to implement it.
Change-Id: I37c255fc70951715edc489d9f67669b01af380b1
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
And remove one of the type id to name mapping that still
existed in QMetaType. QMetaTypeInterface can provide that,
so there's no need to have a second copy of the data.
qMetaTypeTypeInternal() can still map all the names of all
builtin types to ids. That functionality is for now still
required by moc and can't be removed yet.
Change-Id: Ib4f8e9c71e1e7d99d52da9e44477c9a1f1805e57
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Simplifies debugging of issues where the watchdog thread is involved.
Change-Id: I4862167bca4a942c7d4319a9374f1f83f292d831
Reviewed-by: Simon Hausmann <hausmann@gmail.com>
We don't want the signal dumper to pick up signals that our own
test machinery produces, such as the ones emitted from the watchdog
thread startup and shutdown. This would otherwise produce:
tst_Signaldumper::initTestCase() Signal: QThread(7fc969e0d870) started ()
At startup, and at shutdown even more confusingly:
tst_Signaldumper::UnknownTestFunc() Signal: QThread(7fc969e0d870) finished ()
Change-Id: I9e81fa168eaa92551d38d5576973bbf95ac23364
Reviewed-by: Volker Hilsheimer <volker.hilsheimer@qt.io>
