Print logcats only if it's explicitly requested with the respective
argument or if there was a crash, i.e. where we can't get the test's
exit code. Otherweise, in case of normal failure, we don't really need
to pollute the output.
Change-Id: Ic4e6db5d9fd6786bd28204a3ec87f63a7ec9764a
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
(cherry picked from commit 256fd95a712384c6d5b8b1841ab9c749fca89970)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
qdebug.h includes many Qt and STL headers, so if you include a Qt header
you get all those transitive includes, which may affect build time.
- Where appropriate use the printf-like syntax of qDebug() and co.,
these don't need the QDebug streaming operators
- qfloat16 is used in an inline member function, so include it
explicitly
[ChangeLog][Potentially Source Incompatible Changes] Various Qt public
headers don't include QDebug any more; if you need QDebug's streaming
you'll have to include it in your code.
Task-number: QTBUG-132439
Change-Id: I750587e17a3b38fa226cd3af8eaccc8da580f436
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
(cherry picked from commit 185cba6e95a006d2548f20599f84390e5a3ad653)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
Print an error log when the build command fails that make it clear
that the build has failed and not necessarily the test.
Also, it seems according to the code, that if the build command fails
and --skip-install-root is set, the build command is run again.
Prepare the build command after parsing the args considering all the
other args and simply run the final build command in main() to simplify
things.
Pick-to: 6.8
Change-Id: I2f5a9d26bcea78349e1ef1cec4f717c37afd0225
Reviewed-by: Petri Virkkunen <petri.virkkunen@qt.io>
Re-shuffle the way logcat is handled, so now fetch the logcat by
default after the test run, then manually filter out the test pid
logs. Then try to look in the logs for potential ANR cases, if
found, add also logs belonging to the system_service pid which are
responsible for printing ANR details and usually the reason for the
ANR as well. At last, look for logcat crash marker that ndk-stack uses
and pass only that portion of the logs to ndk-stack command.
Fixes: QTBUG-130818
Pick-to: 6.8
Change-Id: I162d9d884a977e8f66a3253e33eebad4453c95cb
Reviewed-by: Petri Virkkunen <petri.virkkunen@qt.io>
Those two commands don't need to print info that is not necessarily
needed.
Pick-to: 6.8
Change-Id: I0a76c08848786ee9a9aa9e3cb876c66a79acaa3a
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Avoid using two processes where one is the input of the other and
having to do any dance to safely terminate each one in case the
other fails.
Do the following instead:
* Get the crash logs output first, if it's not possible return.
* Get the abi for ndk-stack command, if getting the value from
the device fails, we still get it from the build dir if the
emulator doesn't repond for some reason.
* Fall host architecture if failed to get ABI
* Call ndk-stack with the already obtainted logs
* Override the output only if ndk-stack gives back some output
* Print the output
This would avoid errors/warnings like this:
QProcess: Destroyed while process ("/opt/android/sdk/platform-tools/
adb") is still running.
QProcess: Destroyed while process ("/opt/android/android-ndk-r26b/ndk-
stack") is still running.
QProcess: Destroyed while process ("/opt/android/sdk/platform-tools/
adb") is still running.
Pick-to: 6.8
Change-Id: I2f508998c0376d901947b169632b22a1bbba74b9
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
We only need a snapshot of the crash logs, so pass -d to make the call
non-blocking, this means we we can check right away for start/finish
of that process. This would also, the error:
Error: failed to run ndk-stack command.
agent:2024/10/21 16:31:42 build.go:404: QProcess: Destroyed while
process ("/opt/android/sdk/platform-tools/adb") is still running.
Pick-to: 6.8
Change-Id: I2314d0b9567865934c2aa8f6d40ace2b4288ebb2
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
On device, use proper file names the same as the one passed
to the test. Also, along the way, make sure to also pull the
output of the same format of the stdout result. Currently,
it's not the case, for example, if we pass `-o name.txt,txt -o -,txt`,
the name.txt is not being pulled from the device, and
with this patch it will be.
Pick-to: 6.8
Change-Id: I5ecd22e6c37e5abfbe68b19ae102a852b01dd7e0
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Try to pull result files for 3 times in case they fail,
just to ensure less flakiness.
Pick-to: 6.8
Change-Id: Id052fe2359d190fe87e304bbd22fc2096d50cadb
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
And leave non-zero possitive exit codes to reflect the
number of test fails.
Pick-to: 6.8
Change-Id: If2871be14cdf46812471c9d2f5409dd8f3ddd3d9
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
If it's empty and the command didn't fail then there's no
crash so no need to print anything.
Pick-to: 6.8
Change-Id: Id9045138a6535220abf819b8d334c629765a6547
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Use colors only for non-ci environment since color characters
just make the output noisy and they are not rendered correctly.
Pick-to: 6.8
Change-Id: I14c5f9c215fbbe9a020011943455d2df615d9bee
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
androidtestrunner should've used ANDROID_SERIAL adb env var, instead
of introuducing the new one ANDROID_DEVICE_SERIAL, but for now
support both.
Pick-to: 6.8
Change-Id: Id0fc75fc64d7d3dd033c60e6ba9c6c3350ddd899
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
This option is mandatory in code, but it's documented as optional.
Pick-to: 6.8
Change-Id: I91a664b3dbfec45980f3bfb4313e13321610782b
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
The testrunner was always printing the results only at the end of the
test run. If tests are long, it means waiting long to have any sort of
result. With this, the results are printed to stdout whenever they are
written by qtest on the device.
Fixes: QTBUG-129975
Pick-to: 6.8
Change-Id: If4315eb74e73f6274a735e56be7e6989563bb514
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
After being able to return the real return code for the test app,
we don't need to rely on custom parsers anymore, because qt tests
already return the number of failed tests in the return code.
This also, would allow using the -silent qtest option where it was
not possible before because the test fail/success was dependent on
getting full test results to stdout.
Pick-to: 6.8
Change-Id: I75ecddd49235fcf25e1c0856b639228884a8cfba
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Write the exit code for qt tests at the end of the test run where
it would be available to androidtestrunner to find out the exact
number of failed tests, this would make the custom parses in the
androidtestrunner redandant.
Fixes: QTBUG-129976
Pick-to: 6.8
Change-Id: I3c2ea9780505f1d4a2a46ae94534cd1e4f07334c
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Move common args to function and add user Id to
info struct, also to be used more in upcoming patches.
Pick-to: 6.8
Change-Id: I14fd07f7daa52ef0369df16aceca05f0fa96f473
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Check if any or specified device is running and fail with
appropriate message instead of attempting to run other adb
commands that might fail with other reasons that hide the
actual root cause.
Pick-to: 6.8
Change-Id: I61078ca988f381e834b41140e36b0e4a5f77714f
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Try to check if the device is running few times while waiting
for the app to finish, just in case the adb ps command fails
for some reason while the app is actually still running.
Also, use ps -p <pid> to directly return only pid and avoiding
to filter out all the other processes with grep. grep can be then
used to only filter out the test package name and leaving out the
ps header.
Pick-to: 6.8
Fixes: QTBUG-127488
Change-Id: I85fe61106381fd1a7a26ef6347f8ab4a4a6678bb
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Forward QT_* env vars to the test same like QTEST_* vars,
this can be useful when debugging, this would allow us to
pass QT_DEBUG_PLUGINS.
Pick-to: 6.8
Change-Id: I9404ff7c7276f87e3e7d9cf57fa8f5b1eed02eb7
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
by default qDebug() prints to stderr which means any process expecting
the test output under stdout would not get any output, and in any case,
the test output shouldn't be printed to stderr. This also applies to
verbose commands output.
Pick-to: 6.8
Change-Id: Ifa77757176060f6452172768f3c7950019863775
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
The device serial number is being appended to the adb command which
is being treated as one single command and not as an argument, so pass
the serial number, if provided as an argument instead.
Pick-to: 6.8
Change-Id: I5c4a0945aca95e8d31f569b5cf99a85a37d22af2
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Currently, if the cmake program path passed to androidtestrunner
contains space, QProcess will get an invalid program path. To fix
that, quote the cmake command passed to androidtestrunner and use
QProcess::splitCommand() to get the program and args list.
Task-number: QTBUG-105524
Pick-to: 6.7
Change-Id: Icb665033884127972bff205157ff22aa469c135c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
* Don't add \n when using qDebug()
* Trim testAppArgs of extra space at the end.
* Don't print get-current-user command by default, that value
is not that useful to be always printed.
* Update outdated bit of the help printout to reflect what the
test runner does.
Pick-to: 6.7
Change-Id: I998bafc3aef524b368654f7b35a2a8e084d17542
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
This contains test specific variables like pid, sdkversion,
isPackageInstalled and isTestRunnerInterrupted which don't strictly
fall under 'Options'.
Pick-to: 6.7
Change-Id: I7c4a3422813f464b407d9b2be98c5b4a807c4f1e
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Leave a log message to know when the test runner timed out or the test
finished.
Pick-to: 6.7 6.6 6.5
Change-Id: If56ecaa5b0e3af22b3e26480a584e2d52ac97553
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
If the test runner was interrupted, stop waiting for the test
to start or to finish, go fetch the available test results
and logcat and uninstall the test app.
Also, set CMake TIMEOUT_SIGNAL_NAME to SIGINT and
TIMEOUT_SIGNAL_GRACE_PERIOD to 10 seconds to allow enough
time to fetch the logs and uninstall the test app.
Task-number: QTBUG-106479
Pick-to: 6.7 6.6 6.5
Change-Id: I4820cfe58f05d15179b4af819caa92e475881634
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
First, replace double quote characters with 3 escaped double quotes
to allow QProcess::splitCommand() to treat it as an actual quote
character that's part of the tag.
Then, escape single quote characters so they don't interfere with
the shell command and also to be treated as part of the argument.
Lastly, surround the args with escaped double quote so that args
with spaces are also treated as one.
Amends b044323c1656aeeec508afab8457755cc1e8c587.
Example of this:
tst_qkeyevent::modifiers("M","e","t","a") for double quotes
tst_qunicodetools::wordBreakClass(two words) for spaces
tst_QSpinBox::stepSelectAll("don't select all") for single quotes
Task-number: QTQAINFRA-5703
Change-Id: Ie4317e4350bbac619bac41e41f42613f50cf1ad4
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Cleanup the includes, and remove unused include dir in CMakeLists.txt.
Task-number: QTQAINFRA-5928
Change-Id: I477024291042042765b666bda882bab4a5f8313f
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Using QProcess would make the test runner more robust when dealing
with quoted arguments since it won't be using the system shell and
handles quoting under the hood.
Fixes: QTBUG-105524
Fixes: QTQAINFRA-5703
Change-Id: Ib666ffea33302f1dfc7e8972bd7750f14065c4fc
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
This has two parts, under testrunner, make sure to pass test args
that has quotes as \\\" so that the Android app gets a \" after parsing
the intent extras. The app args are then is passed to C++ where
QProcess::splitCommand() is used to parse the args string into a list,
and to preserve the quotes in this case, the former call expects triple
quotes to to get the quote character preserved in the split list.
Fixes: QTQAINFRA-5703
Change-Id: Iad81cadf24a6def21ef1536c882ecd510c4426cf
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Pass a CMake test TIMEOUT argument to androidtestrunner, using
a value of 95% of that timeout to allow time for the test runner
to do any cleanup before being killed.
If no test argument is provided, use the value from CMake property
DART_TESTING_TIMEOUT or CTEST_TEST_TIMEOUT. If that's not provided
default to 25 minutes which is the default for DART_TESTING_TIMEOUT.
Along the way set the default androidtestrunner timeout to 10 minutes
and fix the wrong timeout in the help menu.
Fixes: QTBUG-106479
Pick-to: 6.6 6.5
Change-Id: I12cd531583dd94954caf8044c37c22382d53d43c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Check if the common testlib environment variables are set on the
host and pass them to the test app. This include any env variable
starting with "QTEST_".
Fixes: QTBUG-106478
Task-number: QTBUG-106479
Pick-to: 6.6 6.5
Change-Id: I99e1b314b106cda20a66e3cac9a92b463b94f5c9
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Currently, if the test runner is interrupted by SIGINT and SIGTERM,
the semaphore is never released and thus consecutive test runner
executions will wait forever. To fix that, a signal handler for the
former signals is added to release the QSystemSemaphore. Also,
remove RunnerLocker and simply use QSystemSemaphore directly,
and explicitly release the semaphore at the end of the test runner
execution.
Fixes: QTBUG-115298
Pick-to: 6.6 6.5
Change-Id: I4f6cc0e4d837460de9a66248e09d3cbaf24ce959
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Reviewed-by: Zoltan Gera <zoltan.gera@qt.io>
Use ndk-stack [*] tool provided by the Android NDK to pretty
print crash stacktraces to contain <source-file>:<line-number>
lines for calls, making debugging crashes much easier.
The ndk-stack path can be provided with the new param --ndk-stack,
otherwise, it would be deduced using ANDROID_NDK_ROOT env var.
If the tool is not found or the unstripped libs path cannot be
obtained, the default crash report is printed instead.
[*] https://developer.android.com/ndk/guides/ndk-stack
Task-number: QTQAINFRA-5928
Pick-to: 6.6 6.5
Change-Id: I22c3ba131a44050c8fcbfd176d5ced096761d229
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
Currently, under waitToFinish(), we wait for the app to start and if
it doesn't start or starts and exits too quickly for the test runner
to catch it, it fails. The test runner tries to get the pid of the test
after it detects the app has started. However, we don't really need to
fail, the test runner could simply continue the execution and assume
the test was run and finished, and proceed to fetching the test results.
Since, the results fetching don't anyway rely on the pid, the test
runner can let that operation decide whether to fail or not (if not
output is found).
Also, along the way, instead of issuing a different command to get the
pid (i.e. adb shell pidof), we can use the same "adb shell ps" command
that is used to check if the app is running, to obtain the pid.
Fixes: QTQAINFRA-5928
Fixes: QTBUG-88508
Pick-to: 6.6 6.5
Change-Id: Ice945fcb686c4ef21b5f1c143aa22922ae928333
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
For better debugging for test failures, print both logcat and crash
stacktrace (since the start of the current test only), to have an
immediate idea of the reason of the failure or crash if any.
The crash report and the logcat for Android before level 23, is fetched
from the device's time of the app start since those two cases don't
have an option for filtering by pid.
Task-number: QTQAINFRA-5928
Fixes: QTBUG-114898
Pick-to: 6.6 6.5
Change-Id: I7760985032f342da4165cdb7573d4cfe5369ae03
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Axel Spoerl <axel.spoerl@qt.io>
To make debugging easier, print a log for each step in the test
output retrieval process instead of one error message that cover
multiple potential error cases.
Task-number: QTQAINFRA-5928
Pick-to: 6.6 6.5
Change-Id: Ia68c15e34cf4b0f8e2b0d351af785bb753807721
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
And deprecate the non-native key support API. Qt 7 may not even store
the old, non-native Qt.
Documentation in a new commit, when the dust settles.
Change-Id: I12a088d1ae424825abd3fffd171d2b549eeed040
Reviewed-by: Fabian Kosmale <fabian.kosmale@qt.io>
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
We already have:
a, a timeout as part of QtTest. By default it lets each test function
run for 5 minutes. This timeout can be configured using
QTEST_FUNCTION_TIMEOUT.
b, maxTimeBetweenOutput in the CI. The CI will kill the process if too
much time passes between individual output lines of a test.
c, maxTimeInSeconds in the CI. This does exactly the same as the
androidtestrunner timeout.
The CI timeouts can be centrally tuned per platform and Qt module. This
is preferable over a special timeout just for android.
As other people may be using androidtestrunner for unrelated projects,
don't delete the timeout, but simply disable it from CMake by setting
it to -1.
Task-number: QTBUG-106479
Task-number: QTBUG-101596
Task-number: QTBUG-100242
Change-Id: If4ce00948e204182bb12ac4859d3b0dd193de7ad
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Currently, arguments passed to the app through applicationArguments
extra bundle treat every space as an argument separator. This then
doesn't handle the case where an argument is a space separated quoted
multi-word. This is more apparent when androidtestrunner is passing
test arguments to the app where an argument can be a test case with
a data tag that contains a space, which then is treated as two separate
tag names.
This change makes sure that androidtestrunner quotes each argument,
and the app doesn't split the arguments list by spaces, but rather
passed the argument string directly to c++ where
QProcess::splitCommand() is used to get the correct set of arguments
that will be passed to main().
Pick-to: 6.4 6.3 6.2
Task-number: QTBUG-104730
Change-Id: I45d8ca979d90f2a383c84623f0eb2eec29bba727
Reviewed-by: Dimitrios Apostolou <jimis@qt.io>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Replace the current license disclaimer in files by
a SPDX-License-Identifier.
Files that have to be modified by hand are modified.
License files are organized under LICENSES directory.
Task-number: QTBUG-67283
Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Lars Knoll <lars.knoll@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
Move shellquote helper functions into a common place instead of having a
copy in each tool's code.
Change-Id: I9723c11f894a211864788a7635773610c0fde739
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Required for porting away from QLatin1Char/QLatin1String in scope of
QTBUG-98434.
As a drive-by, fix qsizetype -> int narrowing conversion warnings for
the touched lines.
Change-Id: Iebcbdbd7cecac09d0a7039e3ef6a4509d33039ba
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
