Make sure androidtestrunner reports the following cases as failures:
* XPASS test case
* The test output doesn't have closing block.
* The test result read from output.txt is empty
Pick-to: 6.2 6.3
Fixes: QTBUG-100312
Task-number: QTBUG-100470
Change-Id: Ifbb3c31fa22159df0c39b264e95e2f1ccd16e5ff
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
qmake had a feature to set androidtestrunner's timeout. This
feature is not yet ported to CMake. So with CMake, we always
have androidtestrunner's default timeout. Which was
five minutes. We have several tests which take just shy of five
minutes and are thus flaky.
This patch extends the timeout to eight minutes. This should be
a temporary workaround until the timeout feature is implemented
in CMake.
Task-number: QTBUG-101596
Pick-to: 6.2 6.3
Change-Id: Ia67ed6ab40a40f1b82861bd15bd04528ee9334ee
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Previously, when running unit tests on Android, the log presented
to the user was the log as known to the Qt logging system. This
does not include log messages generated by Java code using e.g.
Log.d("message"). Neither does it include log messages by system
libs. This patch changes androidtestrunner to capture the full
logcat output for the unit test. This improves the developer
experience when working with unit tests.
Fixes: QTBUG-93438
Change-Id: I580f728349041eb8a84a32d187754b7b5448f512
Reviewed-by: Edward Welbourne <edward.welbourne@qt.io>
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
It happens on Android Automotive emulator, that output file is in other
directory.
According android spec [1]:
"If a default user isn't specified, each adb subcommand has a different
user. The best practice is to retrieve the user ID with am
get-current-user and then explicitly use --user <userId> for any
command that supports it."
That is the reason why output file can be found in
/data/user/USER_ID/PACKAGE_NAME directory.
Checking path related to current user was added as backup solution.
[1]https://source.android.com/devices/tech/admin/multi-user-testing
Pick-to: 6.2
Change-Id: Id7e6ddef74f4f20b7469a07bba6a71d3622c4e20
Reviewed-by: Assam Boudjelthia <assam.boudjelthia@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
When doing a cross build for Android we need only the host tools which
are available under cmake's host_tools target, but androiddeployqt and
androidtestrunner are not part of that. This fixes that.
Pick-to: 6.2
Change-Id: Icdbc4a78ca050b66ec8df656d9ec766ef6c9f4b9
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Some Qt modules/tests go through features that require run-time
Android permissions which are granted by user input, that of
course is not ideally possible while running unit tests. So when
installing the app adb can do that for us.
Pick-to: 6.2
Change-Id: I92d33d5213c3779d7ca246ec0fb359d2ead4fa6f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
The format was renamed in 27db9e458cef512fca3a6b5c9ebbcda7a8172428.
Pick-to: 6.2
Change-Id: I53975c7467d8768dc9dc9ac2d89c42eefa12e22f
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
This reverts commit 1365a1c7a76a73838909d63acb60ebdfcbdff62e.
Reason for revert: Causes the issue in QTBUG-94445.
Pick-to: 6.2
Task-number: QTBUG-88508
Fixes: QTBUG-94445
Change-Id: I131234c9b1daeecd5f5f4921a9150d65a26db972
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Add new line between each argument type in the help print to make it
easier to read and not a big wall of text.
Change-Id: I7c7b2d2b59ef3a0889d1bb8c7af3d00b82bf820d
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
Before, building ${target}_make_apk always re-built the apk, instead of
rebuilding the apk only when inputs changed. This patch fixes that by
moving the creation code from a custom target to a custom command with
proper dependencies.
The androidtestrunner tool now does not check for the existence of an
apk anymore and always runs the make command that is supposed to build
the apk.
The ${target}_prepare_apk_dir target is not needed anymore by the Qt
build but is still used by Qt Creator's Android support. Add a
clarifying comment.
Fixes: QTBUG-93431
Change-Id: I00d65d616fef9511b03b65f879c4bc6cb92dfc30
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
make apk takes a LOT of time, showing that it's not hanged it's useful.
Change-Id: I4fb11e0f535cf2311d71dff3710135b465ecc06b
Reviewed-by: Andy Shaw <andy.shaw@qt.io>
First and foremost the condition was wrong ...
In order to help running qml test cases on Android without any android
specific hack on
user's .pro files we copy the entire project folder to assets. I copy the
entire folder
and not only the tst_*.qml files because it might contain data which is
needed by the tests to run.
Change-Id: I06323d9d52904317410dd2f440de65a0766a48b5
Reviewed-by: Tor Arne Vestbø <tor.arne.vestbo@qt.io>
Reviewed-by: Ville Voutilainen <ville.voutilainen@qt.io>
Add the above option to android test runner since its not required
when building with Cmake.
Change-Id: Ie5c9b0af29cfa76802723960071b3bd2a37a56a5
Reviewed-by: Qt CMake Build Bot
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
This is really just an optimization, but I suspect the author of the code
assumed that the bitwise &= operator short-cicuits. (It doesn't). So this
patch should then fix the code to have the intended behavior.
We don't have to check for failures in the remaining output files once
we've found one. pullFiles() returns just a bool indicating if one of the
output files has a recorded fail, so checking the remaining output files
after the first found failure is just a waste (and ideally they should
contain the same failure (however, flaky tests might break this ideal)).
Drive-by fix.
Change-Id: I951e500a69198e7ed124be32199ac81bbddb9bf7
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
g_options.buildPath did not have native separators. This caused
INSTALL_ROOT to not have native path separators, which again made
'mingw32-make check' fail to deploy the .apk. (The actual failure was that
mkdir failed because it did not accept paths with forward-slashes, so the
install_target rule failed. For the record, the command of that rule had
the following call to mkdir:
mkdir $(INSTALL_ROOT:@msyshack@%=%)\libs\x86
Change-Id: Id792c36986b52a527546d48aa9f7d9587e7a18d9
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
Reviewed-by: Frederik Gladhorn <frederik.gladhorn@qt.io>
First, we cannot quote the whole command in double quotes, since the
command itself very often have double quotes inside it.
Secondly, we should use the same that the ssh(1) command does:
Instead of adb shell setprop foo 'a b' we should use
adb shell setprop foo "'a b'" as explained here:
[https://developer.android.com/studio/command-line/adb#shellcommands]
Last hunk in isRunning():
The pipe character got eaten by the shell if we used a single quote (so
the stuff after the pipe (grep) was actually run locally).
Switching to ssh-style quoting fixed that.
Change-Id: I3075cdf8595ac2549cec8019f2cba79f77815e0b
Reviewed-by: BogDan Vatra <bogdan@kdab.com>
androidtestrunner now checks is the apk is build and if it is, it will skip the build phase.
Now we can build the apks in parallel (which takes most of the time) and run them sequentially.
This way running tests on Android is much faster.
Change-Id: I82f34723ac08f7728cc0daab3366e03821335eed
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
Ant is history for so many years, yet we forgot to remove it from
androiddeployqt.
[ChangeLog][Android] Remove ant support from androiddeployqt
Change-Id: I093295e18e8710c565e9d101e21c49b3c62f6322
Reviewed-by: Eskil Abrahamsen Blomfeldt <eskil.abrahamsen-blomfeldt@qt.io>
androidtestrunner is a tool needed to run qt tests on Android.
Now you can run tests as simple as you run them on Linux, macOS,
Windows.
"$ make check" it's all you need to run tests on the default android
device.
ANDROID_DEVICE_SERIAL env variable can be used to use a specific android
serial.
Use cases:
$ make -j1 check
-j1 is needed to make sure we don't run multiple tests in parallel.
$ ANDROID_DEVICE_SERIAL="emulator-5554" make check
Run the test on "emulator-5554"
$ make TESTARGS="-- -xml" check
Switch to xml output. All params after -- are passed to test
application.
$ make TESTARGS="-- -o out.xml,xml -o out.txt,txt -o -,tap -vs" check
Create two files out.xml and out.txt in the current folder and print
"tap" format to stdout and enable logging of every signal emission.
[ChangeLog][Android] Make it easy to run Qt tests on Android.
"$ make check" is all it's needed to run a test on an Android device.
Change-Id: I1a7f64b62608f7367b5a6aabf5d6c6e7e50242e6
Reviewed-by: Jędrzej Nowacki <jedrzej.nowacki@qt.io>
