69 Commits

Author SHA1 Message Date
Thiago Macieira
d6bef9a6f9 Switch tools and apps to non-strict C++ builds too
This complements commit 30e04340dac26ebd09f9bc8ceb598e873ab63ba7
("Switch to non-strict C++ builds so QT_SUPPORTS_INT128 is true") and
adds the same modification to tools and apps.

Otherwise qtconnectivity's sdspscanner won't compile, as it needs
quint128.

Task-number: QTBUG-119901
Change-Id: Ib9a6fe573e9420a471fdfffd76a2e43d3c5496d8
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
(cherry picked from commit 7e37962cd0d1faf65249e08020619ef7175a280e)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2024-08-25 17:03:29 +00:00
Alexandru Croitor
9be3d7a7a0 CMake: Allow specifying most SBOM options to qt_internal_add_foo
Previously, only a very short subset of options related to attribution
files could be specified to qt_internal_add_module /
qt_internal_extend_target.

It is more convenient to allow specifying most (safe) options, instead
of calling another function.

Unsafe are considered paths like INSTALL_PATH and derivatives, TYPE
which is too generic, and some other ones like LIBRARIES which would
be duplicated, and causes warnings in cmake_parse_arguments if
duplicated.

Change the code to allow specifying most SBOM options and forwarding
them to _qt_internal_extend_sbom.

Task-number: QTBUG-122899
Change-Id: I6eb723e165edf59973d83c66eace43acdce237de
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
(cherry picked from commit 81162cc8b6931c8659dc722be97eb788e344812a)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2024-06-19 08:15:39 +00:00
Alexandru Croitor
65bf57ce7c CMake: Generate an SPDX v2.3 SBOM file for each built repository
This change adds a new -sbom configure option to allow generating and
installing an SPDX v2.3 SBOM file when building a qt repo.

The -sbom-dir option can be used to configure the location where
each repo sbom file will be installed.

By default it is installed into

 $prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx

which is basically ~/Qt/sbom/qtbase-6.8.0.spdx

The file is installed as part of the default installation rules, but
it can also be installed manually using the "sbom" installation
component, or "sbom_$lower_project_name" in a top-level build. For
example: cmake install . --component sbom_qtbase

CMake 3.19+ is needed to read the qt_attribution.json files for
copyrights, license info, etc. When using an older cmake version,
configuration will error out. It is possible to opt into using an
older cmake version, but the generated sbom will lack all the
attribution file information.
Using an older cmake version is untested and not officially supported.

Implementation notes.

The bulk of the implementation is split into 4 new files:

- QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing
  and dispatching the generation of various pieces of the SBOM document
  e.g. a SDPX package associated with a target like Core, a SDPX
  file entry for each target binary file (per-config shared library,
  archive, executable, etc)

- QtPublicSbomGenerationHelpers.cmake - for non-Qt specific
  implementation of SPDX generation. This also has some code that was
  taken from the cmake-sbom 3rd party project, so it is dual licensed
  under the usual Qt build system BSD license, as well as the MIT
  license of the 3rd party project

- QtPublicGitHelpers.cmake - for git related features, mainly to embed
  queried hashes or tags into version strings, is dual-licensed for
  the same reasons as QtPublicSbomGenerationHelpers.cmake

- QtSbomHelpers.cmake - Qt-specific functions that just forward
  arguments to the public functions. These are meant to be used in our
  Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones
  for naming consistency. These function would mostly be used to
  annotate 3rd party libraries with sbom info and to add sbom info
  for unusual target setups (like the Bootstrap library), because most
  of the handling is already done automatically via
  qt_internal_add_module/plugin/etc.

The files are put into Public cmake files, with the future hope of
making this available to user projects in some capacity.

The distinction of Qt-specific and non-Qt specific code might blur a
bit, and thus the separation across files might not always be
consistent, but it was best effort.

The main purpose of the code is to collect various information about
targets and their relationships and generate equivalent SPDX info.

Collection is currently done for the following targets: Qt modules,
plugins, apps, tools, system libraries, bundled 3rd party libraries
and partial 3rd party sources compiled directly as part of Qt targets.

Each target has an equivalent SPDX package generated with information
like version, license, copyright, CPE (common vulnerability
identifier), files that belong to the package, and relationships on
other SPDX packages (associated cmake targets), mostly gathered from
direct linking dependencies.

Each package might also contain files, e.g. libQt6Core.so for the Core
target. Each file also has info like license id, copyrights, but also
the list of source files that were used to generate the file and a
sha1 checksum.

SPDX documents can also refer to packages in other SPDX documents, and
those are referred to via external document references. This is the
case when building qtdeclarative and we refer to Core.

For qt provided targets, we have complete information regarding
licenses, and copyrights.

For bundled 3rd party libraries, we should also have most information,
which is usually parsed from the
src/3rdparty/libfoo/qt_attribution.json files.
If there are multiple attribution files, or if the files have multiple
entries, we create a separate SBOM package for each of those entries,
because each might have a separate copyright or version, and an sbom
package can have only one version (although many copyrights).

For system libraries we usually lack the information because we don't
have attribution files for Find scripts. So the info needs to be
manually annotated via arguments to the sbom function calls, or the
FindFoo.cmake scripts expose that information in some form and we
can query it.

There are also corner cases like 3rdparty sources being directly
included in a Qt library, like the m4dc files for Gui, or PCRE2 for
Bootstrap.
Or QtWebEngine libraries (either Qt bundled or Chromium bundled or
system libraries) which get linked in by GN instead of CMake, so there
are no direct targets for them.
The information for these need to be annotated manually as well.

There is also a distinction to be made for static Qt builds (or any
static Qt library in a shared build), where the system libraries found
during the Qt build might not be the same that are linked into the
final user application or library.

The actual generation of the SBOM is done by file(GENERATE)-ing one
.cmake file for each target, file, external ref, etc, which will be
included in a top-level cmake script.

The top-level cmake script will run through each included file, to
append to a "staging" spdx file, which will then be used in a
configure_file() call to replace some final
variables, like embedding a file checksum.

There are install rules to generate a complete SBOM during
installation, and an optional 'sbom' custom target that allows
building an incomplete SBOM during the build step.

The build target is just for convenience and faster development
iteration time. It is incomplete because it is missing the installed
file SHA1 checksums and the document verification code (the sha1 of
all sha1s). We can't compute those during the build before the files
are actually installed.

A complete SBOM can only be achieved at installation time. The install
script will include all the generated helper files, but also set some
additional variables to ensure checksumming happens, and also handle
multi-config installation, among other small things.

For multi-config builds, CMake doesn't offer a way to run code after
all configs are installed, because they might not always be installed,
someone might choose to install just Release.
To handle that, we rely on ninja installing each config sequentially
(because ninja places the install rules into the 'console' pool which
runs one task at a time).
For each installed config we create a config-specific marker file.
Once all marker files are present, whichever config ends up being
installed as the last one, we run the sbom generation once, and then
delete all marker files.

There are a few internal variables that can be set during
configuration to enable various checks (and other features) on the
generated spdx files:

- QT_INTERNAL_SBOM_VERIFY
- QT_INTERNAL_SBOM_AUDIT
- QT_INTERNAL_SBOM_AUDIT_NO_ERROR
- QT_INTERNAL_SBOM_GENERATE_JSON
- QT_INTERNAL_SBOM_SHOW_TABLE
- QT_INTERNAL_SBOM_DEFAULT_CHECKS

These use 3rd party python tools, so they are not enabled by default.
If enabled, they run at installation time after the sbom is installed.
We will hopefully enable them in CI.

Overall, the code is still a bit messy in a few places, due to time
constraints, but can be improved later.

Some possible TODOs for the future:
- Do we need to handle 3rd party libs linked into a Qt static library
  in a Qt shared build, where the Qt static lib is not installed, but
  linked into a Qt shared library, somehow specially?
  We can record a package for it, but we can't
  create a spdx file record for it (and associated source
  relationships) because we don't install the file, and spdx requires
  the file to be installed and checksummed. Perhaps we can consider
  adding some free-form text snippet to the package itself?

- Do we want to add parsing of .cpp source files for Copyrights, to
  embed them into the packages? This will likely slow down
  configuration quite a bit.

- Currently sbom info attached to WrapFoo packages in one repo is
  not exported / available in other repos. E.g. If we annotate
  WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be
  available when looking up WrapZLIB in qtimageformats.
  This is because they are IMPORTED libraries, and are not
  exported. We might want to record this info in the future.

[ChangeLog][Build System] A new -sbom configure option can be used
to generate and install a SPDX SBOM (Software Bill of Materials) file
for each built Qt repository.

Task-number: QTBUG-122899
Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by:  Alexey Edelev <alexey.edelev@qt.io>
(cherry picked from commit 37a5e001277db9e1392a242171ab2b88cb6c3049)
Reviewed-by: Qt Cherry-pick Bot <cherrypick_bot@qt-project.org>
2024-06-13 14:55:07 +00:00
Alexandru Croitor
a1445670f5 CMake: Fix configs passed to qt_get_install_target_default_args
Because the configs passed to the ALL_CMAKE_CONFIGS option were quoted
cmake_parse_arguments(PARSE_ARGV) would escape the semicolon in
the set value, effectively doing
 set(arg_ALL_CMAKE_CONFIGS "Release\;Debug")

Then the
 list(GET arg_ALL_CMAKE_CONFIGS 0 first_config)
call would essentially do
 set(first_config "Release;Debug")
and the
 if(all_configs_count GREATER 1
    AND NOT arg_CMAKE_CONFIG STREQUAL first_config)
condition would never trigger because a single config string can never
equal a double config string.

Remove the quotes to ensure correct behavior.

This won't really trigger any behavior change, because we exclude
installation of Debug executables in -debug-and-release builds,
but it will make --trace-expand logs less confusing.

Amends f240d94f140ba1614828804efafd2fc5e6d00099

Pick-to: 6.7
Change-Id: I53179511c7698c90b33cb3ff2762cef680a99815
Reviewed-by:  Alexey Edelev <alexey.edelev@qt.io>
2024-05-29 23:01:47 +02:00
Alexey Edelev
816071d440 Move QT_DISABLE_DEPRECATED_UP_TO and QT_WARN_DEPRECATED_UP_TO to header
The new approach allows to imply the macro definitions for every Qt
submodule and user project without the need of setting it explicitly
from CMake. This also prevent users from introducing the
incompatibility between Qt modules due to defining
the QT_DISABLE_DEPRECATED_UP_TO version lower than qtbase was built
with.

Task-number: QTBUG-124765
Change-Id: I7ba481f62cb9073ae0343c400ffc26f239f080f1
Reviewed-by: Ivan Solovev <ivan.solovev@qt.io>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2024-05-22 18:38:09 +02:00
Alexey Edelev
dc9c67c260 Allow filtering modules that will use pre-built tools
Add the support for QT_INTERNAL_FORCE_FIND_HOST_TOOLS_MODULE_LIST
argument which allows to set the list of tool targets that should not
build their tools if QT_FORCE_FIND_TOOLS is set to ON.

This mechanism doesn't work when crosscompiling.

Change-Id: I597ed766cc741cacf9cb73dd8dd255644c4c9e66
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2024-03-20 10:07:21 +01:00
Alexey Edelev
0b79ee6d3b Make possible partial tool lookup when using QT_HOST_PATH
Consider QT_HOST_PATH when setting the QT_WILL_BUILD_TOOLS flag.
In this case if we do not crosscompile or require building tools
aka QT_FORCE_BUILD_TOOLS=ON, we may by pass tool_FOUND check and
assume that we will try building the missing tool.
Set qt_require_find_tools GLOBAL property to indicate the tools
lookup requirement.

For tools that can be found we will try re-using them from
QT_HOST_PATH, but not building from scratch.

Change-Id: I94e92f62eb799308e38721b4d580052bb4bb35f9
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2024-02-14 10:47:40 +01:00
Alexey Edelev
d14f15f380 Allow disabling QT_USE_QSTRINGBUILDER definition for tool targets
Currently QT_USE_QSTRINGBUILDER is added for all Qt tool targets by
default, and it's not possible to remove this definition.
Replace this definition with the generator expression that is
propagated by the PlatformToolInternal target.

Change-Id: Iac3bd3ea76e7b439cf7957146b4b6dd20ecdbe3a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-08-09 15:53:07 +02:00
Marc Mutz
51b3269eef cmake helpers: support NO_PCH_SOURCES wherever NO_UNITY_BUILD_SOURCES are
The former implies the latter, so it makes little sense to treat them
differently. Even if some types of targets are (currently) never
compiled with PCH, users of qt_internal_all_x() shouldn't need to know
such details.

Pick-to: 6.6 6.5
Change-Id: I8ead238a8d9e55da632b2929638b67724a42d73c
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2023-08-07 23:56:10 +02:00
Alexandru Croitor
3f6041a24f CMake: Add missing check for tool target promotion
We shouldn't try to promote tool targets if they were not created
when Qt6FooTools_FOUND is FALSE due to missing dependencies and
Qt6FooToolsTargets.cmake is not included.

Add a check for Qt6FooTools_FOUND to prevent errors like:

 CMake Error at lib/cmake/Qt6/QtPublicTargetHelpers.cmake:257
 (get_property):
  get_property could not find TARGET Qt6::qtprotobufgen.  Perhaps it
  has not yet been created.

Pick-to: 6.5 6.6
Change-Id: Ie26db637d4d8ef682a0ada5ea36ef0e8ceced008
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2023-07-05 15:09:33 +02:00
Amir Masoud Abdol
45e17162ba Set QT_COMMAND_LINE_PARSER_NO_GUI_MESSAGE_BOXES before TRY_RUN
Set `QT_COMMAND_LINE_PARSER_NO_GUI_MESSAGE_BOXES` before trying to run
qt tools for correctness. This is to suppress QCommandLinerParser from
showing a message box when console is not available.

Pick-to: 6.5.2 6.5 6.6
Task-number: QTBUG-114530
Change-Id: Ib3d264a799a5da1f620d2bebe55539bafc43da0f
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2023-06-20 07:34:09 +02:00
Amir Masoud Abdol
331f316b91 Disable TRY_RUN when cross compiling
Fixes: QTBUG-114625
Pick-to: 6.5.2 6.5 6.6
Change-Id: Idec5ea438ebf3f88d63da7d956ab52e2a0fa1821
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Samuli Piippo <samuli.piippo@qt.io>
2023-06-17 20:26:12 +02:00
Amir Masoud Abdol
69d74afd43 Add TRY_RUN_FLAGS argument to customize the TRY_RUN command
Some of our tools don't have the `-h`, or `-v` flag, or it could be
that the `-v` flag also prints the entire `--help` as well, e.g.,
`androiddeployqt`. When running in Jenkins, this may lead to a message
box being shown and consequently stopping the build. By customizing the
flag per tool, and limiting the TRY_RUN to tools that support `-v` or
`--version`, we can avoid this.

Also removed TRY_RUN from `macdeployqt` which doesn't need it anyway.

Amend 41b32cd2c4706fa280fc779d5dec132ee9edf0f6

Pick-to: 6.5.2 6.5 6.6
Fixes: QTBUG-114530
Change-Id: I78e3344d2553c0050c285ae86f2310bd373c6c57
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2023-06-15 11:46:00 +00:00
Amir Masoud Abdol
a33dd60e21 Add TRY_RUN option to qt_internal_add_tool
To make sure that Qt essential tools can be executed after a successful
build, we can now pass the TRY_RUN argument to `qt_internal_add_tool`.
On Windows, this option creates a custom command and a custom target
(${target}_try_run) for the tool, and tries to run the tool from a batch
script. If the program fails to run because of missing libraries, an
error will be shown, and build halts; otherwise,
`${target_name}_try_run_passed` file will be generated and the build
continues.

Pick-to: 6.5
Task-number: QTBUG-113273
Task-number: QTBUG-112747
Change-Id: I760588714bcf9db69505abe3df717733352a8284
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-06-01 12:02:28 +02:00
Amir Masoud Abdol
da2b0307f8 Replace STREQUAL cases with variable checks
As I was investigating the NO_UNITY_BUILD_SOURCES issue, I realized that
we don't need to pass these quoted, especially now that we have moved
to `cmake_parse_arguments(PARSE_ARGV`, and we can check their existence
just by checking the `arg_*`, and that should be sufficient.

I also left a warning that we are aware of this.

Pick-to: 6.5
Change-Id: I4d939e80dc4671ea3ae9dc61516279f69ba2c5a5
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-04-04 08:55:02 +02:00
Amir Masoud Abdol
ed85df8008 Improvement to NO_UNITY_BUILD_SOURCES, and fix related bugs
The source of the problem was in `qt_set_target_info_properties` which
was not able to process the NO_UNITY_BUILD_SOURCES, and therefore
leaking it into the `TARGET_COPYRIGHT`, ie., the last argument. So, I
decided to pass Unity Build arguments before them, and closer to
SOURCES, which is nicer to read, and avoid similar situation. And
I reverted the work around in the amend commit, and passing the
arguments normally.

This happens because we pass an unfiltered ${ARGN} from
qt_internal_add_executable to qt_set_target_info_properties and that the
current change is merely a workaround that ensures they get
circumstantially filtered out, because the NO_UNITY_BUILD_SOURCES option
appears before any of the first TARGET_ props.

Amend cd12c1f33281452d478bb94744d76bead5c7363a

Task-number: QTBUG-99238
Task-number: QTBUG-109394
Pick-to: 6.5
Change-Id: Idb37937cf53e708425402c90f55bda8816e27f29
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-04-03 10:41:13 +00:00
Amir Masoud Abdol
cd12c1f332 Fix an issue with passing NO_UNITY_BUILD_SOURCES
Amend 31518f1a4e3c4a7c77ae2e008e903a849ba75dd6
Amend 360293623094a31586981206e59c92aa6235163d

Pick-to: 6.5
Task-number: QTBUG-109394
Change-Id: Ibb2fd881c02b6496f06b8b623fa6009358755a26
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-03-15 18:18:01 +01:00
Amir Masoud Abdol
31518f1a4e Add some utilities for UNITY_BUILD
- The following commands accepts NO_UNITY_BUILD, and
  NO_UNITY_BUILD_SOURCES arguments to opt out of the unity build, and
  to exclude some source files from unity build, respectively.
  - qt_internal_add_executable
  - qt_internal_add_module
  - qt_internal_add_plugin
  - qt_internal_add_tool
  - qt_internal_extend_target
  - qt_internal_add_common_qt_library_helper
  - qt_internal_add_cmake_library
  - qt_internal_add_simd_part
- Unity build is disabled by default in these:
  - qt_internal_add_test
  - qt_internal_add_test_helper
  - qt_internal_add_benchmark
  - qt_internal_add_3rdparty_library
- qt_update_ignore_pch_source also excludes the files from unity_build

Pick-to: 6.5
Task-number: QTBUG-109394
Change-Id: I5d0e7df633738310a015142a6c73fbb78b6c3467
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-02-08 23:47:07 +01:00
Amir Masoud Abdol
7e82577680 Retire the qt_parse_all_arguments
This replaces the qt_parse_all_arguments macro with the built-in
`cmake_parse_arguments(PARSE_ARGV`. In addition, a new function,
_qt_internal_validate_all_args_are_parsed, can be used to check whether
any _UNPARSED_ARGUMENTS have been passed to the function.

Fixes: QTBUG-99238
Change-Id: I8cee83dc92dc6acdaaf747ea6ff9084c11dc649b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2023-01-18 14:47:12 +01:00
Alexey Edelev
4e4cd7416a Add the 'QT_INTERNAL_AVOID_OVERRIDING_SYNCQT_CONFIG' flag
The flag avoids overriding of the default CMake build config when
building syncqt.

Add extra checks to the make sure that configure-time tools use the
correct build type.

Fixes: QTBUG-109792
Pick-to: 6.5
Change-Id: I572fed60c58e59297fa559aea6eb86af94b979b7
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2023-01-13 14:38:16 +01:00
Alexey Edelev
5761f254a0 Add configure-time tools to the 'host_tools' target
Rebuild configure-time tools within the 'host_tools' meta target.

Pick-to: 6.5
Task-number: QTBUG-109792
Change-Id: I1bd69c8e71185b6ec236c0f32064ba8f2c9f5ab0
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2023-01-05 20:01:21 +01:00
Cristian Adam
501018ea33 Win32: Add "longPathAware" manifest / rc for internal tools
This will enable all qt internal tools to be able to access files
on paths longer than 255 characters.

Two examples that were not working before: moc and windeployqt.

Fixes: QTBUG-109207
Change-Id: I93f9770f1d3c4f3f2cca4655e4bed89c95b9786b
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-12-08 17:56:47 +01:00
Amir Masoud Abdol
ca24ee14df Create the versioned variant of Qt tools in cross platform build
In cross builds, we are not creating versioned links for qt tools. This
patch addresses that. I've changed the signature of the
`qt_internal_install_versioned_link` such that it can be used for
non-target as well, so in cross build the qmake or qtmake.bat can be
processed with the same function.

Fixes: QTBUG-109024
Change-Id: I246621c18325d084622ca92b422e815ed06f1381
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-12-07 07:09:15 +01:00
Alexey Edelev
b89d63515b Replace the syncqt.pl script with syncqt tool
syncqt.pl adds an extra dependency on perl when building Qt. Modern C++
provides the convenient cross-platform way to access a filesystem and
to use regular expressions, so we may replace the perl script with C++
application. The syncqt executable is built at configure time and
installed as QtCore tool. It's running at configure time to deliver the
required header files for IDE to build a consistent code model and at
the build time to keep tracking changes in header files and generate
the missing aliases without reconfiguring. 'syncqt' only parses header
files from a CMake build tree, so the resulting Qt installation only
contains interfacing headers that belong to the platform that Qt is
built for. 'sync.profile' files are not used as the 'source of truth'
for sync qt procedure anymore, all the necessary information is taken
from either CMake files at configure time or from the module header
files while parsing them.

syncqt.pl is still in place since it's required as fallback solution
for a smooth transition to the new syncqt implementation for all qt
repositories.

This patchset only enables the C++ based syncqt for 'qtbase'
repository.

From the performance perspective C++ version works faster then perl
script, also the configure time is reduced significally on subsequent
reconfigurations - up x2 times faster when re-configuring repository,
but it also takes time to compile the tool itself the first time.
Numbers for qtbase:
           syncqt.pl  syncqt.cpp
 initial:  0m16,035s  0m20,413s
 reconfig: 0m6,819s   0m3,725s

The syncing procedure can be run separately for each module using
<ModuleName>_sync_headers targets. The 'sync_headers' target can be
used to sync all the modules at once.

Task-number: QTBUG-87480
Task-number: QTBUG-103196
Change-Id: I8c938bcaf88a8713b39bbfd66d9e7ef12b2c3523
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-27 13:12:11 +02:00
Alexey Edelev
6f12603cb1 Use PARSE_ARGV signature to process arguments instead of escaping
Use PARSE_ARGV signature of the cmake_parse_arguments function to avoid
escaping of semicolons when passing arguments to the
qt_internal_add_configure_time_tool and
qt_internal_add_configure_time_executable function.

Amends ac74b60c9c1101288eb2c558420ba69f675a2ee2

Task-number: QTBUG-87480
Change-Id: I343abbd75107e56aaccab6e388db8dbda0525af3
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-21 14:00:37 +02:00
Alexey Edelev
ac74b60c9c Add function to add and compile executables at configure time
qt_internal_add_configure_time_executable compiles the executable
at configure time and exposes it to the CMake source tree. This is
useful when need to run a small C++ program at configure time.

Task-number: QTBUG-87480
Change-Id: I031efe797c8afa0721d75b46d4f36f67276bf46e
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-13 00:50:47 +02:00
Alexey Edelev
0c1368c7e4 Move the tool look-up functionality to a separate function
Change-Id: Ibe9ef9de981afac4b585068c62eef5af01989417
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-12 20:09:03 +02:00
Ivan Solovev
b83fc9f41a CMake: propagate deprecations definitions to tools and apps
If Qt itself is built without the deprecated APIs, so should be the
tools and apps.

This patch makes sure that the specified QT_DISABLE_DEPRECATED_UP_TO
and QT_WARN_DEPRECATED_UP_TO values are correctly used in the internal
tools and apps.

Fixes: QTBUG-105102
Change-Id: I7a51bddbd839c7b71efa0bff8ec959df64c53b82
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-07 19:51:38 +02:00
Alexey Edelev
0255cce4de Use the INSTALL_CMAKE_NAMESPACE to locate module tools target
Replace hardcoded Qt6 prefix of module tools target when setting
the name of tools package.

Change-Id: Icb6f38cce766c9d32216a65a8a5ce9552d622b72
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-09-01 21:37:11 +02:00
Lucie Gérard
32df595275 Change the license of all CMakeLists.txt and *.cmake files to BSD
Task-number: QTBUG-105718
Change-Id: I5d3ef70a31235868b9be6cb479b7621bf2a8ba39
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-23 23:58:42 +02:00
Alexandru Croitor
e671b852e7 CMake: Improve tool not found error message
Pick-to: 6.4
Change-Id: I7f04e4af80c4d23b855c8c9d5f5017f8afb112ea
Reviewed-by: Morten Johan Sørvig <morten.sorvig@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-12 14:55:03 +02:00
Lucie Gérard
fb1b20eab3 Add license headers to cmake files
CMakeLists.txt and .cmake files of significant size
(more than 2 lines according to our check in tst_license.pl)
now have the copyright and license header.

Existing copyright statements remain intact

Task-number: QTBUG-88621
Change-Id: I3b98cdc55ead806ec81ce09af9271f9b95af97fa
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-08-03 17:14:55 +02:00
Alexandru Croitor
ea8645d846 CMake: Deprecate _add_app/executable/test/tool PUBLIC_LIBRARIES option
Warn projects not to use it because PUBLIC_LIBRARIES don't make
sense for executable targets and it also led to some issues in the
internal functions where some of them did not expect to receive
PUBLIC_LIBRARIES.

To ensure builds don't needlessly break, treat PUBLIC_LIBRARIES values
as regular LIBRARIES. In the future we might add an error instead.

Using PUBLIC_LIBRARIES in qt_internal_add_app, etc, accidentally
worked because the option name and the values following it were
parsed as values of the "previous" option, like SOURCES or
INCLUDE_DIRECTORIES or LIBRARIES, and when those got
passed through to qt_internal_extend_target, things magically worked.

We have a lot of projects using PUBLIC_LIBRARIES, mostly due to the
way qmake pro files were written and how pro2cmake converted them.
We'll have to clean up each repo.

Change-Id: I69e09d34afdf98f0d47c08d324643fc986f8131c
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2022-07-29 18:33:00 +02:00
Alexandru Croitor
dd1030a450 CMake: Record used package version for each target dependency
When recording which package version to look for in
QtFooModuleDependencies.cmake and other files like it,
instead of using PROJECT_VERSION, use the version of the
package that contains the dependency.

For example if we're hypothetically building the qtdeclarative repo
from the 6.4 branch, against an installed 6.2 qtbase, then
the Qt6QmlModuleDependencies.cmake file will have a
find_package(Qt6Core 6.2) call because qtdeclarative's
find_package(Qt6Core) call found a 6.2 Core when it was configured.

This allows switching the versioning scheme of specific Qt modules
that might not want to follow the general Qt versioning scheme.

The first candidate would be QtWebEngine which might want to
follow the Chromium versioning scheme, something like
Qt 6.94.0 where 94 is the Chromium major version.

Implementation notes.
We now record the package version of a target in a property
called _qt_package_version. We do it for qt modules, plugins,
3rd party libraries, tools and the Platform target.

When we try to look up which version to write into the
QtFooModuleDependencies.cmake file (or the equivalent Plugins and
Tools file), we try to find the version
from a few sources: the property mentioned above, then the
Qt6{target}_VERSION variable, and finally PROJECT_VERSION.
In the latter case, we issue a warning because technically that should
never have to happen, and it's a bug or an unforeseen case if it does.

A few more places also need adjustments:
 - package versions to look for when configuring standalone
   tests and generating standalone tests Config files
 - handling of tools packages
 - The main Qt6 package lookup in each Dependencies.cmake files

Note that there are some requirements and consequences in case a
module wants to use a different versioning scheme like 6.94.0.

Requirements.
 - The root CMakeLists.txt file needs to call find_package with a
   version different from the usual PROJECT_VERSION. Ideally it
   should look for a few different Qt versions which are known to be
   compatible, for example the last stable and LTS versions, or just
   the lowest supported Qt version, e.g. 6.2.6 or whenever this change
   would land in the 6.2 branch.
 - If the repository has multiple modules, some of which need to
   follow the Qt versioning scheme and some not,
   project(VERSION x.y.z) calls need to be carefully placed in
   subdirectory scopes with appropriate version numbers, so that
   qt_internal_add_module / _tool / _plugin pick up the correct
   version.

Consequences.
 - The .so / .dylib names will contain the new version, e.g. .so.6.94
 - Linux ELF symbols will contain the new versions
 - syncqt private headers will now exist under a
   include/QtFoo/6.94.0/QtFoo/private folder
 - pri and prl files will also contain the new version numbers
 - pkg-config .pc files contain the new version numbers
 - It won't be possible to write
   find_package(Qt6 6.94 COMPONENTS WebEngineWidgets) in user code.
   One would have to write find_package(Qt6WebEngineWidgets 6.94)
   otherwise CMake will try to look for Qt6Config 6.94 which won't
   exist.
 - Similarly, a
   find_package(Qt6 6.4 COMPONENTS Widgets WebEngineWidgets) call
   would always find any kind of WebEngine package that is higher than
   6.4, which might be 6.94, 6.95, etc.
 - In the future, if we fix Qt6Config to pass EXACT to its
   subcomponent find_package calls,
   a find_package(Qt6 6.5.0 EXACT COMPONENTS Widgets WebEngineWidgets)
   would fail to find WebEngineWidgets, because its 6.94.0 version
   will not be equal to 6.5.0. Currently we don't pass through EXACT,
   so it's not an issue.

Augments 5ffc744b791a114a3180a425dd26e298f7399955

Task-number: QTBUG-103500
Change-Id: I8bdb56bfcbc7f7f6484d1e56651ffc993fd30bab
Reviewed-by: Michal Klocek <michal.klocek@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-07-01 10:50:57 +02:00
Kai Köhne
04cc705947 CMake: Fix typos
Found by codespell

Pick-to: 6.4
Change-Id: I4907e423b6b345acf82f2d7e0ed62479719d694e
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-06-13 15:37:36 +02:00
Joerg Bornemann
7f94aa23f5 CMake: Fix initialization of QT_BUILD_TOOLS_BY_DEFAULT
...when QT_BUILD_TOOLS_WHEN_CROSSCOMPILING is ON.

When QT_BUILD_TOOLS_WHEN_CROSSCOMPILING is ON, we want to set
QT_FORCE_BUILD_TOOLS.  But this happened too late: after the
initialization of QT_BUILD_TOOLS_BY_DEFAULT.  This value depends on
QT_FORCE_BUILD_TOOLS.

This amends commit acfbe3b7795c741b269fc23ed2c51c5937cd7f4f.

Change-Id: Ibdba54da943aea1b55618f10d2b8485f4390878a
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-05-11 18:53:37 +02:00
Alexey Edelev
f087e63bf5 CMake: Make possible building Qt tools without the use of core library
Replace BOOTSTRAP option with the single value CORE_LIBRARY argument
in qt_internal_add_tool and qt_internal_add_executable functions.
The introduced argument now may accept 'Bootstap' and 'None' values.
Use 'Bootstap' to link Qt::Boostrap library instead Qt::Core or 'None'
to avoid any core library linking. This is useful for tools that need
to use the CMake deployment routines, but not require the Qt::Core
functionality.

Task-number: QTBUG-87480
Change-Id: I64a8b17f16ac5fe43c6b385252dc21def0c88d2c
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-05-06 22:06:22 +02:00
Thiago Macieira
acfbe3b779 CMake: also allow building tools when found elsewhere in host builds
Previously, this was only supported when cross-compiling, but that's an
unnecessary limitation. Instead, make it possible to build the "host"
tools (notably qmake) even when they've been found elsewhere due to
QT_FORCE_FIND_TOOLS=ON and a supplied QT_HOST_PATH.

The combination of QT_FORCE_FIND_TOOLS and QT_FORCE_BUILD_TOOLS set to
ON is useful for developers who touch content that ends up in the
bootstrap library.

QT_BUILD_TOOLS_WHEN_CROSSCOMPILING is deprecated in favor of
QT_FORCE_BUILD_TOOLS.

[ChangeLog][CMake] QT_BUILD_TOOLS_WHEN_CROSSCOMPILING has been
deprecated in favor of QT_FORCE_BUILD_TOOLS. The latter can be used in
combination with QT_FORCE_FIND_TOOLS and QT_HOST_PATH to use tools from
a host Qt even for a non-cross build and still build the tools.

Fixes: QTBUG-99683
Change-Id: I0e5f6bec596a4a78bd3bfffd16c8fb486181f9b6
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
2022-05-04 15:20:35 +02:00
Joerg Bornemann
687f22671f CMake: Fix typo in error message
Pick-to: 6.2 6.3
Change-Id: Iace4fe19c0bdbcb61f667363d86b22abf6ec7d24
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-05-02 23:24:41 +02:00
Alexandru Croitor
251033fb0a CMake: Work around build rpath issue when CMAKE_STAGING_PREFIX is set
CMake has logic to rewrite build rpaths that contain
CMAKE_STAGING_PREFIX to instead point to CMAKE_INSTALL_PREFIX.

This breaks running executables from the build directory, because
their build rpath will point to a location where the libraries might
not exist yet (we didn't install Qt yet).

Work around this by setting CMAKE_STAGING_PREFIX to a fake path, so
that CMake does not do the rewriting anymore.

CMAKE_STAGING_PREFIX needs to be set at subdirectory scope, not
function scope, which is why
qt_internal_apply_staging_prefix_build_rpath_workaround() is a macro
that is called from within each Qt internal function that creates
a target.

The workaround can be disabled by configuring with
-DQT_NO_STAGING_PREFIX_BUILD_RPATH_WORKAROUND=ON

The downside of this workaround is that it breaks per-subdirectory
install rules like 'ninja src/gui/install'.
Regular global installation like 'ninja install' works fine.

This is similar to what we do for tests in
qt_set_up_fake_standalone_tests_install_prefix()
introduced by 20292250d44e08437306096e9096fc655cc9fb8b
The reason it's not as good for other target types is because in
contrast to tests, we do want to install them.

In case if someone does call `ninja src/gui/install' they will most
likely get a permission error, telling them it's not possible to
install into
/qt_fake_staging_prefix/
 check_qt_internal_apply_staging_prefix_build_rpath_workaround

Fixes: QTBUG-102592
Change-Id: I6ce78dde1924a8d830ef5c62808ff674c9639d65
Reviewed-by: Jörg Bornemann <joerg.bornemann@qt.io>
2022-04-22 14:34:47 +02:00
Joerg Bornemann
006224eb17 CMake: Fix cross-build without qt.toolchain.cmake
If there's no qt.toolchain.cmake used, QT_HOST_PATH_CMAKE_DIR is not
set.  Use the location calculated from the Qt6HostInfo package in that
case.

This amends commit e044c94a61f0cd2bdea1e89be4ec3c68007f7a5c.

Pick-to: 6.2 6.3
Change-Id: I99aed8e920b8c1aa6efd8f18301cc34aca5559ca
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
2022-02-18 13:44:40 +01:00
Joerg Bornemann
e044c94a61 CMake: Introduce QT_ADDITIONAL_HOST_PACKAGES_PREFIX_PATH
When installing conan packages we have one installation prefix per
package.  When cross-building a conan package, we need to make known
those multiple host prefixes to Qt, analogous to
QT_ADDITIONAL_PACKAGES_PREFIX_PATH.

Pick-to: 6.2 6.3
Fixes: QTBUG-94524
Change-Id: I68cbede350f95266a5fd3559e9b9c214c5858eed
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2022-02-15 14:48:20 +01:00
Alexandru Croitor
c82bca9efe CMake: Fix forcing usage of host tools when doing a desktop build
One can now set QT_FORCE_FIND_TOOLS to ON together with passing a
QT_HOST_PATH to ensure that a new desktop build uses
already existing tools from a different Qt host (desktop) installation.

Depends on a0e56294c1e80f34147c5a992b314776e1b6c757
to work, which is not in 6.2, but this change is still included in
6.2 because it cleans up the conditions a bit to make them more clear.

Amends 42d3b21c92525ea6a430c67e577a5991d679fa1d
and 5a779a4ad350accadc4337d332eedb29ba1cc26b

Pick-to: 6.2
Fixes: QTBUG-95099
Task-number: QTBUG-97658
Change-Id: If6258fb1091c6c1e457f22ae5f468b811bd20d57
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-10-29 15:34:12 +02:00
Alexandru Croitor
9ed274a48d CMake: Enforce lookup of host Tools packages in qt_internal_add_tool
When cross-building, qt_internal_add_tool would accidentally find device
QtFooTools CMake packages as a result of calling find_package. It should
have found host Tools packages instead.

The reason was due to a combination of setting
CMAKE_FIND_ROOT_PATH_MODE_PACKAGE to BOTH, find_package preferring
the lookup of packages in CMAKE_FIND_ROOT_PATH before CMAKE_PREFIX_PATH
and there being a Tools package in the device sysroot.

Because qt_internal_add_tool didn't adjust CMAKE_FIND_ROOT_PATH to
contain the host path, the device package was picked up.

Change the implementation not to set CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
and instead modify both CMAKE_FIND_ROOT_PATH and CMAKE_PREFIX_PATH
to prefer the host packages instead.

This aligns to the behavior that was introduced in
ec90f9013b4c6b63d7e03a964f66f97329be7885
which is used in QtModuleDependencies when looking for Tools packages.

Pick-to: 6.2
Fixes: QTBUG-97599
Change-Id: I8e38284774ae97981ccfd5071465609f3de80f01
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-10-29 15:34:12 +02:00
Alexandru Croitor
5ffc744b79 CMake: Allow disabling package version check
When building Qt repos, all find_package(Qt6) calls request a
PROJECT_VERSION version which is set in .cmake.conf via
QT_REPO_MODULE_VERSION.

This means trying to configure qtsvg from a 6.3 branch using a
6.2 qtbase won't work, because qtsvg will call find_package(Qt6 6.3)
and no such Qt6 package version exists.

There are certain scenarios where it might be useful to try to do
that though.
One of them is doing Qt development while locally mixing branches.
Another is building a 6.4 QtWebEngine against a 6.2 Qt.

Allow to opt out of the version check by configuring each Qt repo
with -DQT_NO_PACKAGE_VERSION_CHECK=TRUE. This setting is not
recorded and will have to be set again when configuring another
repo.

The version check will also be disabled by default when configuring
with the -developer-build feature. This will be recorded and embedded
into each ConfigVersion file.

If the version check is disabled, a warning will be shown mentioning
the incompatible version of a package that was found but that package
will still be accepted.
The warning will show both when building Qt or using Qt in a user
project.
The warnings can be disabled by passing
-DQT_NO_PACKAGE_VERSION_INCOMPATIBLE_WARNING=TRUE

Furthermore when building a Qt repo, another warning will show when an
incompatible package version is detected, to suggest to the Qt builder
whether they want to use the incompatible version by disabling the
version check.

Note that there are no compatibility promises when using mixed
non-matching versions. Things might not work. These options are only
provided for convenience and their users know what they are doing.

Pick-to: 6.2
Fixes: QTBUG-96458
Change-Id: I1a42e0b2a00b73513d776d89a76102ffd9136422
Reviewed-by: Craig Scott <craig.scott@qt.io>
2021-10-27 16:41:26 +02:00
Joerg Bornemann
a0e56294c1 Skip unnecessary commands when cross-building tools
Introduce a new macro qt_internal_return_unless_building_tools which
simply calls return() if tools are not built.  This macro is supposed to
be called after qt_internal_add_tool().

Using this macro avoids having to special-case code for when
qt_internal_add_tool() creates imported targets in cross-builds.

Adjust pro2cmake accordingly.

Task-number: QTBUG-85084
Change-Id: I9e1c455c29535dd8c318efa890ebd739c42effc1
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
2021-09-27 16:34:23 +02:00
Martin Vejdarski
3c148323c0 CMake: Exclude dSYM INSTALL commands for already excluded tools in debug
Debug tools are excluded from the ALL target for debug_and_release
builds. However, when using the -separate-debug-info option, the same
exclusion wasn't being applied for their dSYM INSTALL commands,
resulting in a CMake install error.

Pass any additional install args like

  EXCLUDE_FROM_ALL COMPONENT "ExcludedExecutables"

to the installation rules of qt_enable_separate_debug_info and
install dSYMs for executables per-config in a multi-config build.

All the non-main config executable install rules are optional because
the non-main config executables are excluded from ALL.

Amends 5b136abd21803988f96b9b66c992822efbef97ec

Pick-to: 6.2
Fixes: QTBUG-93999
Change-Id: I95c3ce28215c3ee535551e4b7a5fa9731f8f1c28
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-08-17 02:55:12 +02:00
Alexandru Croitor
7b3d9efc04 CMake: Place internal apps in the correct output directory
In a -debug-and-release build, apps were placed under bin/Release
rather than just bin.

Apply the logic we use for tools for apps as well. Rename and move
the common functions into QtTargetHelpers.cmake.

Pick-to: 6.2
Fixes: QTBUG-95028
Change-Id: I5a9082ea50c9238c8fcf0c6dd099708fbc571bf8
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-07-09 09:44:58 +02:00
Alexandru Croitor
561fc8107f CMake: Allow promoting the Qt libraries to be global targets
User projects can set the QT_PROMOTE_TO_GLOBAL_TARGETS variable to
true so that the various imported targets created by find_package(Qt6)
are promoted to global targets.

This would allow a project to find Qt packages in a subdirectory scope
while using those Qt targets from a different scope.

E.g. it fixes errors like

  CMake Error at CMakeLists.txt:5 (target_link_libraries):
  Error evaluating generator expression:

    $<TARGET_OBJECTS:Qt6::Widgets_resources_1>

  Objects of target "Qt6::Widgets_resources_1" referenced but no such
  target exists.

when trying to use a static Qt from a sibling scope.

Various 3rd party dependency targets (like Atomic or ZLIB) are not
made global due to limitations in CMake, but as long as those targets
are not mentioned directly, it shouldn't cause issues.

The targets are made global in the generated
QtFooAdditionalTargetInfo.cmake file.

To ensure that resource object libraries promoted, the generation
of the file has to be done at the end of the defining scope
where qt_internal_export_additional_targets_file is called,
which is achieved with a deferred finalizer.

Replaced all occurrences of target promotion with a helper function
which allows tracing of all promoted targets by specifying
--log-level=debug to CMake.

Pick-to: 6.2
Fixes: QTBUG-92878
Change-Id: Ic4ec03b0bc383d7e591a58c520c3974fbea746d2
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-06-16 13:22:17 +02:00
Robert Griebl
6eea6e30c6 cmake: Allow tools to enable exceptions
Needed for the QtApplicationManager tools.

Pick-to: 6.2
Change-Id: I987b297406b6659d777ce5c00f9fc1b89ccc80ab
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
2021-06-14 18:32:47 +00:00