Previously when configuring on the first run,
when qt_configure_process_path was called with the default "./plugins"
for INSTALL_PLUGINS, we would write that value directly to
qconfig.cpp.
After a build, if we reconfigured qtbase, the function would then
canonicalize the path via file(RELATIVE_PATH), and write 'plugins' to
qconfig.cpp, which would cause unnecessary rebuilds.
Make sure we canonicalize the path on the first configuration as well,
to avoid the rebuilds.
Simplify the code a bit, and fix a drive-by where we set rel_path to
"." before, but never actually set the cache variable in that case.
Amends 48dbcefe57860f70e9bc4859983d2596634ea8f3
Amends c269d8f0862fd2c581d57584e8d7e2493f387ee7
Change-Id: I8749a85946e93cdf8672113638b499d0d3a31e5c
Pick-to: 6.7 6.8
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This change adds a new -sbom configure option to allow generating and
installing an SPDX v2.3 SBOM file when building a qt repo.
The -sbom-dir option can be used to configure the location where
each repo sbom file will be installed.
By default it is installed into
$prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx
which is basically ~/Qt/sbom/qtbase-6.8.0.spdx
The file is installed as part of the default installation rules, but
it can also be installed manually using the "sbom" installation
component, or "sbom_$lower_project_name" in a top-level build. For
example: cmake install . --component sbom_qtbase
CMake 3.19+ is needed to read the qt_attribution.json files for
copyrights, license info, etc. When using an older cmake version,
configuration will error out. It is possible to opt into using an
older cmake version, but the generated sbom will lack all the
attribution file information.
Using an older cmake version is untested and not officially supported.
Implementation notes.
The bulk of the implementation is split into 4 new files:
- QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing
and dispatching the generation of various pieces of the SBOM document
e.g. a SDPX package associated with a target like Core, a SDPX
file entry for each target binary file (per-config shared library,
archive, executable, etc)
- QtPublicSbomGenerationHelpers.cmake - for non-Qt specific
implementation of SPDX generation. This also has some code that was
taken from the cmake-sbom 3rd party project, so it is dual licensed
under the usual Qt build system BSD license, as well as the MIT
license of the 3rd party project
- QtPublicGitHelpers.cmake - for git related features, mainly to embed
queried hashes or tags into version strings, is dual-licensed for
the same reasons as QtPublicSbomGenerationHelpers.cmake
- QtSbomHelpers.cmake - Qt-specific functions that just forward
arguments to the public functions. These are meant to be used in our
Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones
for naming consistency. These function would mostly be used to
annotate 3rd party libraries with sbom info and to add sbom info
for unusual target setups (like the Bootstrap library), because most
of the handling is already done automatically via
qt_internal_add_module/plugin/etc.
The files are put into Public cmake files, with the future hope of
making this available to user projects in some capacity.
The distinction of Qt-specific and non-Qt specific code might blur a
bit, and thus the separation across files might not always be
consistent, but it was best effort.
The main purpose of the code is to collect various information about
targets and their relationships and generate equivalent SPDX info.
Collection is currently done for the following targets: Qt modules,
plugins, apps, tools, system libraries, bundled 3rd party libraries
and partial 3rd party sources compiled directly as part of Qt targets.
Each target has an equivalent SPDX package generated with information
like version, license, copyright, CPE (common vulnerability
identifier), files that belong to the package, and relationships on
other SPDX packages (associated cmake targets), mostly gathered from
direct linking dependencies.
Each package might also contain files, e.g. libQt6Core.so for the Core
target. Each file also has info like license id, copyrights, but also
the list of source files that were used to generate the file and a
sha1 checksum.
SPDX documents can also refer to packages in other SPDX documents, and
those are referred to via external document references. This is the
case when building qtdeclarative and we refer to Core.
For qt provided targets, we have complete information regarding
licenses, and copyrights.
For bundled 3rd party libraries, we should also have most information,
which is usually parsed from the
src/3rdparty/libfoo/qt_attribution.json files.
If there are multiple attribution files, or if the files have multiple
entries, we create a separate SBOM package for each of those entries,
because each might have a separate copyright or version, and an sbom
package can have only one version (although many copyrights).
For system libraries we usually lack the information because we don't
have attribution files for Find scripts. So the info needs to be
manually annotated via arguments to the sbom function calls, or the
FindFoo.cmake scripts expose that information in some form and we
can query it.
There are also corner cases like 3rdparty sources being directly
included in a Qt library, like the m4dc files for Gui, or PCRE2 for
Bootstrap.
Or QtWebEngine libraries (either Qt bundled or Chromium bundled or
system libraries) which get linked in by GN instead of CMake, so there
are no direct targets for them.
The information for these need to be annotated manually as well.
There is also a distinction to be made for static Qt builds (or any
static Qt library in a shared build), where the system libraries found
during the Qt build might not be the same that are linked into the
final user application or library.
The actual generation of the SBOM is done by file(GENERATE)-ing one
.cmake file for each target, file, external ref, etc, which will be
included in a top-level cmake script.
The top-level cmake script will run through each included file, to
append to a "staging" spdx file, which will then be used in a
configure_file() call to replace some final
variables, like embedding a file checksum.
There are install rules to generate a complete SBOM during
installation, and an optional 'sbom' custom target that allows
building an incomplete SBOM during the build step.
The build target is just for convenience and faster development
iteration time. It is incomplete because it is missing the installed
file SHA1 checksums and the document verification code (the sha1 of
all sha1s). We can't compute those during the build before the files
are actually installed.
A complete SBOM can only be achieved at installation time. The install
script will include all the generated helper files, but also set some
additional variables to ensure checksumming happens, and also handle
multi-config installation, among other small things.
For multi-config builds, CMake doesn't offer a way to run code after
all configs are installed, because they might not always be installed,
someone might choose to install just Release.
To handle that, we rely on ninja installing each config sequentially
(because ninja places the install rules into the 'console' pool which
runs one task at a time).
For each installed config we create a config-specific marker file.
Once all marker files are present, whichever config ends up being
installed as the last one, we run the sbom generation once, and then
delete all marker files.
There are a few internal variables that can be set during
configuration to enable various checks (and other features) on the
generated spdx files:
- QT_INTERNAL_SBOM_VERIFY
- QT_INTERNAL_SBOM_AUDIT
- QT_INTERNAL_SBOM_AUDIT_NO_ERROR
- QT_INTERNAL_SBOM_GENERATE_JSON
- QT_INTERNAL_SBOM_SHOW_TABLE
- QT_INTERNAL_SBOM_DEFAULT_CHECKS
These use 3rd party python tools, so they are not enabled by default.
If enabled, they run at installation time after the sbom is installed.
We will hopefully enable them in CI.
Overall, the code is still a bit messy in a few places, due to time
constraints, but can be improved later.
Some possible TODOs for the future:
- Do we need to handle 3rd party libs linked into a Qt static library
in a Qt shared build, where the Qt static lib is not installed, but
linked into a Qt shared library, somehow specially?
We can record a package for it, but we can't
create a spdx file record for it (and associated source
relationships) because we don't install the file, and spdx requires
the file to be installed and checksummed. Perhaps we can consider
adding some free-form text snippet to the package itself?
- Do we want to add parsing of .cpp source files for Copyrights, to
embed them into the packages? This will likely slow down
configuration quite a bit.
- Currently sbom info attached to WrapFoo packages in one repo is
not exported / available in other repos. E.g. If we annotate
WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be
available when looking up WrapZLIB in qtimageformats.
This is because they are IMPORTED libraries, and are not
exported. We might want to record this info in the future.
[ChangeLog][Build System] A new -sbom configure option can be used
to generate and install a SPDX SBOM (Software Bill of Materials) file
for each built Qt repository.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Qt already runs on Vision Pro as "Designed for iPad", using Qt
for iOS. This change enables building Qt for visionOS directly,
which opens the door to visionOS specific APIs and use-cases
such as volumes and immersive spaces.
The platform removes some APIs we depend on, notably UIScreen,
so some code paths have been disabled or mocked to get something
up and running.
As our current window management approach on UIKit platforms
depends on UIWindow and UIScreen there is currently no way to
bring up QWindows. This will improve once we refactor our
window management to use window scenes.
To configure for visionOS, pass -platform macx-visionos-clang,
and optionally add -sdk xrsimulator to build for the simulator.
Change-Id: I4eda55fc3fd06e12d30a188928487cf68940ee07
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Introduce a new libexec/qt-internal-configure-examples script that
allows to configure and build "standalone examples" just like
"standalone tests".
This is a prerequisite for using deployment api in examples for prefix
builds, otherwise deployment api gets confused not finding various
information that it expects from an installed qt.
Because the various conditions in the build system for standalone
examples are similar to standalone tests, introduce a new
QT_BUILD_STANDALONE_PARTS variable and use that in the conditions.
The variable should not be set by the user, and is instead set by the
build system whenever QT_BUILD_STANDALONE_TESTS/EXAMPLES is set.
Unfortunately due to no common file being available before the first
project() call, in qtbase builds, per-repo builds and top-level builds,
we need to duplicate the code for setting QT_BUILD_STANDALONE_PARTS for
all three cases.
Task-number: QTBUG-90820
Task-number: QTBUG-96232
Change-Id: Ia40d03a0e8f5142abe5c7cd4ff3000df4a5f7a8a
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Qt CI Bot <qt_ci_bot@qt-project.org>
Before this, when trying to build iOS tests, CMake could not find the
`LaunchScreen.storyboard` in the source directory, and therefore the
configuration was failing.
In addition, QtPublicFinalizerHelpers module was missing in QtBuild,
resulting in another configuration failure when configuring iOS tests.
Pick-to: 6.7
Change-Id: I592121892a2716973a92ec044414fa729fd3b15f
Reviewed-by: Alexandru Croitor <alexandru.croitor@qt.io>
Split all code in QtSetup into separate functions and macros, put them
in more appropriate files, and call them in
qt_internal_setup_build_and_global_variables.
A new QtBuildOptionsHelpers.cmake is created which takes care of
computing the default values of user-customizable options that are not
pure configure features, like the cmake build type, whether to build
tests / examples, whether to enable ccache, etc.
The new function calls added in
qt_internal_setup_build_and_global_variables
try to preserve the previous code flow when QtBuild was included
in-between the code that was run in QtSetup.
Macros that have dependencies on various global variables were marked
as such with inline comments for easier navigation and comprehension.
After this change, QtSetup.cmake just includes QtBuild.cmake. We leave
it to exist for easier git blaming, but new code should not be added
to it unless really necessary.
The intent is to merge a variant of this change to 6.6 and 6.5 as
well.
Task-number: QTBUG-86035
Change-Id: I3409c2d3ea8ee19a69104b12ab2692966ba5f9cf
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
My motivation to do this:
- it got big and tangled again
- sometimes functions need to be added to QtBuild.cmake rather than
to a separate file because they need to be called before some of the
global variables are set, to determine the value of those global
variables (in my case install paths needed to be modified when
building with xcframework support)
- some of the global variable assignments have dependencies on other
variables already being set and it's hard to keep track where that
happens
Split the contents of the file into smaller functions and macros
and place them into pre-existing files when appropriate, or
into new files. The new files are:
- QtBuildHelpers.cmake
- QtBuildPathsHelpers.cmake
- QtMkspecHelpers.cmake
The idea is to have Helpers file only define functions and never call
them, so it's easy to include the file where needed without being
scared of side effects.
QtBuild.cmake will just include the helpers and call one entry point
function to set up everything that was done by the file before.
QtBuild.cmake is not merged into QtSetup, to make it easier to git
blame (it's hard to blame a removed file).
No new features were added as part of the refactoring.
Some function names were renamed (but not all of them) to include
the qt_internal prefix.
Some lines were reformatted so they don't pass 100 chars limit after
the code was placed into a function / macro.
The Helpers includes were re-sorted.
Some function calls were re-ordered where the order call didn't
matter.
Some of the code in QtAndroidHelpers.cmake was wrapped into a macro
so that including the file does not cause side-effects by default.
I'd like to follow up with similar changes for QtSetup.cmake and
QtBuildInternalsConfig.cmake where possible, because having a few
"entry points" into building a Qt submodule is also confusing,
especially for those that aren't familiar with the build system and
why certain things go into certain places.
The intent is to cherry-pick this also to 6.5 and 6.6.
Amends 44cce1a2ea9dadd8b2de93f40de34269dda703c0
Task-number: QTBUG-86035
Change-Id: I02ceff8ceb9b6e9c78bc85d6a42deb02fca3e46b
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Orkun Tokdemir <orkun.tokdemir@qt.io>
Alexandru Croitor
Tor Arne Vestbø
Amir Masoud Abdol