The external doc references need to be appended as strings, not list
elements, otherwise there will be semicolons in the output SPDX file.
This issue was inherited as-is from the upstream SBOM generation
project.
We are fortunate the json converted files are missing the semicolons
due to implementation specifics.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: I2b91775d59d64709b72ff7130dd4f6b506ef8244
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
For build time sboms to be found in a no-prefix qt build of a repo,
specifically when finding external sbom documents, the build time
sboms have to be in the same location.
Previously they were in each repo's build dir.
Align the location to be in the qtbase build dir, just like for super
builds.
Also check for QT_BUILDING_QT because we don't want to apply the logic
for QT_WILL_INSTALL not being set in user projects.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: I04a066fc0efc91a4f2ee938c190b8632c8c946aa
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Add a few internal functions to allow
- getting the sbom project supplier
- project supplier url
- project namespace
- computing a project namespace
- computing a project file name
- getting a sanitized spdx id given a hint
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: I0dc3df274eaf6882a6af021aabee75501b5083f8
Reviewed-by: Moss Heim <moss.heim@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Replaces content like timestamps, version strings and checksums with
constant values to make the SBOM generation deterministic.
This is useful for reproducible builds, to allow cleaner inter-diffs
while developing new SBOM features.
Can be enabled by configuring with
-DQT_SBOM_FAKE_DETERMINISTIC_BUILD=ON
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: Id0003bdd23fe57abd70213a2a108885861693242
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Because it's in a Public cmake file.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: If3f92af848193cb824b88e93e17965ffed93cf2b
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Initially the git vars were assigned to the parent scope of the
_qt_internal_sbom_begin_project function, with the intent to set them
in the global scope. But the function was later wrapped in other
functions, so the variables stopped being accessible.
Instead of playing with recursive PARENT_SCOPEs, save the variables in
global properties like we do for other info, and use a new
_qt_internal_sbom_get_git_version_vars() function to query the vars in
the code that needs them.
This fixes generated purls to contain the git version and hashes.
Also add a new internal API wrapper macro called
qt_internal_sbom_get_git_version_vars to allow calling it
in other repos.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: I061b34f418c1ecc1c66c8c01ef758d2f40611ede
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
The SBOM implementation got somewhat large. Split the code into
several new QtPublicSbomFooHelpers.cmake files, to make it more
manageable.
No code or behavior was changed.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Change-Id: Ia0ca1792eec21d12c4bb4cabe63279e1f5c07e3d
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Add a new qt_internal_sbom_generate_tag_value_spdx_document function
that takes an input SPDX JSON file and generates a tag/value SPDX
file. This is needed by WebEngine to convert the Chromium JSON file to
a tag/value SPDX file so we can reference it as an external document.
To ensure the external document is found, we now always add the
current sbom build directories as install prefixes. This was
previously done only for top-level builds.
To ensure the converted external document is referenced only after it
is converted, it needs to be converted before any targets mention
packages from within it.
A sample usage might be:
qt_internal_sbom_generate_tag_value_spdx_document(
OPERATION_ID qt5compat
INPUT_JSON_FILE_PATH "${external_sbom_file_path}"
OUT_VAR_OUTPUT_FILE_NAME external_output_file_name
OUT_VAR_OUTPUT_ABSOLUTE_FILE_PATH external_output_file_path
)
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTBUG-129901
Task-number: QTBUG-131377
Change-Id: I5d5397f788c8c7960b6fc233c2868244e5816e0b
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Add a new SBOM_RELATIONSHIPS option to qt_internal_extend_target and
friends that allows adding custom relationships to the current target.
Add a new function qt_internal_sbom_add_project_relationship that
allows adding custom relationships to the current project SBOM
document.
A sample usage might be:
qt_internal_sbom_get_project_spdx_id(project_spdx_id)
qt_internal_sbom_get_target_spdx_id(Svg svg_spdx_id)
qt_internal_extend_target(Svg
SBOM_RELATIONSHIPS
"${svg_spdx_id} DESCENDANT_OF ${project_spdx_id}"
)
qt_internal_sbom_add_project_relationship(
RELATIONSHIPS
"${svg_spdx_id} CONTAINS NOASSERTION"
"${svg_spdx_id} DESCRIBES NOASSERTION"
"${project_spdx_id} DESCRIBES NOASSERTION"
)
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTBUG-129901
Task-number: QTBUG-131377
Change-Id: Ie0119ca71b047c7515e1deaf84a5a67ea01b5274
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
We shouldn't be using the direct project name in spdx relationships,
but rather the sanitized name which is prefixed with
'SPDXRef-Package-', to ensure sbom validation succeeds.
Also we should default PROJECT_FOR_SPDX_ID to
"Package-${arg_PROJECT}" if it's not set.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTBUG-129901
Task-number: QTBUG-131377
Change-Id: I354babcf4fea5f6efd9b32422dd8d3835ef50f15
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
This change modifies the existing implementation of the
external references API to have better named options and better
behavior.
It also adds comments and exposes an internal API wrapper around the
existing implementation.
It's meant to be used by qtwebengine to reference the
Chromium-generated SBOM.
As a drive-by, it removes the previously unused RENAME option.
A sample usage could be:
qt_internal_sbom_get_external_document_ref_spdx_id(
"qt5compat" document_ref_spdx_id)
qt_internal_sbom_add_external_reference(
EXTERNAL_DOCUMENT_FILE_PATH "/path/to/qt5compat-6.8.1.spdx.json"
EXTERNAL_DOCUMENT_SPDX_ID "${document_ref_spdx_id}"
)
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTBUG-129901
Task-number: QTBUG-131377
Change-Id: I13d0fe4d803449bec42f8b454c8131e4d727669a
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Reviewed-by: Moss Heim <moss.heim@qt.io>
Allow including custom cmake files at predefined steps of the sbom
generation using the new internal
qt_internal_sbom_add_cmake_include_step API.
This covers pre-existing steps like BEGIN, END, POST_GENERATION, and
VERIFY, as well as two new steps, BEFORE_CHECKSUM and AFTER_CHECKSUM.
A sample usage of the new API could be:
set(step_path "${CMAKE_CURRENT_BINARY_DIR}/run_some_cmake.cmake")
file(WRITE "${step_path}"
"
# This is a CMake script to run some CMake code.
message(STATUS \"Running some CMake code...\")
message(STATUS \"Done running some CMake code.\")
"
)
foreach(step IN ITEMS BEGIN END BEFORE_CHECKSUM
AFTER_CHECKSUM POST_GENERATION VERIFY)
qt_internal_sbom_add_cmake_include_step(
STEP "${step}"
INCLUDE_PATH "${step_path}"
)
endforeach()
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTBUG-129901
Task-number: QTBUG-131377
Change-Id: I6f9512bf3a3bc3ebeb7d21426e5a3833280e42a5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Previously Python and other sbom tooling was looked up at SBOM
project end time.
This was fine for the use cases we had so far, but it prevents
introducing new API that wants call the spdx tools before the end of
the project. Such API will be introduced in a follow up change.
Move the python interpreter, python dependency lookup and other sbom
tooling setup to happen at project begin time.
Pick-to: 6.8 6.9
Task-number: QTBUG-122899
Task-number: QTBUG-129901
Task-number: QTBUG-131377
Change-Id: Ic8884e378c0ffd9720ede26b0c61f4122f3bb9d9
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
When installing a top-level build using the DESTDIR env var mechanism,
the SBOM generation process could not find the qtbase sbom document,
while generating the qtshadertools document.
Make sure to prepend the DESTDIR env var to the install-time install
prefix, when looking for external SBOM documents.
Pick-to: 6.8 6.9
Fixes: QTBUG-132188
Change-Id: I1cdb75842fc552b43d982f05444b5ddc1fe58595
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Certain repositories like qtwebengine contain multiple projects from
the perspective of online installer packaging. In this case the
QtWebEngine and QtPdf projects are expected to have separate SBOM
documents.
Introduce a new QT_SKIP_SBOM_AUTO_PROJECT variable that can be set
before qt_build_repo to disable the auto-generation of an SBOM
document for the current repo project.
Introduce two new internal functions
qt_internal_sbom_begin/end_qt_repo_project to allow to manually start
and end the SBOM generation for a project within a repo.
Because the intermediate file names that assemble the SBOM use the
project name as a key, and the project name would be the same for
qtwebengine, allow differentiating between the current project name
and the real qt repo project name.
The current project name is used for the file names, whereas the
real qt repo project name is used to extract the dependencies on
other repos, to ensure correct dependency build rules.
As a drive-by, improve the document dir path search list when an SBOM
document can't be found.
Pick-to: 6.8
Task-number: QTBUG-128893
Task-number: QTBUG-122899
Change-Id: I61b68098242e7c49b98420265c29af78303c3233
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Before this change, the SBOM generation only considered the
CMAKE_INSTALL_PREFIX set at configure time for the purposes of file
checksum computation, external document lookup, and SBOM file
installation.
When cmake --install . --prefix <path> is used, the install time
CMAKE_INSTALL_PREFIX variable is overridden with the provided path,
and can be different from the configure time value.
This path was applied to the installation of regular files, libraries,
etc, but not to the SBOM generation. This caused issues like not being
able to find a library to compute its checksum.
Instead of hard-coding the value to QT_STAGING_PREFIX, just don't pass
a value at all, and rely on the new default of using an
install-time-evaluated \${CMAKE_INSTALL_PREFIX}.
Keep the ability of specifying a custom prefix just in case.
Modify all the code that used a hard-coded CMAKE_INSTALL_PREFIX to use
a install-time-evaluated one instead.
As a drive-by, also set a proper default value for the INSTALL_SBOM_DIR
option.
Pick-to: 6.8
Fixes: QTBUG-131883
Change-Id: Ifde6ab282ac40f10c5bf51976121065c7dc631eb
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
It causes the SBOM to be non-reproducible, as the path is different on
each machine.
Embedding just the compiler id and version should be sufficient.
Pick-to: 6.8
Task-number: QTBUG-122899
Task-number: QTBUG-130557
Change-Id: Iad66ac3f90c4969d27307c77bef46d2eba15c175
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Previously SBOM generation was opt-in.
This patch changes the generation of the plain-text tag:value SBOM
to be enabled by default, except for:
- developer builds
- no-prefix builds
- standalone tests or examples
- cmake build tests
The JSON SBOM generation and the verification steps have also been
changed to be enabled by default, but only if the Python dependencies
can be found. If the dependencies are not found, the build will
skip the generation and verification steps.
Four new configure options have been added to control these aspects:
-(no-)sbom-json: Allows explicitly enabling or disabling JSON SBOM
generation
-(no-)sbom-json-required: Fails the build if JSON SBOM generation
Python dependencies are not found
-(no-)sbom-verify: Allows explicitly enabling or disabling SBOM
verification
-(no-)sbom-verify-required: Fails the build if SBOM verification
Python dependencies are not found
There are corresponding CMake variables for each of the configure
options, see the cmake mapping document.
[ChangeLog][Build Systems] SBOM generation is now enabled by default,
when building Qt, except for developer builds and no-prefix builds.
JSON SBOM generation is enabled by default if the required Python
dependencies are available.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I6dbe1869f8342154a89ff2ab84ac53c9ef1b2eb7
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
This will allow us to run only the first, but not the second, if the
second won't have it's dependencies met.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I141b4bd3b76a71495c760a118bdf1397ee7e16b5
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Introduce a bunch of helper functions to split out the finding of a
specific python interpreter, a specific python dependency within a
given python interpreter, and the handling of dependencies for each of
the possible SBOM operations: GENERATE_JSON, VERIFY_SBOM, and
RUN_NTIA.
In a future change this will allow us to conditionally enable certain
operations, depending on which dependencies are available, without
failing the build outright.
One behavior change is better logging of what goes wrong if a
dependency is missing.
Another is the double look up of python on macOS, to account for the
system framework python not having the required dependencies.
These was previously an internal opt in, but now it is done
automatically.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I3054f2649c0092dc5f2d3e299065f0f62dc6c5fb
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
The DESTDIR env var needs to be evaluated at install time, not
configure time, so the $ENV{} needs to be escaped in the generated
script.
Also improve the error message a bit, to say which path exactly it
expected to read.
This fixes SBOM generation for the debian package builds.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I6869f0054601dcb35ae7ad7b55f808df4dc12311
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Add code to generate and install a source SBOM SPDX file for every
repo. It relies on the python 'reuse' tool being installed and
available in PATH.
Also add code to allow running 'reuse lint', which checks compliance
with the reuse specification.
The features are only enabled when configuring with
-DQT_GENERATE_SBOM=ON
-DQT_GENERATE_SOURCE_SBOM=ON
-DQT_LINT_SOURCE_SBOM=ON
which will be the case for our CI in a follow up patch.
Because most of our repos are not yet reuse compliant, the actual
generation of the source SBOM and the linting is skipped if the
project root directory does not contain a REUSE.toml file.
This allows incremental handling of each repository, while also
enforcing the compliance at installation time when the REUSE.toml file
is actually there.
The source SBOM generation and linting will run at installation time,
but they can also be manually triggered at build time using the
ninja 'sbom' and 'reuse_lint' custom targets.
Various opt outs are provided as a fail safe:
- QT_FORCE_SOURCE_SBOM_GENERATION to force source sbom generation
even if a REUSE.toml file is not present in the root source dir
- QT_FORCE_REUSE_LINT_ERROR to force linting to error out, even if
a REUSE.toml file is not present
- QT_FORCE_SKIP_REUSE_LINT_ON_INSTALL to skip linting at installation
time, but allow running it at build time
These can be set either locally or conditionally passed to CMake
inside repo-specific Coin instructions.
Pick-to: 6.8
Task-number: QTBUG-122899
Task-number: QTBUG-125211
Change-Id: I664e69830936c4427688143ee86b98782c1733ab
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
Some operations might want to install additional files next to the
repo sbom, with the same file base name as the repo sbom file name.
E.g install qtbase.source.spdx.json next to qtbase.spdx.json.
Make the output file path without the extension available in a new
QT_SBOM_OUTPUT_PATH_WITHOUT_EXT variable during installation.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I0b9442cffa3f3b0fd2387e77569a94e43ce387a5
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
It triggers vulnerabilities when loading the SBOM into tools that look
for CVEs matching those CPEs.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: Ic4949ca5749ce4e297344760d3500d982c9a4449
Reviewed-by: Kai Köhne <kai.koehne@qt.io>
We need to pass all dirs where sboms will be generated, for external
document referencing.
We also need to set up dependencies between the repo sbom custom
targets, so that qtsvg sbom is only run after qtbase sbom.
We use the dependencies.yaml info to set up the dependencies.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: Id3331e11742bc2c86e7ed52ce26b3ff21eace359
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Repositories like qtwebengine and qtinterfaceframework might search
for a Python interpreter earlier than _qt_internal_sbom_find_python is
called and might find an older version of Python that is not
sufficient for SBOM purposes.
In that case Python3_EXECUTABLE would have been set, we would not try
to look for Python 3.9 and then fail trying to find python
dependencies or when executing one of the SBOM steps.
Rework the lookup process to be contained in a function rather than a
macro, make sure we temporarily reset any found python executable so
far and explicitly look for Python 3.9 after which we cache it into
custom cache var.
This allows finding and using a Python interpreter for SBOM purposes,
without interference from a possibly older version already found for a
different purpose.
Because the macro is now a function, we can get rid of some of the
backup state management code.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: Ia846843a4e5cc8c91963ea76557639fc19e25913
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
If the VerifySBOM feature is enabled, pass additional options to
CMake configuration:
- runs syntactic verification of the SBOM
- so it converts the generated SBOM files into JSON files (and
installs them). This does additional useful validations.
- runs the NTIA SBOM verifier
- shows some of the content in the SBOM in a more user-friendly table
format via the sbom2doc python app
- runs sbomaudit to show things like packages without a license
expression, doesn't exit with errors if issues are found.
To ensure the tables are wide enough in the log on the CI, we set
an explicit value for COLUMNS env var, which is used by sbom2doc
to determine the table size.
To ensure the sbom2doc and sbomaudit python applications are found,
we supply additional locations where they can be found, via the
env vars that coin python provisioning sets.
We also make sure to pass the found application paths when executing
the python apps, because they might not be in PATH by default.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I0baef8b9c949209b15ab304e1e840b4dcdf5a61c
Reviewed-by: Toni Saario <toni.saario@qt.io>
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
This change adds a new -sbom configure option to allow generating and
installing an SPDX v2.3 SBOM file when building a qt repo.
The -sbom-dir option can be used to configure the location where
each repo sbom file will be installed.
By default it is installed into
$prefix/$archdatadir/sbom/$sbom_lower_project_name.sdpx
which is basically ~/Qt/sbom/qtbase-6.8.0.spdx
The file is installed as part of the default installation rules, but
it can also be installed manually using the "sbom" installation
component, or "sbom_$lower_project_name" in a top-level build. For
example: cmake install . --component sbom_qtbase
CMake 3.19+ is needed to read the qt_attribution.json files for
copyrights, license info, etc. When using an older cmake version,
configuration will error out. It is possible to opt into using an
older cmake version, but the generated sbom will lack all the
attribution file information.
Using an older cmake version is untested and not officially supported.
Implementation notes.
The bulk of the implementation is split into 4 new files:
- QtPublicSbomHelpers.cmake - for Qt-specific collecting, processing
and dispatching the generation of various pieces of the SBOM document
e.g. a SDPX package associated with a target like Core, a SDPX
file entry for each target binary file (per-config shared library,
archive, executable, etc)
- QtPublicSbomGenerationHelpers.cmake - for non-Qt specific
implementation of SPDX generation. This also has some code that was
taken from the cmake-sbom 3rd party project, so it is dual licensed
under the usual Qt build system BSD license, as well as the MIT
license of the 3rd party project
- QtPublicGitHelpers.cmake - for git related features, mainly to embed
queried hashes or tags into version strings, is dual-licensed for
the same reasons as QtPublicSbomGenerationHelpers.cmake
- QtSbomHelpers.cmake - Qt-specific functions that just forward
arguments to the public functions. These are meant to be used in our
Qt CMakeLists.txt instead of the public _qt_internal_add_sbom ones
for naming consistency. These function would mostly be used to
annotate 3rd party libraries with sbom info and to add sbom info
for unusual target setups (like the Bootstrap library), because most
of the handling is already done automatically via
qt_internal_add_module/plugin/etc.
The files are put into Public cmake files, with the future hope of
making this available to user projects in some capacity.
The distinction of Qt-specific and non-Qt specific code might blur a
bit, and thus the separation across files might not always be
consistent, but it was best effort.
The main purpose of the code is to collect various information about
targets and their relationships and generate equivalent SPDX info.
Collection is currently done for the following targets: Qt modules,
plugins, apps, tools, system libraries, bundled 3rd party libraries
and partial 3rd party sources compiled directly as part of Qt targets.
Each target has an equivalent SPDX package generated with information
like version, license, copyright, CPE (common vulnerability
identifier), files that belong to the package, and relationships on
other SPDX packages (associated cmake targets), mostly gathered from
direct linking dependencies.
Each package might also contain files, e.g. libQt6Core.so for the Core
target. Each file also has info like license id, copyrights, but also
the list of source files that were used to generate the file and a
sha1 checksum.
SPDX documents can also refer to packages in other SPDX documents, and
those are referred to via external document references. This is the
case when building qtdeclarative and we refer to Core.
For qt provided targets, we have complete information regarding
licenses, and copyrights.
For bundled 3rd party libraries, we should also have most information,
which is usually parsed from the
src/3rdparty/libfoo/qt_attribution.json files.
If there are multiple attribution files, or if the files have multiple
entries, we create a separate SBOM package for each of those entries,
because each might have a separate copyright or version, and an sbom
package can have only one version (although many copyrights).
For system libraries we usually lack the information because we don't
have attribution files for Find scripts. So the info needs to be
manually annotated via arguments to the sbom function calls, or the
FindFoo.cmake scripts expose that information in some form and we
can query it.
There are also corner cases like 3rdparty sources being directly
included in a Qt library, like the m4dc files for Gui, or PCRE2 for
Bootstrap.
Or QtWebEngine libraries (either Qt bundled or Chromium bundled or
system libraries) which get linked in by GN instead of CMake, so there
are no direct targets for them.
The information for these need to be annotated manually as well.
There is also a distinction to be made for static Qt builds (or any
static Qt library in a shared build), where the system libraries found
during the Qt build might not be the same that are linked into the
final user application or library.
The actual generation of the SBOM is done by file(GENERATE)-ing one
.cmake file for each target, file, external ref, etc, which will be
included in a top-level cmake script.
The top-level cmake script will run through each included file, to
append to a "staging" spdx file, which will then be used in a
configure_file() call to replace some final
variables, like embedding a file checksum.
There are install rules to generate a complete SBOM during
installation, and an optional 'sbom' custom target that allows
building an incomplete SBOM during the build step.
The build target is just for convenience and faster development
iteration time. It is incomplete because it is missing the installed
file SHA1 checksums and the document verification code (the sha1 of
all sha1s). We can't compute those during the build before the files
are actually installed.
A complete SBOM can only be achieved at installation time. The install
script will include all the generated helper files, but also set some
additional variables to ensure checksumming happens, and also handle
multi-config installation, among other small things.
For multi-config builds, CMake doesn't offer a way to run code after
all configs are installed, because they might not always be installed,
someone might choose to install just Release.
To handle that, we rely on ninja installing each config sequentially
(because ninja places the install rules into the 'console' pool which
runs one task at a time).
For each installed config we create a config-specific marker file.
Once all marker files are present, whichever config ends up being
installed as the last one, we run the sbom generation once, and then
delete all marker files.
There are a few internal variables that can be set during
configuration to enable various checks (and other features) on the
generated spdx files:
- QT_INTERNAL_SBOM_VERIFY
- QT_INTERNAL_SBOM_AUDIT
- QT_INTERNAL_SBOM_AUDIT_NO_ERROR
- QT_INTERNAL_SBOM_GENERATE_JSON
- QT_INTERNAL_SBOM_SHOW_TABLE
- QT_INTERNAL_SBOM_DEFAULT_CHECKS
These use 3rd party python tools, so they are not enabled by default.
If enabled, they run at installation time after the sbom is installed.
We will hopefully enable them in CI.
Overall, the code is still a bit messy in a few places, due to time
constraints, but can be improved later.
Some possible TODOs for the future:
- Do we need to handle 3rd party libs linked into a Qt static library
in a Qt shared build, where the Qt static lib is not installed, but
linked into a Qt shared library, somehow specially?
We can record a package for it, but we can't
create a spdx file record for it (and associated source
relationships) because we don't install the file, and spdx requires
the file to be installed and checksummed. Perhaps we can consider
adding some free-form text snippet to the package itself?
- Do we want to add parsing of .cpp source files for Copyrights, to
embed them into the packages? This will likely slow down
configuration quite a bit.
- Currently sbom info attached to WrapFoo packages in one repo is
not exported / available in other repos. E.g. If we annotate
WrapZLIB in qtbase with CPE_VENDOR zlib, this info will not be
available when looking up WrapZLIB in qtimageformats.
This is because they are IMPORTED libraries, and are not
exported. We might want to record this info in the future.
[ChangeLog][Build System] A new -sbom configure option can be used
to generate and install a SPDX SBOM (Software Bill of Materials) file
for each built Qt repository.
Pick-to: 6.8
Task-number: QTBUG-122899
Change-Id: I9c730a6bbc47e02ce1836fccf00a14ec8eb1a5f4
Reviewed-by: Joerg Bornemann <joerg.bornemann@qt.io>
Reviewed-by: Alexey Edelev <alexey.edelev@qt.io>
