From 3a9cb4c1d7a4f577bb540dcda28491a0c31a4663 Mon Sep 17 00:00:00 2001 From: Anson Chung Date: Fri, 27 May 2022 11:30:24 -0700 Subject: [PATCH] Fix Ninja builds on Gitlab-CI by limiting parallelism In previous versions it was stated that MDEV-25968 was causing other jobs in the pipeline to fail if not run with "-j 2" but this bug was not affecting fedora-ninja. This is still true for the public gitlab runners. However, running the fedora-ninja job on custom runners with more processors without the "-j 2" flag will cause the compiler to crash. When running the build with 2,4,8,16,32 threads, build times were consistent indicating that the typical bottleneck is I/O and not CPU cores. Therefore, "-j 2" is not a big drawback and it was chosen in order to remain consistent with the other builds affected by MDEV-25968. All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc. --- .gitlab-ci.yml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6db89cd332f..2254164da4b 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -118,8 +118,10 @@ fedora-ninja: - mkdir builddir; cd builddir - cmake -DRPM=generic $CMAKE_FLAGS -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON -G Ninja .. 2>&1 | tee -a ../build-$CI_JOB_NAME-$CI_COMMIT_REF_SLUG.log - ninja -t graph > ../dependencies.dot && dot -Tpng -o ../dependencies.png ../dependencies.dot - - eatmydata ninja package --verbose 2>&1 | tee -a ../build-$CI_JOB_NAME-$CI_COMMIT_REF_SLUG.log - # Ninja builds are not affected by bug https://jira.mariadb.org/browse/MDEV-25968 + - eatmydata ninja package -j 2 --verbose 2>&1 | tee -a ../build-$CI_JOB_NAME-$CI_COMMIT_REF_SLUG.log + # @TODO: Unlike other builds, the Ninja builds using Gitlab.com runners don't get stuck, but they do get + # stuck on runners with more processors, see https://jira.mariadb.org/browse/MDEV-25968. + # Thus, use the same limitation on Ninja builds as well to ensure it never gets stuck due to this bug. - ninja test - *rpm_listfiles - mkdir ../rpm; mv *.rpm ../rpm