mirrors
/
mesa
mirror of https://gitlab.freedesktop.org/mesa/mesa
1
0
Fork 0
Code Issues 9 Packages Projects Releases Wiki Activity
mesa/.gitlab-ci/prepare-artifacts.sh

52 lines
1.7 KiB
Bash
Raw Normal View History

gitlab-ci: Move artifact preparation to separate script It's currently only needed for the meson-main and meson-arm64 jobs, not the other meson build jobs. Also remove MESON_SHADERDB, just run .gitlab-ci/run-shader-db.sh directly from the meson-main job. v2: * Also run prepare-artifacts.sh in meson-arm64 script v3: * Move tarball creation into the new script as well, as it prevented ccache --show-stats from running in after_script Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1 Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-23 17:42:53 +01:00
#!/bin/bash
set -e
set -o xtrace
gitlab-ci: Consolidate container and build stages for LAVA Use the normal build job to also prepare the artifacts for LAVA jobs. For that, the build container needs to also build the test suites, kernel, ramdisk, etc. Then the build job will place the just-built Mesa in the ramdisk and the test job can generate a LAVA job and point to those artifacts. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295>
2019-12-17 10:50:14 +00:00
CROSS_FILE=/cross_file-"$CROSS".txt
gitlab-ci: Delete install/bin from artifacts as well This cuts the x86 artifacts zip file size in less than half. Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-06 18:58:19 +00:00
# Delete unused bin and includes from artifacts to save space.
rm -rf install/bin install/include
gitlab-ci: Move artifact preparation to separate script It's currently only needed for the meson-main and meson-arm64 jobs, not the other meson build jobs. Also remove MESON_SHADERDB, just run .gitlab-ci/run-shader-db.sh directly from the meson-main job. v2: * Also run prepare-artifacts.sh in meson-arm64 script v3: * Move tarball creation into the new script as well, as it prevented ccache --show-stats from running in after_script Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1 Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-23 17:42:53 +01:00
# Strip the drivers in the artifacts to cut 80% of the artifacts size.
if [ -n "$CROSS" ]; then
STRIP=`sed -n -E "s/strip\s*=\s*'(.*)'/\1/p" "$CROSS_FILE"`
if [ -z "$STRIP" ]; then
echo "Failed to find strip command in cross file"
exit 1
fi
else
STRIP="strip"
fi
find install -name \*.so -exec $STRIP {} \;
# Test runs don't pull down the git tree, so put the dEQP helper
# script and associated bits there.
gitlab-ci: Upload tracie artifacts to MinIO Upload failed images and the results.yml file to MinIO, to facilitate debugging. Also, fix version checking when git is installed as Mesa is going to output a different renderer string if git is installed. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Reviewed-By: Rohan Garg <rohan.garg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5472>
2020-07-09 21:29:39 +01:00
echo "$(cat VERSION) (git-$(git rev-parse HEAD | cut -b -10))" >> install/VERSION
ci/bare-metal: Stop fetching the git tree. Like for LAVA, make the tradeoff of moving the test scripts and data (55k) into the artifacts in order to make the per-build jobs not have to pull down the git tree (hundreds of MB when you don't hit a cached container for your specific user, which I see happen multiple times a day in my CI runs). To do this, we have to be a bit more careful in some places about our working directory potentially being dirty. Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-08 23:36:16 +01:00
cp -Rp .gitlab-ci/bare-metal install/
gitlab-ci: Place files from the Mesa repo into the build tarball There's some files from the .gitlab-ci directory that are needed in the test stage and that, because the Mesa repository isn't checked out in that stage, need to be made available through other means. Because those files are going to be needed in LAVA devices, place them ino the tarball containing the built files so it's available to both gitlab-ci runners and LAVA devices. Before those files were passed in the artifacts of the Gitlab CI job, but this commit places them into the built tarball so scripts later in the pipeline don't need to account for this discrepancy. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4295>
2020-03-24 11:58:30 +00:00
cp -Rp .gitlab-ci/deqp* install/
cp -Rp .gitlab-ci/piglit install/
ci: Split traces.yml file per driver As drivers have been tested with more and more traces, the yml file is becoming a bit unwieldy. As more drivers are going to be tested with traces, and more traces will be used, split them in per-driver files so the size stays manageable. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Reviewed-By: Rohan Garg <rohan.garg@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6185>
2020-08-05 09:17:26 +01:00
cp -Rp .gitlab-ci/traces*.yml install/
gitlab-ci: Place files from the Mesa repo into the build tarball There's some files from the .gitlab-ci directory that are needed in the test stage and that, because the Mesa repository isn't checked out in that stage, need to be made available through other means. Because those files are going to be needed in LAVA devices, place them ino the tarball containing the built files so it's available to both gitlab-ci runners and LAVA devices. Before those files were passed in the artifacts of the Gitlab CI job, but this commit places them into the built tarball so scripts later in the pipeline don't need to account for this discrepancy. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4295>
2020-03-24 11:58:30 +00:00
cp -Rp .gitlab-ci/fossils.yml install/
cp -Rp .gitlab-ci/fossils install/
cp -Rp .gitlab-ci/fossilize-runner.sh install/
cp -Rp .gitlab-ci/deqp-runner.sh install/
ci/deqp-runner: Add a post-deqp-run filter list for known flakes. So far, we've been putting our known flakes that intermittently fail CI into the skips list. This has two downsides: 1) You don't know when the flakes stop happening and when to delist them from skips, unless you go do a bunch of manual runs with the skips list cleared. 2) If the flake was because the previous test left some broken state in the HW, you may just move your intermittent to a new test. With this new path, you can list your flakes in the flakes file to keep them from erroring out people's pipelines. They still get run and reported as is. Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6392>
2020-08-19 18:30:12 +01:00
cp -Rp .gitlab-ci/deqp-*.txt install/
gitlab-ci: Move artifact preparation to separate script It's currently only needed for the meson-main and meson-arm64 jobs, not the other meson build jobs. Also remove MESON_SHADERDB, just run .gitlab-ci/run-shader-db.sh directly from the meson-main job. v2: * Also run prepare-artifacts.sh in meson-arm64 script v3: * Move tarball creation into the new script as well, as it prevented ccache --show-stats from running in after_script Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1 Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-23 17:42:53 +01:00
# Tar up the install dir so that symlinks and hardlinks aren't each
# packed separately in the zip file.
gitlab-ci: Place files from the Mesa repo into the build tarball There's some files from the .gitlab-ci directory that are needed in the test stage and that, because the Mesa repository isn't checked out in that stage, need to be made available through other means. Because those files are going to be needed in LAVA devices, place them ino the tarball containing the built files so it's available to both gitlab-ci runners and LAVA devices. Before those files were passed in the artifacts of the Gitlab CI job, but this commit places them into the built tarball so scripts later in the pipeline don't need to account for this discrepancy. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Eric Anholt <eric@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4295>
2020-03-24 11:58:30 +00:00
mkdir -p artifacts/
gitlab-ci: Move artifact preparation to separate script It's currently only needed for the meson-main and meson-arm64 jobs, not the other meson build jobs. Also remove MESON_SHADERDB, just run .gitlab-ci/run-shader-db.sh directly from the meson-main job. v2: * Also run prepare-artifacts.sh in meson-arm64 script v3: * Move tarball creation into the new script as well, as it prevented ccache --show-stats from running in after_script Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1 Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-23 17:42:53 +01:00
tar -cf artifacts/install.tar install
gitlab-ci: Consolidate container and build stages for LAVA Use the normal build job to also prepare the artifacts for LAVA jobs. For that, the build container needs to also build the test suites, kernel, ramdisk, etc. Then the build job will place the just-built Mesa in the ramdisk and the test job can generate a LAVA job and point to those artifacts. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295>
2019-12-17 10:50:14 +00:00
gitlab-ci: More stable URL for kernel and ramdisk artifacts, for LAVA Place the kernel and ramdisk into a place in the file server so the URL will only change when the contents also change. Also put the Mesa build into a separate tarball so the ramdisk's contents don't change every build. With proper caching in place, all devices in the same farm need only to download the mesa tarball once, saving time. As we switch to MinIO for making kernels and rootfs available to LAVA devices, we can stop using Docker to distribute them. Instead, build when needed in separate jobs that push directly to MinIO, from where LAVA devices can download them. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5515>
2020-04-01 12:07:46 +01:00
if [ -n "$UPLOAD_FOR_LAVA" ]; then
ci: prepare-artifacts: Make the indent here match previously in the file Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3883>
2020-01-30 23:20:35 +00:00
# Pass needed files to the test stage
cp $CI_PROJECT_DIR/.gitlab-ci/generate_lava.py artifacts/.
ci: add piglit jobs to LAVA and remove tracie ones v2: - Squashed the commit to remove tracie jobs (Eric). v3: - Rename *-piglit-traces jobs with *-traces. Signed-off-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> [v2] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6388>
2020-08-27 20:18:00 +01:00
cp $CI_PROJECT_DIR/.gitlab-ci/lava-*.yml.jinja2 artifacts/.
gitlab-ci: More stable URL for kernel and ramdisk artifacts, for LAVA Place the kernel and ramdisk into a place in the file server so the URL will only change when the contents also change. Also put the Mesa build into a separate tarball so the ramdisk's contents don't change every build. With proper caching in place, all devices in the same farm need only to download the mesa tarball once, saving time. As we switch to MinIO for making kernels and rootfs available to LAVA devices, we can stop using Docker to distribute them. Instead, build when needed in separate jobs that push directly to MinIO, from where LAVA devices can download them. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5515>
2020-04-01 12:07:46 +01:00
ci: specify MinIO's host URL in a global variable Signed-off-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6388>
2020-08-23 21:32:40 +01:00
MESA_ARTIFACT=mesa-${DEBIAN_ARCH}.tar.gz
gzip -c artifacts/install.tar > ${MESA_ARTIFACT}
MINIO_PATH=${MINIO_HOST}/artifacts/${CI_PROJECT_PATH}/${CI_PIPELINE_ID}
ci: Generate MinIO credentials within LAVA jobs As these credentials are valid only for 15 minutes, generate them closer to when they are going to be used. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6124>
2020-07-30 10:09:38 +01:00
ci-fairy minio login $CI_JOB_JWT
ci: specify MinIO's host URL in a global variable Signed-off-by: Andres Gomez <agomez@igalia.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6388>
2020-08-23 21:32:40 +01:00
ci-fairy minio cp ${MESA_ARTIFACT} minio://${MINIO_PATH}/${MESA_ARTIFACT}
gitlab-ci: Consolidate container and build stages for LAVA Use the normal build job to also prepare the artifacts for LAVA jobs. For that, the build container needs to also build the test suites, kernel, ramdisk, etc. Then the build job will place the just-built Mesa in the ramdisk and the test job can generate a LAVA job and point to those artifacts. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295>
2019-12-17 10:50:14 +00:00
fi