KonstantinSeurer
/
mesa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
mesa/.gitlab-ci/common/generate-env.sh

123 lines
2.7 KiB
Bash
Raw Normal View History

Revert "ci: Convert generate-env.sh to a POSIX compliant script" This reverts commit 9904ea2c76e246629fab8bda455eec07aa84f3f9 since it is not able to properly escape all special characters (i.e. [']). The POSIX conversion is not needed anymore, as we have made 'bash' available in LAVA rootfs. Reported-by: Daniel Stone <daniels@collabora.com> Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15524>
2022-03-23 14:29:34 +00:00
#!/bin/bash
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
for var in \
ci: add ACO_DEBUG to the list of variables to pass down for testing This parameter is used by radv-ci to perform extra validation while running VKCTS. This change catches more issues, which are will be addressed in !16248: - dEQP-VK.subgroups.ballot_broadcast.compute.subgroupbroadcast_i8vec2,Crash - dEQP-VK.subgroups.ballot_broadcast.compute.subgroupbroadcast_u8vec2,Crash Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org> Suggested-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16277>
2022-05-02 14:33:35 +01:00
ACO_DEBUG \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
ASAN_OPTIONS \
ci/lava: Use per-job rootfs overlay for environment Trying to get arbitrary strings suitably quoted for shell, embedded in a YAML file, processed by Python templating, is like seven bad ideas all embedded into one big can of bees. Reuse the same script we use for bare-metal to generate the environment, tar that up into a per-job overlay which is added to the inter-pipeline-reusable rootfs built by the container jobs and the intra-pipeline-reusable overlay built by the build jobs. @anholt wrote a chunk of this - replacing the $ENV_VARS GitLab CI variable with a Python loop across the POSIX job environment - in !11192, but this still had YAML quoting nightmares, and was more needless duplication between LAVA and bare-metal. The diff is large and annoying, but is mostly a sed job to get ENV_VARS="FOO=bar BAZ=quux" into FOO: bar\nBAZ: quux. Signed-off-by: Daniel Stone <daniels@collabora.com> Co-authored-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 17:00:32 +01:00
BASE_SYSTEM_FORK_HOST_PREFIX \
BASE_SYSTEM_MAINLINE_HOST_PREFIX \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
CI_COMMIT_BRANCH \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
CI_COMMIT_REF_NAME \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
CI_COMMIT_TITLE \
CI_JOB_ID \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
CI_JOB_JWT_FILE \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
CI_JOB_NAME \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
CI_JOB_URL \
CI_MERGE_REQUEST_SOURCE_BRANCH_NAME \
CI_MERGE_REQUEST_TITLE \
CI_NODE_INDEX \
CI_NODE_TOTAL \
CI_PAGES_DOMAIN \
CI_PIPELINE_ID \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
CI_PIPELINE_URL \
ci/lava: Use per-job rootfs overlay for environment Trying to get arbitrary strings suitably quoted for shell, embedded in a YAML file, processed by Python templating, is like seven bad ideas all embedded into one big can of bees. Reuse the same script we use for bare-metal to generate the environment, tar that up into a per-job overlay which is added to the inter-pipeline-reusable rootfs built by the container jobs and the intra-pipeline-reusable overlay built by the build jobs. @anholt wrote a chunk of this - replacing the $ENV_VARS GitLab CI variable with a Python loop across the POSIX job environment - in !11192, but this still had YAML quoting nightmares, and was more needless duplication between LAVA and bare-metal. The diff is large and annoying, but is mostly a sed job to get ENV_VARS="FOO=bar BAZ=quux" into FOO: bar\nBAZ: quux. Signed-off-by: Daniel Stone <daniels@collabora.com> Co-authored-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 17:00:32 +01:00
CI_PROJECT_DIR \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
CI_PROJECT_NAME \
CI_PROJECT_PATH \
CI_PROJECT_ROOT_NAMESPACE \
CI_RUNNER_DESCRIPTION \
CI_SERVER_URL \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
CROSVM_GALLIUM_DRIVER \
CROSVM_GPU_ARGS \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
DEQP_BIN_DIR \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
DEQP_CASELIST_FILTER \
DEQP_CASELIST_INV_FILTER \
DEQP_CONFIG \
DEQP_EXPECTED_RENDERER \
DEQP_FRACTION \
DEQP_HEIGHT \
DEQP_RESULTS_DIR \
DEQP_RUNNER_OPTIONS \
ci/freedreno: Move freedreno's deqp testing to suite support. Using suites makes load-balancing our jobs much easier, keeps the CPU busy handling the a630_gles_others.sh test sets (and improves the output and baseline handling for them), and makes it trivial to add in more short test sets. a306: still 5 jobs, and we add KHR-GLES2 (KHR-GLES3 is unstable) a530: still 5 jobs, added KHR-GLES* a630_gl: 5 jobs becomes 4, and we add KHR-GLES* a630_vk: still 3 jobs, now 1/3 of all VK instead of 1/4. a630_vk_full: still 2 jobs, now includes full bypass testing, partial no-force testing, and testing of pre-merge-skipped tests. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12256>
2021-07-29 00:30:19 +01:00
DEQP_SUITE \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
DEQP_TEMP_DIR \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
DEQP_VARIANT \
DEQP_VER \
DEQP_WIDTH \
DEVICE_NAME \
DRIVER_NAME \
EGL_PLATFORM \
ci/bare-metal: add etnaviv Add deqp gles2 CI run for GC2000. Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com> Acked-by: Daniel Stone <daniels@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12852>
2020-06-12 12:23:44 +01:00
ETNA_MESA_DEBUG \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
FDO_CI_CONCURRENT \
FDO_UPSTREAM_REPO \
FD_MESA_DEBUG \
FLAKES_CHANNEL \
ci/turnip: Increase the hangcheck timer to 2 seconds. We get a lot of useful coverage from running graphicsfuzz with spilling enabled, but it's also pretty slow and can cause intermittent hangcheck failures. I thought I'd categorized them when merging !14839 (device loss on reset), but it looks like not all of them and we're now more likely to have flakes take out the whole test run when a single flake makes the rest of the caselist a flake. This is a little unfortunate in that it means our test environment is not the same as a stock system you would want to run deqp on to submit conformance, but I think it's an improvement in the test maintenance work vs needing to fix things up later. We have some other tests besides turnip that can trigger hangchecks which we might also like this increase for (some disabled traces, for example). However, freedreno GL has a 5-second timeout waiting for idle when mapping, and a couple of 2-second timeouts in a row can result in spurious failures in other tests! Fixes: #6163 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15435>
2022-03-16 22:50:00 +00:00
FREEDRENO_HANGCHECK_MS \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
GALLIUM_DRIVER \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
GALLIVM_PERF \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
GPU_VERSION \
ci: Enable testing radeonsi's libva using libva-util unit tests. We've noticed issues with these tests when uprevving Mesa in Chrome OS. This CI catches some existing failures, and some debug-build assertion failures as well. To do this, uprev deqp-runner for its new gtest-runner command. This runner is not as efficient as I would hope, due to some expensive code in gtest. I've reported the issue to gtest and it should be easily fixable, but for now it at least means we get to use the same baseline/skip/flake handling we have from deqp and piglit runners. I also fixed build-libdrm for our rootfses to not throw away libdrm's share directory, which was causing a bunch of test-time spam from radeon's libdrm when trying to look up its marketing name tables (not that big of a deal for deqp-runner, but really noisy for piglit and libva-utils which make gallium screens approximatly per-test). Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13419>
2021-10-08 21:17:27 +01:00
GTEST \
GTEST_FAILS \
GTEST_FRACTION \
GTEST_RESULTS_DIR \
GTEST_RUNNER_OPTIONS \
GTEST_SKIPS \
ci/bare-metal: Set CPU and GPU governors to max, disable GPU runtime PM Give us a bit more predictable performance by making sure we always run at full tilt. Signed-off-by: Daniel Stone <daniels@collabora.com> Acked-by: Martin Peres <martin.peres@mupuf.org> Acked-by: Emma Anholt <emma@anholt.net> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>
2021-06-11 17:09:17 +01:00
HWCI_FREQ_MAX \
ci/bare-metal: Rename BM_KERNEL_MODULES to HWCI_KERNEL_MODULES To try to make init more common. Signed-off-by: Daniel Stone <daniels@collabora.com> Acked-by: Martin Peres <martin.peres@mupuf.org> Acked-by: Emma Anholt <emma@anholt.net> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>
2021-06-11 15:40:08 +01:00
HWCI_KERNEL_MODULES \
ci: Load KVM kernel module for LAVA runners If 'HWCI_KVM' enviroment variable is set, load the KVM kernel module specific to the detected CPU virtualisation extensions: vmx for Intel VT and svm for AMD-V. As an additional optimization, handle HWCI_KERNEL_MODULES probing in the main shell process instead of creating an unnecessary subshell. Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15208>
2022-02-22 14:49:28 +00:00
HWCI_KVM \
ci: Unify {BM,LAVA}_START_XORG environment Why were they ever different ... ? Signed-off-by: Daniel Stone <daniels@collabora.com> Acked-by: Martin Peres <martin.peres@mupuf.org> Acked-by: Emma Anholt <emma@anholt.net> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>
2021-06-11 16:46:57 +01:00
HWCI_START_XORG \
ci: Unify {BARE_METAL,LAVA}_TEST_SCRIPT environment Should also probably never have been different. Signed-off-by: Daniel Stone <daniels@collabora.com> Acked-by: Martin Peres <martin.peres@mupuf.org> Acked-by: Emma Anholt <emma@anholt.net> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>
2021-06-11 16:53:54 +01:00
HWCI_TEST_SCRIPT \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
IR3_SHADER_DEBUG \
ci: Add JOB_ARTIFACTS_BASE variable Used for both LAVA (uploading results to MinIO because we don't yet have non-ephemeral NFS storage) and Piglit (for the Tracie dashboard). Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 16:29:39 +01:00
JOB_ARTIFACTS_BASE \
ci/lava: Use per-job rootfs overlay for environment Trying to get arbitrary strings suitably quoted for shell, embedded in a YAML file, processed by Python templating, is like seven bad ideas all embedded into one big can of bees. Reuse the same script we use for bare-metal to generate the environment, tar that up into a per-job overlay which is added to the inter-pipeline-reusable rootfs built by the container jobs and the intra-pipeline-reusable overlay built by the build jobs. @anholt wrote a chunk of this - replacing the $ENV_VARS GitLab CI variable with a Python loop across the POSIX job environment - in !11192, but this still had YAML quoting nightmares, and was more needless duplication between LAVA and bare-metal. The diff is large and annoying, but is mostly a sed job to get ENV_VARS="FOO=bar BAZ=quux" into FOO: bar\nBAZ: quux. Signed-off-by: Daniel Stone <daniels@collabora.com> Co-authored-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 17:00:32 +01:00
JOB_RESULTS_PATH \
JOB_ROOTFS_OVERLAY_PATH \
ci: Make kernel image available in LAVA for KVM use cases In order to run a VM (e.g. crosvm) through HWCI_TEST_SCRIPT on a LAVA target, it's necessary to download a kernel image on the target device. When HWCI_KVM is set to 'true', we can safely assume HWCI_TEST_SCRIPT contains a command or the path to a script which expects the kernel image to be available under /lava-files/${KERNEL_IMAGE_NAME}. Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15208>
2022-02-22 14:58:47 +00:00
KERNEL_IMAGE_BASE_URL \
KERNEL_IMAGE_NAME \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
LD_LIBRARY_PATH \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
LP_NUM_THREADS \
MESA_BASE_TAG \
ci/lava: Use per-job rootfs overlay for environment Trying to get arbitrary strings suitably quoted for shell, embedded in a YAML file, processed by Python templating, is like seven bad ideas all embedded into one big can of bees. Reuse the same script we use for bare-metal to generate the environment, tar that up into a per-job overlay which is added to the inter-pipeline-reusable rootfs built by the container jobs and the intra-pipeline-reusable overlay built by the build jobs. @anholt wrote a chunk of this - replacing the $ENV_VARS GitLab CI variable with a Python loop across the POSIX job environment - in !11192, but this still had YAML quoting nightmares, and was more needless duplication between LAVA and bare-metal. The diff is large and annoying, but is mostly a sed job to get ENV_VARS="FOO=bar BAZ=quux" into FOO: bar\nBAZ: quux. Signed-off-by: Daniel Stone <daniels@collabora.com> Co-authored-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 17:00:32 +01:00
MESA_BUILD_PATH \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
MESA_DEBUG \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
MESA_GLES_VERSION_OVERRIDE \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
MESA_GLSL_VERSION_OVERRIDE \
MESA_GL_VERSION_OVERRIDE \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
MESA_IMAGE \
MESA_IMAGE_PATH \
MESA_IMAGE_TAG \
ci: fix passing down MESA_LOADER_DRIVER_OVERRIDE for zink-anv-tgl zink-anv-tgl wasn't using Zink at all because this variable was missing and then not passed to the runners... This introduces a list of failures for Zink/Anv and also few tests are skipped because they take too long (> 60s). Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Martin Roukala <martin.roukala@mupuf.org> Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17031>
2022-06-10 16:22:10 +01:00
MESA_LOADER_DRIVER_OVERRIDE \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
MESA_TEMPLATES_COMMIT \
ci: Unify on MESA_VK_IGNORE_CONFORMANCE_WARNING Move and rename warn_non_conformant_implementation() to common location of src/vulkan/util/vk_util.c as vk_warn_non_conformant_implementation(). In freedreno/ci, move MESA_VK_IGNORE_CONFORMANCE_WARNING to common location of .baremetal-deqp-test-freedreno-vk. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11563>
2021-06-23 21:22:03 +01:00
MESA_VK_IGNORE_CONFORMANCE_WARNING \
amd, ci: Drop log level in SPIRV -> NIR code generator. See 786fa3435c for the rationale of this variable, but the point is to avoid many error reports for conformance conformance issues within the VK-CTS shaders. Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Martin Roukala <martin.roukala@mupuf.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14660>
2022-01-10 18:45:40 +00:00
MESA_SPIRV_LOG_LEVEL \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
MINIO_HOST \
ci/lava: Simplify passthrough of the request to upload results/ to minio. We already have a way to pass env vars around, just use that instead of packing/unpacking it on the kernel command line. Cleans up HW runner job log output some more. Acked-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15332>
2022-03-10 22:21:53 +00:00
MINIO_RESULTS_UPLOAD \
nir: add NIR_DEBUG envvar Move all the NIR related debug environmental variables in a single NIR_DEBUG one. Use NIR_DEBUG=help to print all the available options. v2: - Use a macro to simplify (Marcin, Jason) - Remove wrong changes (Marcin) v3 (Marcin): - Remove rendundant NIR mentioning in option descriptions. - Unwrap option descriptions. - Ensure the constant is unsigned. - Use extern array to remove switch. v4: - Add missing kernel shader (Jason). - Add unlikely() (Marcin). Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13840>
2021-11-17 15:10:52 +00:00
NIR_DEBUG \
panfrost/ci: Test panvk on Mali G52 Just run some selected tests for now because we miss a lot of functionality, which would cause so many crashes that the runs aren't practical. Once the core functionality is implemented, we can switch to the master case list with skips. Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Acked-by: Alyssa Rosenzweig <alyssa@collabora.com> Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13016>
2021-07-05 08:39:01 +01:00
PAN_I_WANT_A_BROKEN_VULKAN_DRIVER \
ci/lava: Use per-job rootfs overlay for environment Trying to get arbitrary strings suitably quoted for shell, embedded in a YAML file, processed by Python templating, is like seven bad ideas all embedded into one big can of bees. Reuse the same script we use for bare-metal to generate the environment, tar that up into a per-job overlay which is added to the inter-pipeline-reusable rootfs built by the container jobs and the intra-pipeline-reusable overlay built by the build jobs. @anholt wrote a chunk of this - replacing the $ENV_VARS GitLab CI variable with a Python loop across the POSIX job environment - in !11192, but this still had YAML quoting nightmares, and was more needless duplication between LAVA and bare-metal. The diff is large and annoying, but is mostly a sed job to get ENV_VARS="FOO=bar BAZ=quux" into FOO: bar\nBAZ: quux. Signed-off-by: Daniel Stone <daniels@collabora.com> Co-authored-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 17:00:32 +01:00
PAN_MESA_DEBUG \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
PIGLIT_FRACTION \
ci/lava: Set PIGLIT_NO_WINDOW This got lost in the move away from hardcoded environment variables, and fixes the Iris EGL tests. Signed-off-by: Daniel Stone <daniels@collabora.com> Acked-by: Martin Peres <martin.peres@mupuf.org> Acked-by: Emma Anholt <emma@anholt.net> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11337>
2021-06-12 13:55:26 +01:00
PIGLIT_NO_WINDOW \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
PIGLIT_OPTIONS \
PIGLIT_PLATFORM \
PIGLIT_PROFILES \
PIGLIT_REPLAY_ARTIFACTS_BASE_URL \
PIGLIT_REPLAY_DESCRIPTION_FILE \
PIGLIT_REPLAY_DEVICE_NAME \
PIGLIT_REPLAY_EXTRA_ARGS \
ci: Add PIGLIT_REPLAY_LOOP_TIMES to generate-env.sh PIGLIT_REPLAY_LOOP_TIMES is currently used to override the default configuration when running virgl trace performance tests in LAVA. Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15208>
2022-02-24 20:41:53 +00:00
PIGLIT_REPLAY_LOOP_TIMES \
ci: Use JOB_ARTIFACTS_BASE for Piglit fails It's not Piglit-specific per se, it's just another per-job artifact upload which needs to be made visible through MinIO. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 16:35:03 +01:00
PIGLIT_REPLAY_REFERENCE_IMAGES_BASE \
ci: Use ci-fairy minio login via token file For every CI job, put JWT content into a file and unset CI_JOB_JWT environment var ======= * virgl jobs: - Share JWT token file to crosvm instance - Keep using `export -p` due to high complexity in the scripts of these jobs. At least, the CI_JOB_JWT will not be leaked, since it is being unset at the `before_script` phase of each Mesa CI job. * iris jobs: Update lava_job_submitter to take token file as argument - generate-env with CI_JOB_JWT_TOKEN_FILE - create token file during baremetal init stage * baremetal jobs: Copy token file to bare-metal NFS Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14004>
2021-12-02 13:13:10 +00:00
PIGLIT_REPLAY_SUBCOMMAND \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
PIGLIT_RESULTS \
PIGLIT_TESTS \
ci: Make PIPELINE_ARTIFACTS_BASE a common variable $minio/artifacts/$project/$pipeline/ is common between all our CI. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 16:24:48 +01:00
PIPELINE_ARTIFACTS_BASE \
radv/ci: test the llvm backend on navi21 The LLVM backend is not officially supported by the RADV developers, but it has been useful early during bring-up, or later when users are experiencing what looks like a compiler bug. It is thus beneficial to keep it working. However, maintaining the vkcts expectations for every platform requires more work and machine time than what we would like to commit to. This is why we agreed that we would only keep LLVM tested on the latest family of Radeon GPUs. Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Acked-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16841>
2022-06-02 17:10:15 +01:00
RADV_DEBUG \
radv/ci: enable the experimental support for mesh shaders The experimental support has been merged for months already, but has been left unexercised in CI even though dEQP has a mesh_shader.nv category. This commit enables the experimental support, which should bring additional testing \o/. Suggested-by: Timur Kristóf <timur.kristof@gmail.com> Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org> Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Acked-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16550>
2022-05-17 07:22:09 +01:00
RADV_PERFTEST \
ci: Build skqp on ARM64 images This commit makes `kernel+rootfs_arm64` job build and install skqp on ARM64 devices rootfs. Skia repository has a tool to prepare skqp models located at `tools/skqp/cut-release`, which get files from [Skia Gold](https://skia.org/docs/dev/testing/skiagold/), generate files.checksum, rendertests.txt and unittests.txt. One gives a range of commits to let `cut-release` find the right resources to prepare skqp for the user. However, it is failing, since it fails when trying to get image packages from a range of commits via HTTPS from the host https://public-gold.skia.org but it responds with error 404 every time. I tried a range a thousand of commits, yet it still does not give results. The workaround employed was to recover the most recent `files.checksum` and `rendertests.txt` files from the git history and generate `unittests.txt` from `list_gpu_unit_tests` binary. `skqp` runs two lists of tests, `rendertests.txt` and `unittests.txt`. Both must be located inside the `skqp` assets folder. The first list uses GL and GLES to test rendering scenarios. The second runs some unit tests that do not render an image per se. In order to make the first `a630_skqp` to be green, the crashing tests were removed from the test lists and the expectations of the failing ones were updated. It is worth noting that `rendertests.txt` can bring some detail about each test expectation, so each test can have a max pixel error count, to tell `skqp` that it is OK to have at most that number of errors for that test. See also: https://github.com/google/skia/blob/main/tools/skqp/README_ALGORITHM.md As each render backend has a different error count, two different `rendertests.txt` files were created, `src/freedreno/ci/freedreno-a630-skqp-gl_rendertests.txt`, `src/freedreno/ci/freedreno-a630-skqp-gles_rendertests.txt` and , which one refers to GL and GLES tests respectfully. The unit tests file for a630 is located at `src/freedreno/ci/freedreno-a630-skqp_unittests.txt` ``` aaclip domain formats highcontrastfilter rectangle_texture yuv_make_color_space ``` ``` ProcessorOptimizationValidationTest VkProtectedContext_CreateNonprotectedContext VkYCbcrSampler_DrawImageWithYcbcrSampler VkYCbcrSampler_NoYcbcrSurface ``` Each test was updated with the max_error count equal to the first run result. ``` analytic_antialias_inverse async_rescale_and_read_dog_down async_rescale_and_read_dog_up async_rescale_and_read_rose async_rescale_and_read_text_down async_rescale_and_read_text_up async_rescale_and_read_text_up_large async_rescale_and_read_yuv420_rose complexclip2_path_bw encode-platform imageblur_large lcdtextsize onebadarc onefailarc scale-pixels surfaceprops textfilter_color textfilter_image ``` Considering all the following tests results as wrong. ``` async_rescale_and_read_no_bleed backdrop_imagefilter_croprect_persp complexclip2 imageblurrepeatmode mixerCF overdrawcolorfilter patch_alpha patch_primitive rrect_clip_bw scaledemoji_rendering yuv_splitter ``` v2: a) add link to HTML report on job log b) remove extraneous spaces diff c) remove unnecessary conditions from build-skqp.sh d) use fixed skqp source commit SHA v3: a) Use only main skia repository to fetch models and build skqp b) Use list_gpu_unit_tests binary to create a base unittests.txt file c) Remove crashing tests d) Set failing tests expectations for the first skqp run v4: a) Remove clang dependency b) Separate each skqp backend result into its folder c) Regroup a630_skqp in one job v5: a) Separate tests files per driver Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5580 Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14146>
2021-12-07 14:13:00 +00:00
SKQP_ASSETS_DIR \
SKQP_BACKENDS \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
TU_DEBUG \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
VIRGL_HOST_API \
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
VK_CPU \
VK_DRIVER \
virgl/ci: make crosvm-runner pass variables in a secure way crosvm-runner.sh was using `export -p` to create an environment script for the virtualized system, but this command will dump every declared environment variable in the system, which includes Gitlab's CI variables with sensitive data, such as passwords and auth tokens. Replacing `export -p` to `generate-env.sh`, which only exports the necessary variables for Mesa CI jobs. Extra changes: * Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a variable used by most prolific coreutils and bash commands, such as `cd` and `pwd`, besides it is set by subshells [1]; changing this variable may lead to complex situations. As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that there is a special folder where dEQP should be run. * Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086 * Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155 [1] ``` $ cd /tmp $ export PWD=test; bash -c 'echo $PWD' /tmp ``` v2: - Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh - Log all the passed variables to stdout, to help with debugging when new variable are needed to be put in `generate-env.sh` v3: - Revert $DEQP_BIN_DIR quoting leftovers Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
2022-01-14 21:43:08 +00:00
VK_ICD_FILENAMES \
ci: add VKD3D_PROTON_RESULTS to the list of variables to pass down for testing Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Martin Roukala <martin.roukala@mupuf.org> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17050>
2022-06-09 11:05:07 +01:00
VKD3D_PROTON_RESULTS \
Revert "ci: Convert generate-env.sh to a POSIX compliant script" This reverts commit 9904ea2c76e246629fab8bda455eec07aa84f3f9 since it is not able to properly escape all special characters (i.e. [']). The POSIX conversion is not needed anymore, as we have made 'bash' available in LAVA rootfs. Reported-by: Daniel Stone <daniels@collabora.com> Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com> Reviewed-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15524>
2022-03-23 14:29:34 +00:00
; do
if [ -n "${!var+x}" ]; then
echo "export $var=${!var@Q}"
fi
ci/bare-metal: Factor out environment to a separate script This will let us reuse the same environment generation for both bare-metal and LAVA. Signed-off-by: Daniel Stone <daniels@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11309>
2021-06-10 10:50:44 +01:00
done