Commit Graph

136 Commits

Author SHA1 Message Date
Tomeu Vizoso d4ca45eca2 ci: Upload traces' reference and actual images to MinIO
Now that the devices have sane dates, we can upload to MinIO.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6136>
2020-08-04 08:18:51 +02:00
Eric Anholt cb82274538 ci/bare-metal: Capture the first devcoredump a job produces.
Connor recently ran into an issue where the chezas were hanging where his
GPUs weren't, and was blocked on getting some feedback on what was
happening.  A devcoredump will help non-cheza-having devs debug (or
hopefully with other intermittent fails).

Closes: #3187
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6036>
2020-07-30 11:41:57 +00:00
Tomeu Vizoso 292882f6bc ci: Fix the overwriting of traces.yml for baremetal
When the lava files were moved out of the container, this stopped
working which caused the traces job for Freedreno to not run any traces
at all.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Fixes: dcd171f5e9 ("gitlab-ci: More stable URL for kernel and ramdisk artifacts, for LAVA")
Acked-by: Andres Gomez <agomez@igalia.com>
Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6021>
2020-07-22 18:07:31 +00:00
Eric Anholt 8b3452a556 ci: Set XDG_CACHE_HOME to tmpfs for bare-metal runners to avoid NFS.
We don't want these files shared between builds (it'll get blown away by
the next rsync), and NFS will just increase our latency for hitting the
cache.

Drops a630 gles31 run from 11-17 minutes to 5.5.  Maximum cache size on a
run I've seen is 153M, which it seems we can easily spare.

Fixes: f97acb4bb4 ("freedreno/ir3: disk-cache support")
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5998>
2020-07-21 11:04:14 +00:00
Christian Gmeiner 096adbe369 ci: bare-metal: use nginx to get results from DUT
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2655
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5661>
2020-07-20 20:21:12 +00:00
Eric Anholt 27c9272c3d docs: Move the gitlab-ci docs to RST.
I tried not to edit too much meaning in the process, but I did shuffle
some stuff around to work as structured documentation.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5510>
2020-07-08 20:13:11 +00:00
Eric Anholt 63805ccd3f ci/bare-metal: Fail early when we get stuck powering on a cheza.
I think I've seen about 3 of this error total so far, but waiting 60
minutes for the scripts to give up wastes marge time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5651>
2020-06-26 16:54:07 +00:00
Eric Anholt b88c46fa11 ci: Add a freedreno a630 tracie run.
This job runs in about one minute on the current set of traces, and has
successfully revealed some bugs in our current rendering.  Takes about 7
minutes currently.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
2020-06-25 17:33:28 +00:00
Rohan Garg 7406d627c8 ci: Include trace replay support in ARM rootfses.
Builds the renderdoc and apitrace programs so we can replay GL traces on
DUTs.

[Separated out from 5472's commit that also enabled the jobs in LAVA,
dropped unnecessary python packages from arm_build, fixed up arm64_test
build, traces-db in baremetal, new commit message by anholt]

Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
2020-06-25 17:33:28 +00:00
Eric Anholt 9079b53987 ci/bare-metal: Skip setting of unset variables at startup.
It's silly to be setting (and logging the setting of!) all the env vars we
*didn't* set in a job.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5433>
2020-06-25 17:33:28 +00:00
Eric Anholt 32143cba4d ci/bare-metal: Terminate the job with an error on kernel panic.
Otherwise, we'll time out after 60 minutes of waiting for the run to
complete.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2651
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt 72fe7b98ea ci/bare-metal: Stop fetching the git tree.
Like for LAVA, make the tradeoff of moving the test scripts and data (55k)
into the artifacts in order to make the per-build jobs not have to pull
down the git tree (hundreds of MB when you don't hit a cached container
for your specific user, which I see happen multiple times a day in my CI
runs).

To do this, we have to be a bit more careful in some places about our
working directory potentially being dirty.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt 109816b518 ci/bare-metal: Use the deqp-runner bits straight out of the artifacts.
We've already uploaded and downloaded them from fd.o and put them in the
rootfs, so we can clean up the extra prep work.

Our test job now extends from .test so that the artifacts' install dir
with all the scripts is extracted.  This required moving the dependency on
meson-testing to the x86 test-gl/test-vk job blocks.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt 445f3eb0ea ci/bare-metal: Make which test to run configurable.
I'll use this to run tracie in a new job I'm working on.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt a13209bdec ci/bare-metal: Reword the final output of the init script on the board.
I'm going to be adding tracie, which isn't deqp.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5393>
2020-06-12 23:34:44 +00:00
Eric Anholt 3d5429d646 ci: Use rsync for initial nfsroot population on cheza.
rm -rf and then copying over all the contents again is a waste of time
when we'll almost always be using the same rootfs.  Saves about 30s of job
time.

Closes: #3065
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
2020-06-09 18:28:18 +00:00
Eric Anholt 9e11cce517 ci: Enable pre-merge fractional vulkan CTS runs on the turnip driver.
Test 1/50th of the CTS on a630 pre-merge, since we've got hardware that
can do it and infrastructure that should handle instability with a
less-mature driver.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
2020-06-09 18:28:18 +00:00
Eric Anholt dd167788ec ci: Build the full VK CTS for baremetal testing.
I'm going to enable the VK CTS on cheza, so swap the deqp we have in the
container.  build-deqp-vk already included GLES deqp binaries and data,
and is a newer branch than the last opengl-es-cts tag.

This brings a few things back over from build-deqp-gl for testlog
extraction, and copyes out the GLES mustpass lists.

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5266>
2020-06-09 18:28:18 +00:00
Christian Gmeiner 4b7de75b4b ci: add U-Boot specific fetch strings
U-Boot's fastboot over udp generates the following output:
  Listening for fastboot command on x.y.z.w

Also add a general 'data abort' error string seen with an
too old U-Boot version:
  https://github.com/u-boot/u-boot/commit/95712af

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5258>
2020-06-05 11:02:35 +00:00
Christian Gmeiner 06d8171994 ci: extend expect-output.sh
We need to support different fastboot fetch strings for different
bootloader solutions. Lets extend expect-output.sh to support
multiple fetch strings (-f) and add support for error catch
strings (-e) to stop the CI run early.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5258>
2020-06-05 11:02:35 +00:00
Christian Gmeiner 2dfc241e36 ci: bare-metal: make it possible to use a script for serial
Makes it possible to use e.g. a ser2net script to talk to the devices.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5268>
2020-06-02 22:20:46 +00:00
Eric Anholt c89a749f66 ci: Add scripts for controlling bare-metal chezas.
This will let us:

- deploy kernels for testing code depending on new kernel featuers
- Ensure a pristine state in the HW before starting our tests
- Avoid disk rot on the chezas taking them out (we'd lost 3/9 in a few
  months).

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5247>
2020-05-29 16:46:44 +00:00
Eric Anholt 46d9b500f4 ci: Move baremetal DEQP_NO_SAVE_RESULTS setup to the yml.
I'm going to want it unset (artifacts enabled) for the cheza jobs.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5225>
2020-05-28 16:23:59 +00:00
Eric Anholt 38f32372aa ci: Improve baremetal's logging of the job env var passthrough.
Trying to read the sh -x script output was rough, just cat the file once
we're done setting it up.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5089>
2020-05-22 16:44:46 +00:00
Eric Anholt ae442c3598 ci: Enable a fractional run with UBO-to-constbuf disabled on a3xx.
This gets us coverage of an important case in the HW that the CTS
otherwise basically doesn't hit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5089>
2020-05-22 16:44:46 +00:00
Eric Anholt b4bccbde36 ci: Don't forget to set NIR_VALIDATE in baremetal runs.
Given that a530 doesn't have cpufreq, we really don't have the time to be
running the validator on all of deqp.  This also helps explain why I had
to go to such a small fraction on the a3xx gles3 run (which we can now
increase).  However, a3xx gles2 seems to be fast enough that we can leave
it enabled and get coverage for older chips.

Because we run more tests now, clear out some stale xfails from the a3xx
list.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5089>
2020-05-22 16:44:46 +00:00
Eric Anholt 2637961d29 ci: Fix the nick used in IRC reporting.
robclark found that we needed unique IDs when multiple runners were trying
to report flakes at the same time, but it turns out due to nick limits (16
chars on freenode) we were just getting all the runners appended with
"-142" (or whatever the prefix of the pipelines are these days).  And, for
the new flake reporting from baremetal, all the runners ended up being
just "google-freedreno".

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4896>
2020-05-06 18:34:38 +00:00
Eric Anholt 2c50176dfe ci: Improve the flakes reports on IRC.
We were incorrectly taking the merge-request on non-MR pipelines (the
master build after merge) due to a missing '$'.  And, for those pipelines,
it would be nice to note whether they're for master or a stable branch.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4896>
2020-05-06 18:34:38 +00:00
Eric Anholt 3b5e71cb18 ci: Enable IRC flake reporting on freedreno baremetal boards.
The IRC channel is useful for me to track and ban flaky tests before they
irritate people too much.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2654
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4896>
2020-05-06 18:34:38 +00:00
Eric Anholt c7bbc211d6 ci: Clean up setup of the job-specific env vars in baremetal testing.
Avoids copy and paste errors when adding more vars.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4896>
2020-05-06 18:34:38 +00:00
Christian Gmeiner 7d476a1360 ci: bare-metal: power down device after tests
Helps to save electricity.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4754>
2020-04-28 07:17:24 +00:00
Eric Anholt 8b221e0914 ci: Add sanity checking that dEQP gets the expected GL_RENDERER.
It's easy to get something wrong in the driver build or container or
something that results in falling back to swrast, and then your only clue
was runtime and how your failure cases suspiciously match a swrast
driver's.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4621>
2020-04-27 22:10:10 +00:00
Eric Anholt a9e6a3ecc7 ci: Enable --compact-display false on all dEQP runs.
We always want to see status updates happening in the logs, otherwise it
can like maybe your machine hung until the run actually completes.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4621>
2020-04-27 22:10:10 +00:00
Tomeu Vizoso 92f3c51560 gitlab-ci: Place files from the Mesa repo into the build tarball
There's some files from the .gitlab-ci directory that are needed in the
test stage and that, because the Mesa repository isn't checked out in
that stage, need to be made available through other means.

Because those files are going to be needed in LAVA devices, place them
ino the tarball containing the built files so it's available to both
gitlab-ci runners and LAVA devices.

Before those files were passed in the artifacts of the Gitlab CI job,
but this commit places them into the built tarball so scripts later in
the pipeline don't need to account for this discrepancy.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4295>
2020-03-26 09:30:48 +00:00
Eric Anholt 4bc15e78fa ci: Enable testing GLES2-3 on a530 (Dragonboard 820c).
Following on from the db410c conversion to baremetal testing, reuse the
same scripts in the same rack to run 7 db820c boards (#4/8 is failing in
the bootloader for unknown reasons).

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4177>
2020-03-17 11:11:51 -07:00
Eric Anholt cf5ba9d409 ci: Make a simple little bare-metal fastboot mode for db410c.
This supports powering up the device (using an external tool you
provide based on your particular lab), talking over serial to wait for
the fastboot prompt, and then booting a fastboot image on a target
device.

I was previously relying on LAVA for this, but that ran afoul of
corporate policies related to the AGPL.  However, LAVA wasn't doing
too much for us, given that gitlab already has a job scheduler and
tagging and runners.  We were spending a lot of engineering on making
the two systems match up, when we can just have gitlab do it directly.

Lightly-reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4076>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4076>
2020-03-11 21:36:47 +00:00