Commit Graph

225 Commits

Author SHA1 Message Date
Samuel Pitoiset 520a77d486 gitlab-ci: add a gl suffix to the x86 test image and all test jobs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:57:46 +01:00
Michel Dänzer 41797a1fed gitlab-ci: Overhaul job run policy
Use new rules: instead of only:

For container stage jobs:

* In the main Mesa project, run them by default.

* In merge requests, run them by default if any files affecting pipeline
  results are changed.

* In all other cases (in particular branches in personal projects),
  don't run them by default but allow triggering them manually.

build & test stage jobs are left at the default (when: on_success), so
they will run automatically once all their dependencies are satisified.
(Using the same rules as above would require these jobs to be manually
triggered as well, which is only possible once all dependency jobs have
passed) Please be considerate of CI runner resources and cancel unneeded
jobs on personal branches with no corresponding merge requests (this can
be done before the jobs start running).

In summary: No more special branch names. Unnecessary job runs are
avoided by default, but jobs which don't run by default can be triggered
manually.

v2:
* Split out LAVA changes to separate commit
* Clarify commit log a little, in particular WRT build/test stage jobs

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> # v1
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> # v1
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> # v1
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-12-06 10:02:01 +01:00
Eric Anholt 3097efe5f0 ci: Disable egl_ext_device_drm tests in piglit.
If the runner has a HW device that would be supported, even without
/dev/dri forwarded into the container, it will be enumerated and the tests
on llvmpipe fail with (for example):

libEGL warning: Not allowed to force software rendering when API explicitly selects a hardware device.
libEGL warning: MESA-LOADER: failed to open i965 (search paths /builds/anholt/mesa/install/lib/dri)

Given that we can't necessarily control the DRI devices present on the
runners (particularly for developers bringing their own runners to reduce
the demands on fd.o's shared resources), just skip these tests in CI.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-05 18:06:10 +00:00
Tapani Pälli 2d26cc077d gitlab-ci: bump piglit checkout commit
Commit also updates the Piglit quick_gl.txt, list modifications happened
due to following Piglit commits: c248bf201,c acff58ca, 5603e2e60.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2019-12-04 15:27:41 +00:00
Michel Dänzer 5585b8eadd gitlab-ci: Run piglit glslparser & quick_shader tests separately
And only use --process-isolation false for the quick_gl tests.

This will hopefully avoid variance in the test results that we've been
seeing lately. But even if it doesn't, it should at least help narrow
down the cause of the variance.

Tested-by: Vasily Khoruzhick <anarsoul@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
2019-12-04 10:36:33 +01:00
Rob Clark b31637c453 gitlab-ci: disable junit results for deqp
They don't seem to be hugely useful, and seem to be bogging down gitlab.

Signed-off-by: Rob Clark <robdclark@chromium.org>
2019-12-03 08:46:39 -08:00
Michel Dänzer 0c88d5952a gitlab-ci: Update to current ci-templates master
Fixes skopeo copy failures.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
2019-12-03 16:03:31 +01:00
Michel Dänzer a3b3d3bfcc gitlab-ci: Put HTML summary in artifacts for failed piglit jobs
This will make it easier to look at details of failed / skipped tests.

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-27 10:20:31 +01:00
Michel Dänzer c9cdb7cef0 gitlab-ci: Stop reporting piglit test results via JUnit
It was basically useless in this form, and processing the JUnit data in
the GitLab backend was pretty expensive.

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-27 10:18:33 +01:00
Rob Clark 9f422cbe1c gitlab-ci/deqp: generate junit results
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-22 13:48:29 -08:00
Rob Clark 8af7551a9e gitlab-ci: bump arm test container
To pick up updated cts_runner and netcat for the flake reporting.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-22 13:48:29 -08:00
Rob Clark fdaf777076 gitlab-ci/deqp: detect and report flakes
If there are a small number of fails, re-run to determine if they are
flakes, and optionally (if `$FLAKES_CHANNEL` configured) report the
flakes.

This way flakes don't interfere with developers working on other
drivers, but get logged so that the developers working on the flaking
driver can monitor the situation.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-22 13:48:29 -08:00
Samuel Pitoiset 6f7ec6ee39 gitlab-ci: reduce the number of scons build
It seems overkill to me to build scons 7x for every pipeline.
Scons is now build with the oldest llvm version in scons-old-llvm
and with the newest llvm version in scons.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-22 10:39:21 +01:00
Eric Anholt dd76a6f198 ci: Move freedreno's parallelism to the runner instead of gitlab-ci jobs.
I set the runners to concurrency=1, so they serve only one gitlab-ci job
at at time.  Swap over to using the parallel runner now to keep the
runners busy, more efficiently than spawning many docker containers and
downloading artifacts multiple times, and producing easier-to-understand
results for browsing on the web.

This bumps the a306 runners to 4x parallel instead of 2x like before, but
cheza gles3 drops from 6 to 4.  Current rough timings of the jobs (if no
container download):

db410c-gles2: 5:00
a630-gles2: 1:30
a630-gles3: 6:00
a630-gles31: 5:30

a630-gles3 is a bit longer than I like, but it should come back down once
I can sort out the NIR algebraic rewinding.
2019-11-21 05:48:17 -08:00
Michel Dänzer 32618ee719 gitlab-ci: Directly use host-mapped directory for ccache
Use hardcoded /cache/mesa/ccache for the cache, so it will be shared by
all jobs of all Mesa projects running on the same runner host. This
should increase the hit rate and decrease the worst case storage used.

Further benefits of directly using a host-mapped directory:

* Saves up to ~1 minute per job for restoring and saving the cache
  contents via the GitLab CI cache mechanism
* Cache contents generated by failed jobs are no longer lost
* Jobs running in parallel on the same runner host can get hits from
  each other

Also enable compression, so the default maximum cache size of 5G might
be sufficient.

v2:
* Move CCACHE_DIR variable to the .build-linux template

Suggested-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Anholt <eric@anholt.net> # v1
2019-11-21 10:13:43 +01:00
Samuel Pitoiset 0d1085ac4a gitlab-ci: remove now useless meson-swr-glvnd build job
All things are already part of meson-main.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset 7362176cfe gitlab-ci: build GLVND in meson-clang
Building GLVND in meson-main doesn't work because this disables
libEGL and it's needed for running shader-db.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset e6d26d77a3 gitlab-ci: build swr in meson-main
Now that debugoptimized isn't set and that all test jobs depend on
meson-testing, enabling swr shouldn't slowdown the CI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset 6cf9b53fa2 gitlab-ci: do not build with debugoptimized for meson-main
This should reduce compile time because optimizations are costly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset 66b5627074 gitlab-ci: add a job that only build things needed for testing
For turnip and RADV testing, we will need a debugoptimized build
without UBSAN. This introduces meson-testing which builds only the
things that are needed by the test stage.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:04 +01:00
Samuel Pitoiset eab328fbe9 gitlab-ci: fix ldd check for Vulkan drivers
The 'dri' directory isn't created when building Vulkan drivers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:34:08 +01:00
Timothy Arceri 530d3b2900 gitlab-ci: bump piglit checkout commit 2019-11-20 05:05:56 +00:00
Michel Dänzer c6c7652753 gitlab-ci: Organize images using new REPO_SUFFIX templates feature
Two benefits:

Most docker image related environment variables can now be defined in
the jobs where they're used instead of globally. The DEBIAN_TAG values
are propagated to other jobs via YAML anchors.

Images on https://gitlab.freedesktop.org/mesa/mesa/container_registry
are now organized in separate repositories with a suffix matching the
name of the job which makes sure the image is there.

Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:23:22 +01:00
Michel Dänzer 506e9d5fc7 gitlab-ci: Rename container install scripts to match job names (better)
Cleans up .gitlab-ci/ a little, and allows using a single DEBIAN_EXEC
line for all container jobs.

v2:
* Use lava_arm.sh instead of arm_lava.sh for consistency with v2 of the
  previous change

Reviewed-by: Eric Anholt <eric@anholt.net> # v1
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:21:10 +01:00
Michel Dänzer 3a48f4565e gitlab-ci: Use functional container job names
This makes it easier to tell which job is which in a pipeline.

v2:
* Use lava_arm{64,hf} instead of arm{64,hf}_lava to keep these jobs
  together in pipeline overviews

Reviewed-by: Eric Anholt <eric@anholt.net> # v1
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:20:16 +01:00
Michel Dänzer 670277846d gitlab-ci: Document that ci-templates refs must be in sync
Otherwise there can be weird breakage.

(Removing the include from .gitlab-ci/lava-gitlab-ci.yml doesn't seem
possible unfortunately:
https://gitlab.freedesktop.org/daenzer/mesa/pipelines/79458)

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:06:54 +01:00
Tapani Pälli 7a893a0d57 gitlab-ci: update Piglit commit, update skips
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-15 12:06:15 +02:00
Eric Engestrom ac78ca4b39 gitlab-ci: auto-cancel CI runs when a newer commit is pushed to the same branch
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-14 17:32:31 +00:00
Samuel Pitoiset 47ba227448 gitlab-ci: build RADV on ARM64
The ARMHF LLVM package is LLVM 7 but RADV requires LLVM 8.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-13 10:52:10 +01:00
Samuel Pitoiset cb19f69ff0 gitlab-ci: build a specific libdrm version for ARM64
RADV requires libdrm-2.4.100 but the distrib package is too old.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-13 10:52:08 +01:00
Eric Anholt 52843ec5d3 ci: Enable all of GLES3/3.1 testing for softpipe.
Now that we're not using so many job slots, it's easy to get these
jobs run in a reasonable amount of time (gles3 took 10 minutes for 4
cores, and gles31 was 15 minutes for 4 cores).

Acked-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-12 12:54:04 -08:00
Eric Anholt f08c810028 ci: Use cts_runner for our dEQP runs.
This runner is a little project by Bas, written in C++, that spawns
threads that then loop grabbing chunks of the (randomly shuffled but
consistently so) test list and hand it to a dEQP instance.  As the
remaining list gets shorter, so do the chunks, so hopefully the
threads all complete effectively at once.  It also handles restarting
after crashes automatically.  I've extended the runner a bit to do
what I was doing in the bash scripts before, like the skip list and
expected failures handling.  This project should also be a good
baseline for extending to handle retesting of intermittent failures.

By switching to it, we can have the swrast tests just take up one job
slot on the shared runners and keep their allotment of CPUs busy,
instead of taking up job slots with single-threaded dEQP jobs.  It
will also let us (eventually, once I reprovision) switch the freedreno
runners over to threading within the job instead of running concurrent
jobs, so that memory scribbles in one pipeline don't affect unrelated
pipelines, and I can experiment with their parallelism (particularly
on a306 where we are frequently backed up) without trashing other
people's jobs.

What we lose in this process is per-test output in the log (not a big
loss, I think, since we summarize fails at the end and reducing log
length keeps chrome from choking on our logs so badly).  We also drop
the renderer sanity checking, since it's not saving qpa files for us
to go poke through.  Given that all the drivers involved have fail
lists, if we got the wrong renderer somehow, we'd get a job failure
anyway.

v2: Rebase on droppong of the autoscale cluster and the arm64
    build/test split.  Use a script to deduplicate the cts-runner
    build.
v3: Rebase on the amd64 build/test container split.

Acked-by: Daniel Stone <daniels@collabora.com> (v1)
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> (v2)
2019-11-12 12:54:04 -08:00
Eric Anholt 66719e0242 ci: Use several debian buster packages instead of hand-building.
This helps cut down our container build time.  I've left a few that
we're likely to rev more frequently or I was less confident in
dropping.

v2: Rebase on the build/test container split, now bumps the build
    container tag in this commit.

Acked-by: Eric Engestrom <eric.engestrom@intel.com> (v1)
Acked-by: Daniel Stone <daniels@collabora.com> (v1)
2019-11-12 12:54:04 -08:00
Eric Engestrom 06347989a0 gitlab-ci: build libdrm using meson instead of autotools
Autotools was deprecated for a while and has now been removed, so let's
start using meson here so that we won't have any issues next time we
update libdrm.

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-12 17:08:02 +00:00
Michel Dänzer aebf43dcc1 gitlab-ci: Use separate docker images for x86 build/test jobs
Same as was done for the ARM images before.

This should make it less painful to update to newer dEQP / piglit as
well as to make changes to the build/test environment.

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:17:21 +01:00
Michel Dänzer 576f7b6ea5 gitlab-ci: Run piglit tests with llvmpipe
One job for the quick_gl profile, one for the glslparser & quick_shader
profiles (doing these together takes hardly any more time than
quick_shader alone).

v2:
* Don't break lava tests
v3:
* Remove piglit test artifacts paths:
* Exclude some quick_shader tests again:
  - Test whose result flips between pass/fail/skip
  - *@vs_in tests, as not the same one of these gets picked every time
v4:
* Do not list passing tests in .gitlab-ci/piglit/*.txt (Eric Anholt)
* Include the test number summary in .gitlab-ci/piglit/*.txt
* Completely disable generating any vs_in tests in the piglit build.
* Remove some more unneded files from the piglit build tree.
* Exclude quick_gl arb_gpu_shader5 tests; they were all skipped anyway,
  as llvmpipe doesn't support this extension yet, but occasionally they
  would spuriously fail instead.
v5:
* Set LD_LIBRARY_PATH, so we actually test the Mesa build from the
  pipeline...
* Verify that wflinfo reports the expected Mesa version
* Pass -noreset to Xvfb
v6:
* Don't use autoscale runners, run piglit with -j4 (Eric Anholt)

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:16:23 +01:00
Michel Dänzer df26e18b9f gitlab-ci: Share dEQP build process between x86 & ARM test image scripts
See https://gitlab.freedesktop.org/mesa/mesa/issues/2056

v2:
* Rename .gitlab-ci/deqp-build.sh => .gitlab-ci/build-deqp.sh
  (Eric Anholt)

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:14:49 +01:00
Michel Dänzer 59fcb019d0 gitlab-ci: Move artifact preparation to separate script
It's currently only needed for the meson-main and meson-arm64 jobs, not
the other meson build jobs.

Also remove MESON_SHADERDB, just run .gitlab-ci/run-shader-db.sh
directly from the meson-main job.

v2:
* Also run prepare-artifacts.sh in meson-arm64 script
v3:
* Move tarball creation into the new script as well, as it prevented
  ccache --show-stats from running in after_script

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:14:26 +01:00
Eric Anholt cb655d2554 Revert "ci: Switch over to an autoscaling GKE cluster for builds."
This reverts commit c9df92bf79.

It turns out that gitlab-runner uses kubernetes all wrong, spawning Pods
and sshing into them to run the script instead of Jobs containing the
script to run.  This means that when anything goes wrong with the pod
(autoscale, preemption, VM maintenance, cluster reconfiguration), the job
fails and only sometimes gets handled as a runner system failure.  Even
worse, due to bugs in either the runner or k8s itself, some classes of
timeout-related failure end up not being reported as failures, and the job
will incorrectly report success!

Disable using the "autoscale" cluster until we can do something else
(docker-machine instead of k8s, or the custom third-party k8s-native
runner).

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Acked-by: Daniel Stone <daniels@collabora.com>
2019-11-06 11:38:07 -08:00
Michel Dänzer b4d3ae2269 gitlab-ci: Use separate arm64 build/test docker images
The image used for test jobs is only about 1/6 as big as before, which
may help avoid some issues with some of the test boards.

Inspired by https://gitlab.freedesktop.org/mesa/mesa/issues/2046 .

v2:
* Leave LIBDRM_VERSION at 2.4.99 (Daniel Stone)
* Delete more build artifacts from dEQP tree (Daniel Stone)
v3:
* Set LD_LIBRARY_PATH for ldd

Acked-by: Daniel Stone <daniels@collabora.com> # v2
Reviewed-by: Eric Anholt <eric@anholt.net> # Except for the ldd line
2019-11-06 13:10:07 +01:00
Michel Dänzer d80dece065 gitlab-ci: Set arm job CCACHE_DIR properly
$PWD doesn't work for variables:, it ended up as "/ccache", always
starting with an empty cache.

v2:
* Use relative path and realpath
v3:
* Use $CI_PROJECT_DIR (Eric Anholt)
* Clear ccache stats in before_script if the cache is in $CI_PROJECT_DIR

Fixes: c9df92bf79 "ci: Switch over to an autoscaling GKE cluster for
                     builds."
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-05 09:27:32 +01:00
Eric Anholt c9df92bf79 ci: Switch over to an autoscaling GKE cluster for builds.
The GKE pool we're using is 1-3 32-core VMs, preemptible (to keep
costs down), with 8 jobs concurrent per system.  We have plenty of
memory (4G/core), so we run make -j8 to try to keep the cores busy even
when one job is in a single-threaded step (docker image download, git
clone, artifacts processing, etc.)  When all jobs are generating work
for all the cores, they'll be scheduled fairly.

The nodes in the pool have 300GB boot disks (over-provisioned in space
to provide enough iops and throughput) mounted to /ccache, and
CACHE_DIR set pointing to them.  This means that once a new
autoscaled-up node has run some jobs, it should have a hot ccache from
then on (instead of having to rely on the docker container cache
having our ccache laying around and not getting wiped out by some
other fd.o job).  Local SSDs would provide higher performance, but
unfortunately are not supported with the cluster autoscaler.

For now, the softpipe/llvmpipe test runs are still on the shared
runners, until I can get them ported onto Bas's runner so they can be
parallelized in a single job.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-10-31 11:19:43 -07:00
Eric Engestrom 9b58ab803d gitlab-ci: build a recent enough version of GLVND (ie. 1.2.0)
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2019-10-31 17:09:59 +00:00
Michel Dänzer aaf1b09270 gitlab-ci: Disable meson-windows job for the time being
It needs a CI runner carrying the mesa-windows tag, but there's none
available currently.
2019-10-30 09:38:20 +01:00
Michel Dänzer 2a38fc1027 gitlab-ci: Only run the pipeline if any files affecting it have changed
E.g. documentation-only changes cannot affect the outcome of the
pipeline, so don't waste resources on running it.

The thing we need to be careful about here is that the container stage
jobs must always run if any later stage jobs using the corresponding
docker images run. We're currently using the same .ci-run-policy
template for all jobs, so this is trivially true.

v2:
* Add bin/ and common.py (Eric Engestrom)

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> # v1
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-29 15:09:56 +00:00
Alexandros Frantzis 1257d06ba7 gitlab-ci: Update required libdrm version
Commit 9edcce2a32 bumped the required libdrm-amdgpu version to
2.4.100. Update the version we use in our CI scripts to avoid CI
build failures.

Also bump the debian image name for this change to take effect.
Note that amdgpu is only built with the debian-buster image,
so only this image requires an update.

Fixes: 9edcce2a ("ac: get tcc_harvested from the kernel")
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
2019-10-29 09:50:09 +00:00
Erik Faye-Lund 15e7f94278 gitlab-ci: fixup debian tags
When resolving a merge-conflict, I accidentally only updated the
ARM64-tag tag. Let's correct this.

Fixes: 3d529c1739 ("gitlab-ci: also build Zink on CI")
2019-10-28 12:07:30 +01:00
Erik Faye-Lund 3d529c1739 gitlab-ci: also build Zink on CI
This prevents accidentally breaking the driver-build while working on
other drivers.

Signed-off-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
2019-10-28 08:51:48 +00:00
Dylan Baker 19851c9ad6 gitlab-ci: Add a job for meson on windows
This adds a new CI job that runs on windows with MSVC. It currently
builds softpipe and osmesa, and runs the related unit tests. It does
rely on meson's wraps for zlib, but I've set up caching of the wrap
dependencies so hopefully that wont be a problem.

I really wanted to user powershell for this, but there just isn't an
easy way to do that, it's much easier to use batch scripts, so thats
what I used.

The leading `/` for .gitlab-ci/lava... must be removed because windows
doesn't understand it, and when it reads the file the job ends in error.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-10-25 22:47:32 +00:00
Dylan Baker 06e4647cb0 gitlab-ci: refactor out some common stuff for Windows and Linux
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-10-25 22:47:32 +00:00
Dylan Baker 54053bc8d0 scons: Print a deprecation warning about using scons on not windows
At this point meson should be able to handle all of the non-windows
platforms just fine; we'd like to be able to stop maintaining scons for
those platforms sooner than later.

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-24 18:33:50 +00:00
Michel Dänzer 75cc8c0b82 gitlab-ci: Enable UBSan for the meson-vulkan job
It doesn't report any errors now.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
2019-10-24 16:21:48 +02:00
Michel Dänzer 793f6b30d9 gitlab-ci: Enable llvmpipe in ARM build jobs
v2:
* Use LLVM 8 from buster-backports
v3:
* Use LLVM 7 again for armhf, llvmpipe is still broken there with LLVM 8

Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-22 10:26:29 +00:00
Michel Dänzer 163ec5d808 gitlab-ci: Use native aarch64 runner for ARM build jobs
This allows running the regression tests.

One downside is that we can't easily build the Vulkan overlay layer,
because only x86 binaries of the glslang validator are available. If
that's important, we could either use those binaries via qemu, or build
it from source.

v2:
* Add :amd64 suffix to existing debian-9/10 job names (Eric Engestrom)

Acked-by: Eric Engestrom <eric.engestrom@intel.com> # v1
2019-10-22 10:26:29 +00:00
Michel Dänzer c5aa2711a4 gitlab-ci: Explicitly list debian-10 in needs: for .deqp-test template
Apparently needs: in a definition overwrites inherited ones. So
.deqp-test effectively didn't declare needs: for debian-10, which means
any jobs based on .deqp-test could spuriously run after the debian-10
job failed or was cancelled.
2019-10-22 10:26:29 +00:00
Dylan Baker 449f831088 Revert "gitlab-ci: Disable meson-mingw32-x86_64 job again for now"
This reverts commit d60b8679a4.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-16 23:26:09 +00:00
Dylan Baker 6e375ff1aa gitlab-ci: Add a pkg-config for mingw
The one debian provides is broken in buster+, so I've just written my
own. This allows meson to find the installed zlib and prevents it from
falling back to wraps.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-16 23:26:09 +00:00
Eric Engestrom 3bcd54f3fc gitlab-ci: set a common job parent for test stage
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-10-15 17:42:39 +01:00
Eric Engestrom aba78c2d38 gitlab-ci: set a common job parent for build stage
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-10-15 17:42:39 +01:00
Eric Engestrom 81b98e99cd gitlab-ci: set a common job parent for container stage
While at it, rename to singular "container" for consistency.

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-10-15 17:42:39 +01:00
Michel Dänzer d60b8679a4 gitlab-ci: Disable meson-mingw32-x86_64 job again for now
The wrapdb.mesonbuild.com SSL certificate expired, causing the job to
fail: https://gitlab.freedesktop.org/mesa/mesa/-/jobs/731864

Switching to http:// doesn't avoid it:
https://gitlab.freedesktop.org/daenzer/mesa/-/jobs/732043
2019-10-11 11:10:01 +02:00
Michel Dänzer eb86cbabe6 gitlab-ci: Add .use-debian-10 template
It simplifies the definitions of jobs using the Debian 10 image.

The needs: was previously missing from the llvmpipe/softpipe test jobs,
so they could spuriously run if the debian-10 job failed or was
cancelled.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-11 10:05:21 +02:00
Michel Dänzer 9691329727 gitlab-ci: Remove redundant .meson-cross template script
It was identical to the one inherited from the .meson-build template.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-11 10:04:45 +02:00
Dylan Baker d905d9b600 gitlab-ci: Add a mingw x86_64 job
Acked-by: Eric Engestrom <eric.engestrom@intel.com> (v1)
Acked-by: Kristian H. Kristensen <hoegsberg@google.com>
2019-10-10 16:33:05 -07:00
Tomeu Vizoso 555c0de8c6 gitlab-ci: Move LAVA-related files into top-level ci dir
In preparation for testing drivers other than Panfrost in LAVA labs.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-06 07:47:41 -07:00
Tomeu Vizoso 7b01f725dd gitlab-ci: Run dEQP on devices with Panfrost
Include Panfrost's gitlab.ci.yml file from Mesa's main .gitlab-ci.yml so
we test on devices with Panfrost.

This uses LAVA to schedule jobs in the devices and will be the base for
testing Etnaviv, Lima, etc.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-06 07:47:21 -07:00
Michel Dänzer 4712fdf7ae gitlab-ci: Use per-job ccache
Instead of a single cache shared between all jobs, but reduce the
maximum cache size to 1.5G (from 5G).

Rationale for smaller cache:

Pulling & pushing a 5G cache could take a long time. Consider
https://gitlab.freedesktop.org/mesa/mesa/-/jobs/684010 (click the "Show
complete raw" button to see timestamps): Pulling the cache took
1569927241-1569927194 = 47 seconds, pushing it 1569927671-1569927519
= 152, for a total of 199 seconds. The actual build took comparable
1569927518-1569927243 = 275 seconds, despite no cache hits from ccache.
In other words, the cache transfers almost doubled the job duration,
and they would have negated any build time benefits from ccache even
with a high cache hit rate.

Also, the smaller caches avoid blowing up storage requirements for them
too much.

Rationale for per-job caches:

Making a single cache significantly smaller might result in cached
build products from one job getting evicted by another job, reducing
the likelihood of cache hits from previous pipelines.

v2:
* Move up "ccache --max-size=1500M" call (Eric Engestrom)

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-03 09:26:11 +02:00
Michel Dänzer e55df4c859 gitlab-ci: Set ccache path for cross compilers in meson cross file
Without this, meson didn't pick up ccache for cross builds.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2019-10-01 11:16:33 +02:00
Michel Dänzer f2b8051d69 gitlab-ci: Add test-container:arm64 to needs: for arm64 test jobs
Without this, it was theoretically possible for the jobs to run before
the docker image was ready.

v2:
* Use - list syntax instead of [] (Eric Engestrom)

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-30 09:17:44 +02:00
Michel Dänzer 42a18280e4 gitlab-ci: Add needs: for x86 buster docker image
This allows most build jobs to run before the stretch or arm64 docker
images are ready.

v2:
* Use - list syntax instead of [] (Eric Engestrom)

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-30 09:17:38 +02:00
Michel Dänzer 88319f2678 gitlab-ci: Declare needs: for stretch docker image
This allows the *-old-llvm jobs to run before the buster docker images
are ready.

v2:
* Use - list syntax instead of [] (Eric Engestrom)

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-30 09:17:00 +02:00
Michel Dänzer 88e5796daa gitlab-ci: Merge scons-nollvm and scons-llvm jobs
The new job tests scons without LLVM and with all LLVM versions >= 6.0.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-09-18 10:36:48 +00:00
Michel Dänzer baa5024e24 gitlab-ci: Test scons with all LLVM versions
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-09-18 10:36:48 +00:00
Michel Dänzer 0374aacac0 gitlab-ci: Move scons build/test commands to a separate shell script
Preparatory, no functional change intended.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-09-18 10:36:48 +00:00
Michel Dänzer 8a19992869 gitlab-ci: Create separate docker images for Debian stretch & buster
Pros:
* Less fragile due to not mixing packages from stretch and buster
* No longer need to use third-party LLVM packages
* The buster image now uses GCC 8 for C++ as well (previously 6 for C++,
  8 for C), allowing to drop some hacks

Con:
* The stretch image now only uses GCC 6 for C as well as C++
* Need separate jobs for testing old LLVM versions

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-09-18 10:36:48 +00:00
Michel Dänzer 2259b45174 gitlab-ci: Reference full ci-templates commit hash
8 digits might become ambiguous at some point.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-09-18 10:36:48 +00:00
Adam Jackson 6ec1259423 ci: Run tests on i386 cross builds
Yes, some tests fail, but we can turn those into XFAILs at meson time.
Better to keep the things that work working than not cover them at all.
Unfortunately XPASS results will not cause the build to fail until we
update CI to meson 0.51 or newer.

Reviewed-by: Daniel Stone <daniels@collabora.com>
2019-09-17 14:53:57 -04:00
Michel Dänzer e536446b60 gitlab-ci: Move up meson-arm64 job definition
This might allow the arm64 tests to start running earlier.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-16 12:51:35 +02:00
Michel Dänzer cccb68b407 gitlab-ci: Move dependencies/needs for meson-main job to .deqp-test
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-16 12:51:35 +02:00
Michel Dänzer 128581d0d8 gitlab-ci: Simplify some job definitions by extending more similar jobs
v2:
* Preserve setting NIR_VALIDATE=0 for all arm64_* jobs
* Preserve setting DEQP_SKIPS=deqp-default-skips.txt for
  arm64_a306_gles2 jobs

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-16 12:51:34 +02:00
Michel Dänzer e426f40097 gitlab-ci: Use multiple inheritance instead of YAML references
Support for multiple inheritance was added to GitLab recently.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-16 12:51:34 +02:00
Michel Dänzer 0173e9b1ca gitlab-ci: Add needs stanza to arm64_a306_gles2 job definition
This allows the arm64_a306_gles2 jobs to run as soon as the meson-arm64
job has finished.

Fixes: 6f0dc087b7 "freedreno: Introduce gitlab-based CI."
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-09-16 12:51:34 +02:00
Eric Engestrom a0f8a07308 gitlab-ci: rename stages to something simpler
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-09-13 13:26:09 +01:00
Eric Anholt 6f0dc087b7 freedreno: Introduce gitlab-based CI.
Since freedreno's kernel and GPU reset seem to be totally solid, we
don't need to have the complexity of the LAVA setup that panfrost has.
Instead, we can register some boards as shared gitlab runners and have
the jobs run out of a docker container just like we do for llvmpipe.
Just make sure that the DRI device node is passed through to the
containers in the gitlab config ('devices = ["/dev/dri"]' under
runners.docker).

If a runner fails (networking dies, kernel panic, etc.) it'll take out
one build but the rest can keep going since gitlab-runner is what
pulls jobs.  Since the runner pulls jobs, it also means that they can
live behind firewalls instead of needing some public address to be
accessed by gitlab.fd.o.

For now, enable it just on db410c (A307) and cheza (A630) as those are
the hardware that I have plenty of.  A307 is only testing GLES2 since
running all of GLES3 takes too long for the number of boards I've
brought up.

Acked-by: Rob Clark <robdclark@chromium.org>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2019-09-12 10:55:42 -07:00
Adam Jackson 3c553d9cff gallium: Require LLVM >= 3.5
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Jose Fonseca <jfonseca@vmware.com>

[ Michel Dänzer: Dropped jessie line from debian-install.sh again ]
2019-09-11 17:00:43 +00:00
Michel Dänzer 8de25ecd6b gitlab-ci: Use new needs: keyword
This way, the test jobs can start running before all build+test jobs
have finished, once the meson-main job has.

Idea suggested by Daniel Stone on IRC.

See https://docs.gitlab.com/ce/ci/directed_acyclic_graph/ and
https://docs.gitlab.com/ce/ci/yaml/README.html#needs for details.

v2:
* Improve commit log (Daniel Stone, Eric Engestrom)

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-08-30 11:27:00 +02:00
Michel Dänzer 42f8d5a531 gitlab-ci: Move up meson-main job definition
In order to increase the chance of it running early.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-08-30 11:25:26 +02:00
Samuel Pitoiset 7d1c091143 gitlab-ci: bump LLVM to 8 for meson-vulkan and meson-clover
To fix pipeline builds.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-08-23 08:13:31 +02:00
Eric Anholt 553cd82d64 gitlab-ci: Enable the GLES2/3 CTS on softpipe.
The GLES2 CTS takes about 8 minutes of total runtime (at parallel 4 is
~2 minutes in the test stage if runners are free), while GLES3 takes
about 25.  Since the GLES3 run is pretty expensive, just do a cheap
touch test of 1 out of every 10 tests in the test list on MRs, until
we can get the runtime down.

v2: Drop the full run for now until we can bring runtime down or bring
    up a dedicated mesa runner.

Reviewed-by: Eric Engestrom <eric@engestrom.ch> (v1)
Reviewed-By: Gert Wollny <gert.wollny@collabora.com> (v1)
2019-08-20 13:31:13 -07:00
Eric Anholt 46daaca55e gitlab-ci: Run the GLES2 CTS on llvmpipe.
This is the start of doing CTS tests on merges to Mesa master.  We use
the surfaceless platform so that we don't need to bother bringing up
weston or X11.  The surface size is kept low to reduce runtime, but
this comes at the cost of many rendering tests skipping due to
too-small render targets (as we see the impact of Mesa on the shared
runner pool, we can reevaluate this and what set of CTS tests we want
to run).

We split the job up across 4 runners (each at 4 llvmpipe threads), so
that the job can load-balance across our shared runners and finish
sooner (since dEQP is very single-thread-performance bound).

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-08-13 10:30:01 -07:00
Eric Anholt ab49873b44 gitlab-ci: Switch the meson-main build type to debugoptimized.
Now that we're running the drivers we build, building with
optimization is important for keeping our runtime down.  Shaves about
4 minutes of runtime off of GLES2 CTS of llvmpipe at 64x64.

v2: Only switch meson-main until we enable CTS for other builds
    on request by Michel.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-08-13 10:30:01 -07:00
Eric Anholt f417ced5cc gitlab-ci: Build the CTS in the debian build image.
This will let us reuse the image for test runs.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-08-13 10:30:01 -07:00
Eric Anholt 030aa6e184 gitlab-ci: Move around which builds cover which swrast.
I want to enable CI of llvmpipe out of the meson-main build.  So, kick
classic swrast/osmesa to meson-i386, then promote llvmpipe to
meson-main (along with nine, now that classic osmesa isn't keeping it
out of there).

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-08-13 10:29:34 -07:00
Eric Engestrom 1c82fa0a92 gitlab-ci: simplify $CROSS option
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2019-08-10 12:11:28 +01:00
Eric Engestrom 9be5ce1d73 gitlab-ci: generate meson cross-files earlier
Suggested-by: Michel Dänzer <michel@daenzer.net>
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-08-09 20:07:50 +01:00
Eric Engestrom 32ce010951 gitlab-ci: don't install autotools deps
These could've been deleted a long time ago, but apparent we forgot.

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
2019-08-07 10:18:25 +01:00
Eric Engestrom c32ebfe003 gitlab-ci: add gtk3 dev files for `-D tools=intel-ui`
We also need to update wayland-protocols and libXrandr (and randrproto),
as they are too old for gdk3 (which gtk3 depends on).

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
2019-08-07 08:19:30 +01:00
Eric Engestrom 6125c93e00 gitlab-ci: just build all the tools
This line was mistakenly added while there is already a `-D tools=all`
a few lines below.

Fixes: f60defa72d ("gitlab-ci: Add a shader-db run using v3d on drm-shim.")
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-08-02 16:41:19 +01:00
Michel Dänzer 5229f27f06 gitlab-ci: Fix scons build directory path
Fixes: dd3d0b2897 "gitlab-ci: Only keep the build logs as artifacts."

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-07-30 16:18:50 +02:00
Eric Anholt f60defa72d gitlab-ci: Add a shader-db run using v3d on drm-shim.
This provides significant compiler coverage during CI at a fairly low
cost in CPU time (~17s per thread for 4 threads on
gst-gitlab-htz-runner3).

I'm leaving wget in the docker image, as once this is in master I'm
planning on having an automatic shader-db comparison between master
and the branch included in the artifacts.  I also haven't done
freedreno yet, because it has some races when run in multithreaded
mode that I'm still tracking down.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-07-25 08:56:55 -07:00