Commit Graph

367 Commits

Author SHA1 Message Date
Tomeu Vizoso ad3ef6d0fc gitlab-ci: Test virgl driver
Add virglrenderer to the container and use the vtest transport to test
the Gallium driver. On the "host", llvmpipe is used.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4433>
2020-04-20 13:53:09 +00:00
Daniel Stone 791134658e Revert "CI: Disable Windows/VS2019 builds"
DNS is now fixed.

This reverts commit 460b8b1758d953b2b820443615d73ccdb1455b5e.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4591>
2020-04-16 22:40:24 +00:00
Daniel Stone 7a794b1de4 CI: Disable Windows/VS2019 builds
An update seems to have poisoned gitlab-runner, and it can no longer
resolve DNS even though the host system can.

Disable this until we can figure out what's going on.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4589>
2020-04-16 14:03:49 +01:00
Alexandros Frantzis 4c6ce826af gitlab-ci: Check the Mesa version used for tracie tests
Verify that the Mesa version used when running tracie tests is the
one that was built by CI, rather than any installed distro version.

Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3894>
2020-04-14 11:34:02 +00:00
Michel Dänzer 6a8e5dde66 gitlab-ci: Use all_paths in .test-manual rules
Without this, the .test-manual jobs could end up as 'when: manual' when
the jobs they depend on were 'when: never', which was flagged as
invalid YAML, preventing the pipeline from being created.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4522>
2020-04-13 10:29:48 +00:00
Michel Dänzer 4176dfa880 gitlab-ci: Run merge request pipelines automatically only for Marge Bot
MR pipelines not triggered by Marge Bot can still be triggered manually.

Motivation: The main & forked Mesa project CI pipelines combined are
currently generating over 1 TB of egress traffic per week. ~80% of this
is from pre-merge pipelines. Assuming this corresponds to 4 pre-merge
and one post-merge pipeline per MR on average, this change could
potentially eliminate up to ~60% of the overall traffic (by preventing
3 of the 4 pre-merge pipelines from running automatically).

(Of course, this could be subverted if all jobs of the other pipelines
were triggered manually anyway... In most cases, manually triggering
just a few jobs should suffice)

v2:
* $GITLAB_USER_NAME was the wrong variable, $GITLAB_USER_LOGIN should
  do the trick.

Suggested-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-07 17:36:15 +00:00
Michel Dänzer 42fe600c0c gitlab-ci: Don't require triggering build/test jobs manually
Let them run automatically once all their dependencies have passed.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-07 17:36:15 +00:00
Michel Dänzer c12576efbe gitlab-ci: Rename "paths" YAML anchor to "all_paths"
To avoid confusion with `paths:` elements.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432>
2020-04-07 17:36:15 +00:00
Eric Anholt c1e7e83d52 ci: Consistently use -j4 across x86 build jobs and -j8 on ARM.
Our shared runners are set up for concurrent jobs ~= CPUs / 4 (x86) or 8
(ARM).  If you use more build processes than that, then jobs may be
fighting each other for shared system resources, possibly to the point of
failure (we've seen one of the runners OOM on some jobs before, though I'm
not sure if this was the cause).

To try to systematically prevent the problem, we make a ninja wrapper in
the containers that passes the -j flags, and set MAKEFLAGS in the
container builds.  This doesn't cover make in non-container builds, but I
believe we don't have any of those.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3782>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3782>
2020-04-01 18:33:58 +00:00
Daniel Stone 9197fd59da CI: Re-enable Windows VS2019 builds
The failures are fixed, but I didn't notice this had been silently
disabled in !4272.

Re-enable the VS2019 build.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4374>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4374>
2020-03-30 16:22:20 +00:00
Jose Fonseca 27d58a1c20 gitlab-ci: Prune all SCons jobs except scons-win64, and allows failures.
Based on the discussion in https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4352

Reviewed-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4363>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4363>
2020-03-30 14:52:34 +00:00
Jose Fonseca 2e92d33819 scons: Prune out unnecessary targets.
This prunes out all targets except libgl-gdi, libgl-xlib, and svga, as
suggested by Marek Olšák.

libgl-xlib will be remove once I have had time to confirm no automated
tests we have rely upon it.

There are also a bunch of Makefile.sources which become orphaned as
result, that are not taken care of in this change.

v2: Prune remainders of swr support.

Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4348>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4348>
2020-03-30 13:38:01 +00:00
Daniel Stone 2db1d73e53 CI: Avoid htz4 runner for VS2019
The htz4 runner needs to be updated in order for our support binaries
like Chocolatey to work. Temporarily restrict jobs to the EC2 runner
until this has happened.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4371>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4371>
2020-03-30 10:19:35 +01:00
Michel Dänzer fcd3377cfe gitlab-ci: Update to current templates
The .fdo.container-ifnot-exists template has been replaced by
.fdo.container-build.

We need to include "debian/" in FDO_REPO_SUFFIX for now, we can drop it
for individual images when their tags are bumped if we want.

Miscellaneous other goodies this gets us:
* The templates now add some labels to images which may be useful for
  garbage collecting unused tags in the future.
* The templates now copy the current tag from the main project
  registry to the forked project's if it already exists in the latter
  but points to a different image hash. This will avoid false failures
  (or passes) due to using the wrong image.

Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4286>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4286>
2020-03-28 16:12:38 +00:00
Rob Clark 6a10397a01 gitlab-ci: disable vs2019 build
Seems to be broken atm and blocking merging anything.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4272>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4272>
2020-03-27 16:34:27 -07:00
Andres Gomez fb8fa83a30 gitlab-ci: add Wine, win64's apitrace and DXVK to the Vulkan testing container
In preparation for having automated testing with DXGI traces.

v2:
  - Updated DXVK version.
  - Merged the new Wine container into the existing Vulkan
    one (Michel).

v3:
  - Updated commit log.
  - Use a particular known-good apitrace version (Alexandros).

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4238>
2020-03-27 13:48:17 +00:00
Andres Gomez 05a3b49308 gitlab-ci: Don't use buster-backports packages by default for x86_test-vk
The backports repository can be temporarily inconsistent between
architectures, which can break the docker image build.

Suggested-by: Michel Dänzer <mdaenzer@redhat.com>
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4238>
2020-03-27 13:48:17 +00:00
Daniel Stone 07885cbcdb CI: Add native Windows VS2019 build
Adds a native build of Mesa using Meson with the Visual Studio 2019
toolchain on a Windows host.

Though Docker is supported on Windows, Docker-in-Docker is not possible,
nor are podman and skopeo available. We handle this by creating the
container from a shell-executor Windows machine, which gives us a native
PowerShell that we can execute Docker from. This attempts to do the same
copy-from-upstream-or-create-if-not-exists optimisation as the
ci-templates do for our Linux builds, albeit open-coded in PowerShell.

The Mesa build itself is executed inside a container, using Meson and
Ninja.

Signed-off-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Acked-by: Jose Fonseca <jfonseca@vmware.com>
Acked-by: Brian Paul <brianp@vmware.com>
Acked-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4304>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4304>
2020-03-27 10:32:47 +00:00
Tomeu Vizoso b123849880 gitlab-ci: Serve files for LAVA via separate service
Currently, we store the kernel and ramdisk for each LAVA job in the
artifacts of the job that built them. Because artifacts are stored in
GCE and LAVA labs aren't, this causes a lot of egress with is expensive.

To avoid this, have runners download most of the data via the (cached)
container images once, and for each job upload the kernel and ramdisk to
a server outside GCE.

Right now we only have Collabora's runner with a local web server, so
jobs that go to Baylibre's lab have been disabled.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4295>
2020-03-26 09:30:48 +00:00
Tomeu Vizoso 92f3c51560 gitlab-ci: Place files from the Mesa repo into the build tarball
There's some files from the .gitlab-ci directory that are needed in the
test stage and that, because the Mesa repository isn't checked out in
that stage, need to be made available through other means.

Because those files are going to be needed in LAVA devices, place them
ino the tarball containing the built files so it's available to both
gitlab-ci runners and LAVA devices.

Before those files were passed in the artifacts of the Gitlab CI job,
but this commit places them into the built tarball so scripts later in
the pipeline don't need to account for this discrepancy.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4295>
2020-03-26 09:30:48 +00:00
Andres Gomez 8bc3d6574c gitlab-ci: add python3-requests to the test-vk container
After 90a39af5f6 ("ci: Drop the git dependency in tracie"), we have
this error in the radv-polaris10-traces job:

"
...

+ /builds/tanty/mesa/artifacts/tracie/tests/test.sh
tracie_succeeds_if_all_images_match: Fail
Traceback (most recent call last):
  File "/tmp/tracie.test.glY0O23HJo/tracie.py", line 6, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

...
"

v2:
  - Updated commit log to be more descriptive (Michel).

Fixes: 90a39af5f6 ("ci: Drop the git dependency in tracie")
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4237>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4237>
2020-03-23 17:17:56 +02:00
Samuel Pitoiset 48e920315c gitlab-ci: add a new stage for RADV CI
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4082>
2020-03-23 12:16:02 +00:00
Samuel Pitoiset e22d562c17 gitlab-ci: compile fossils with more ASICs
I think we want to cover these 3 generations at the barely minimum.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4082>
2020-03-23 12:16:02 +00:00
Samuel Pitoiset 1517e58c1b gitlab-ci: compile fossils with both RADV compiler backends (LLVM/ACO)
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4082>
2020-03-23 12:16:02 +00:00
Eric Anholt 8edaa843ab ci: Move db820c and db410c's gles3 tests to manual, like radv did.
This should make these tests available for clicking on the web ui in
personal branches, while hiding them from marge and the post-merge CI
pipelines.  We had already disabled db410c's gles3, but it wasn't
available in the ui and you had to hack .gitalb-ci.yml.  db820c is now
being disabled by default, due to instaboots mentioned in
https://gitlab.freedesktop.org/mesa/mesa/issues/2649

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4247>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4247>
2020-03-20 18:27:37 +00:00
Michel Dänzer 130c0ba1cc gitlab-ci: Restrict s390x/ppc64el jobs to packet runners
They are hitting timeouts on the gstreamer runners now... *sigh*

Reviewed-by: Adam Jackson <ajax@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4233>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4233>
2020-03-20 16:39:17 +01:00
Eric Anholt f778c48869 ci: Only run the freedreno baremetal tests when freedreno/core changes.
Same as we do for a630 (docker) tests.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Acked-by: Rob Clark <robdclark@chromium.org>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4229>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4229>
2020-03-19 17:21:24 +00:00
Eric Anholt 4bc15e78fa ci: Enable testing GLES2-3 on a530 (Dragonboard 820c).
Following on from the db410c conversion to baremetal testing, reuse the
same scripts in the same rack to run 7 db820c boards (#4/8 is failing in
the bootloader for unknown reasons).

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4177>
2020-03-17 11:11:51 -07:00
Eric Anholt af7dca3560 ci: Update the ci-templates commit.
There has been a big rename of variables in the upstream repo to make it
clear what's being handed to ci-templates.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4099>
2020-03-17 10:44:37 -07:00
Michel Dänzer a0591863db gitlab-ci: Enable more Gallium drivers in meson-i386 job
These are the ones which can be enabled with the current x86_build
docker image and which build without warnings.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4166>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4166>
2020-03-17 11:21:45 +01:00
Michel Dänzer c56f09124b gitlab-ci: Move classic driver testing to a new meson-classic job
The motivation is to allow llvmpipe to be enabled instead in the
meson-i386 job.

v2: (Eric Engestrom)
* Rename meson-main job to meson-gallium
* Remove stale comment above meson-i386 job

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4166>
2020-03-17 11:20:23 +01:00
Michel Dänzer c3727ae431 gitlab-ci: Fold scons-swr job into scons job
Should be fast enough.

Acked-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4166>
2020-03-17 11:17:10 +01:00
Rohan Garg 90a39af5f6 ci: Drop the git dependency in tracie
Instead of using git, use python and the Gitlab API
to fetch traces. This helps us slim down our ramdisks
in preparation for integrating trace replay on LAVA
devices.

Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4000>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4000>
2020-03-17 07:23:27 +01:00
Tomeu Vizoso 2ca662fb61 gitlab-ci: Update renderdoc
Get closer to upstream to avoid accumulating changes.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4000>
2020-03-17 07:12:36 +01:00
Samuel Pitoiset 299fad5585 gitlab-ci: bump Vulkan CTS to 1.2.1.0
Vulkan CTS 1.1.6.0 is quite old.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4179>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4179>
2020-03-16 14:36:41 +00:00
Samuel Pitoiset af6d8dea00 gitlab-ci: do not set the number of deqp-parallel jobs for RADV CTS
Let's the runner uses the maximum number of jobs to speedup CTS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4179>
2020-03-16 14:36:41 +00:00
Eric Anholt cf5ba9d409 ci: Make a simple little bare-metal fastboot mode for db410c.
This supports powering up the device (using an external tool you
provide based on your particular lab), talking over serial to wait for
the fastboot prompt, and then booting a fastboot image on a target
device.

I was previously relying on LAVA for this, but that ran afoul of
corporate policies related to the AGPL.  However, LAVA wasn't doing
too much for us, given that gitlab already has a job scheduler and
tagging and runners.  We were spending a lot of engineering on making
the two systems match up, when we can just have gitlab do it directly.

Lightly-reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4076>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4076>
2020-03-11 21:36:47 +00:00
Samuel Pitoiset 529c0ba219 gitlab-ci: build RADV in meson-i386 to avoid 32-bit build failures
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4044>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4044>
2020-03-11 19:30:13 +00:00
Michel Dänzer 86d270cde4 gitlab-ci: Don't restrict ppc64el/s390x build jobs to gstreamer runners
The packet runners have beefier CPUs now and don't seem to run into
test timeouts anymore.

Reviewed-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4128>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4128>
2020-03-11 17:20:40 +00:00
Andres Gomez f5235a5b73 gitlab-ci: Remove unneeded python3-pilkit dependency
It was added with tracie, but it doesn't depend on it.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
2020-03-11 16:17:05 +02:00
Samuel Pitoiset 8f5543990e gitlab-ci: add rules:changes for RADV
Including mesa_core_file_list is probably not the best but it's
better than nothing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4117>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4117>
2020-03-11 08:04:05 +01:00
Pierre-Eric Pelloux-Prayer 2a9d6fdd8c gitlab-ci: rules:changes to test on tested drivers changes
For now tests only use these drivers:
  * llvmpipe
  * softpipe
  * freedreno
  * lima
  * etnaviv
  * panfrost

So using rules:changes gitlab feature to run the tests when the changes
made are potentially affecting these drivers.

A few notes:
  * the following code:
      .piglit-test:
        extends:
          - .test-gl
          - .llvmpipe-rules
    makes gitlab replace .test-gl "rules:changes" values by the one from
    ".llvmpipe-rules".
  * rules:changes always matches for non-MR new branches so jobs will always be
    created (and they'll be run if their dependencies are run). For pushes to
    existing branches the files changed by the push are used to match the
    rules:changes path.
  * the same gitlab feature could be used for some build jobs

Acked-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2569>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2569>
2020-03-09 16:31:55 +01:00
Michel Dänzer 0103f02acb gitlab-ci: Always name artifacts archive after the job producing it
This will help determine which artifacts generate how much traffic.

v2:
* Add "mesa_" prefix to make it obvious which project the artifacts are
  from.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4085>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4085>
2020-03-07 11:09:50 +01:00
Samuel Pitoiset 42a3d821cb gitlab-ci: add a job that runs Fossilize on RADV/Polaris10
RADV_FORCE_FAMILY forces creating a null device that allows RADV
to be instanced without AMDGPU.

The Fossilize database only contains pipelines from the Sascha
Vulkan triangle demos at the moment. I will add more once this
is merged.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3960>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3960>
2020-03-05 20:33:56 +00:00
Samuel Pitoiset af1cd45858 gitlab-ci: enable building the test image for VK unconditionally
It was diabled because RADV is the only driver that tests Vulkan
and running CTS on my personal machine and without recovery is
not safe enough for CI (too long and too unstable).

Now that we are going to test Fossilize with RADV, it's needed to
build the test image for VK unconditionally. As RADV now supports
creating NULL devices, the fossilize jobs can run everywhere.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3960>
2020-03-05 20:33:56 +00:00
Samuel Pitoiset 1cdb6edbe6 gitlab-ci: add Fossilize support to detect compiler regressions
Fossilize is equivalent to vkpipeline-db but it's definitely more
robust. This is based on the CI traces system.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3960>
2020-03-05 20:33:56 +00:00
Samuel Pitoiset 93fcc9ad57 gitlab-ci: build Fossilize in the test image for VK
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3960>
2020-03-05 20:33:56 +00:00
Michel Dänzer cc9493f78e gitlab-ci: Distribute jobs across more stages
The stages and mapping of jobs to them are somewhat arbitrary; the goal
is to avoid having to scroll through large numbers of jobs.

v2: (Pierre-Eric Pelloux-Prayer)
* Use even more stages for test jobs
* Give somewhat meaningful names to stages

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3995>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3995>
2020-03-05 12:46:51 +01:00
Michel Dänzer 71436f9640 gitlab-ci: Drop "test-" prefix from llvmpipe/softpipe job names
Redundant.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3995>
2020-03-05 12:45:51 +01:00
Andres Gomez 0ac731b1ff gitlab-ci: Add jobs to be able to test Vulkan
Also, adds an example job for radv.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
2020-03-04 15:24:03 +02:00
Andres Gomez 5c65f8b377 gitlab-ci: Add gfxreconstruct traces support
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
2020-03-04 15:24:03 +02:00
Andres Gomez 1d75595da4 gitlab-ci: Change devices format to <api-vendor-deviceId>
In preparation to having "vk" (Vulkan) along "gl" (OpenGL/ES).

This is so it is clearer which traces belong to which API and also for
the build jobs.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
2020-03-04 15:22:04 +02:00
Andres Gomez f1b7b8c0ee gitlab-ci: build VulkanTools into the Vulkan testing container
In preparation for having automated testing with Vulkan traces.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
2020-03-04 15:21:58 +02:00
Andres Gomez 028ab482bf gitlab-ci: build gfxreconstruct into the Vulkan testing container
In preparation for having automated testing with Vulkan traces.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
2020-03-04 15:21:47 +02:00
Andres Gomez fc2338dc44 gitlab-ci: add missing popd to the build-deqp-vk.sh script
Since we are at it, replace "cd" with pushd / popd and homogenize how
VK-GL-CTS is built in comparison with other build scripts.

Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
2020-03-04 15:21:39 +02:00
Eric Anholt aea8c9c7b1 ci: Flip db410c back to docker mode.
Turns out there's corporate policy to not deploy AGPL software, so I
have to take down the LAVA lab until we sort out how to do it without
a local server.

Reviewed-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4038>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4038>
2020-03-03 12:49:09 -08:00
Eric Anholt 5865944635 ci: Switch testing on db410c over to LAVA.
This should get us better stability of the db410c boards by having a
smaller per-board software stack, with no disks involved (just initramfs).
Additionally, the new cluster is 7 (soon 8) db410cs, while currently the
docker cluster only has 1/4 of its db410cs still running.

Unfortunately, we have to prepare the fastboot boot image during the ARM
drivers build stage, because LAVA relies on publicly available URLs for
the images to load into the bootloaders of the boards, and the only thing
we have for that is gitlab's artifacts.

Note that this testing relies on the boards being freshly flashed with the
linaro v136 firmware to pick up the initramfs size fixes and to stop the
boot at fastboot.

Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3661>
2020-03-02 11:38:46 -08:00
Eric Anholt 3c7c021ffc ci: Enable -Werror on meson-vulkan and meson-testing.
I want to make sure that I don't introduce warnings in turnip where we
have active work going on, and I also want to make sure that the drivers
we care about testing are warnings-clean.

As with the previous -Werror change, this is for CI only and doesn't
affect end-user builds.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3607>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3607>
2020-02-27 21:59:34 -08:00
Eric Anholt 33f38605e9 ci: Include db410c support in the ARM container.
I'm working on moving the db410c CI from docker to LAVA, which means we
get to boot a custom kernel.  To do that, we need to enable ARCH_QCOM in
the kernel, save the dtb around, and include abootimg in our container so
that we can generate combined kernel/dtb/ramdisk images for fastboot.

LAVA's fastboot support is unable to pack the overlay into an abootimg
image, just a cpio rootfs.  We could flash the cpio rootfs after overlay
addition, but that takes 2 minutes to do, and causes wear on the devices.
Instead, we'll bring up the network at boot and use wget to fetch the
overlay.  We'll want network support anyway, so that we can transfer the
failure xmls back to the gitlab job's artifacts at some point.

Since the msm GPU and realtek network firmware increase our payload by
3MB, add in firmware compression so that it doesn't waste as much RAM on
devices not using it.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3928>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3928>
2020-02-27 09:36:26 -08:00
Eric Anholt 257415863b ci: Remove LLVM from ARM test drivers.
The LLVM libraries were a significant fraction of the entire payload
(55M/250M uncompressed) into the initramfs of the test boards, but
LLVM is only used for the draw module used in select/feedback (which
isn't even tested in CI on ARM yet).

Assume that llvmpipe draw is safe enough for ARM given the coverage on
x86, and disable LLVM for these jobs.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3928>
2020-02-27 09:36:19 -08:00
Rohan Garg 9c0bbba856 ci: Split out radv build-testing on arm64
radv needs libllvm which increases our ramdisk size
significantly. Since this driver is only build tested,
we can split it out into a separate job.

Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3928>
2020-02-27 09:36:16 -08:00
Tomeu Vizoso ebd071d8cf gitlab-ci: Move to 5.5 kernel plus fixes for Panfrost
There's two fixes that help with stability when running dEQP on Kevin
Chromebooks.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3876>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3876>
2020-02-26 14:03:04 +01:00
Eric Anholt 54dbb55ea8 ci: Make sure that we have a proper shell prompt for LAVA.
LAVA finds a '#' early in boot and races to emit its shell commands.
Apparently for the current boards those serial commands end up getting
buffered such that things work out, but for db410c and db820c, the buffer
is lost and LAVA gets stuck waiting for the prompt.  By setting a prompt,
we can delay our commands until we're actually supposed to emit them (and
suppress a complaint from the lava dispatcher that we're using a risky
prompt!)

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3883>
2020-02-21 15:28:38 -08:00
Alexandros Frantzis 803ab5d6be gitlab-ci: Automated testing with OpenGL traces
Introduce automated testing of Mesa by replaying traces with Renderdoc
or Apitrace.

For now only LLVMPipe is tested, but other drivers can be tested if
there's runners with the necessary hardware.

Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2935>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2935>
2020-02-20 08:06:13 +01:00
Dave Airlie 8f5a252d35 ci: bump debian image and change llvm deps to 8
v3: remove version in a few places (Michel)

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3805>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3805>
2020-02-15 04:15:00 +00:00
Fritz Koenig 2a98cf3b2e Revert "gitlab-ci: disable a630 tests as mesa-cheza is down (again)"
This reverts commit 18657c0c0a

Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3804>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3804>
2020-02-13 18:13:55 +00:00
Michel Dänzer 2303762735 gitlab-ci: Only use gstreamer runners for the s390x job for now
The fdo-packet-* runners keep hitting the (already quite long) timeouts
for some of the tests, taking many times as long for them as the
gstreamer runners.

The fdo-gitlab-gce-runner3 runner would work as well, but it doesn't
have any tags we could use.

Acked-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3760>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3760>
2020-02-11 09:59:08 +01:00
Samuel Pitoiset 18657c0c0a gitlab-ci: disable a630 tests as mesa-cheza is down (again)
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3758>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3758>
2020-02-10 10:34:41 +01:00
Kristian H. Kristensen 26ab38f144 ci: Drop turnip opt-in option
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3742>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3742>
2020-02-07 01:36:58 +00:00
Eric Anholt 658eb691fc ci: Bump the GLES CTS version to 3.2.6.1.
This brings in the surfaceless fixes so we don't need to check out the
whole repo to cherry pick any more (which was bothering me as I debugged
things late in the painfully slow ARM container build process).

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3662>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3662>
2020-02-06 15:18:24 -08:00
Michel Dänzer 8be81f8a2a gitlab-ci: Build radeonsi & RADV in the ppc64el job
This requires cross-building libdrm for ppc64el.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3643>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3643>
2020-02-05 10:52:31 +00:00
Michel Dänzer 65610ec774 gitlab-ci: Add ppc64el and s390x cross-build jobs
Using LLVM 8 for ppc64el and 7 for s390x (which hits some coroutine
related issues with LLVM 8).

There are some test failures we need to ignore for now. Also, the
timeout needs to be bumped from the default 30s for some tests, because
they can take longer under emulation.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3643>
2020-02-05 10:52:31 +00:00
Michel Dänzer 84fefa206c gitlab-ci: Update to latest ci-templates HEAD
Among other things, this increases robustness when copying a docker
image from the main Mesa project to a forked project, avoiding spurious
image rebuilds from scratch.

Also drop the comment about .gitlab-ci/lava-gitlab-ci.yml, it doesn't
include the templates anymore.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3643>
2020-02-05 10:52:31 +00:00
Tomeu Vizoso 9afdcd64f2 gitlab-ci: Switch kernel for LAVA jobs to 5.5
All fixes we were carrying in our branch have been merged already.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3692>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3692>
2020-02-04 15:46:34 +01:00
Fritz Koenig 42f7e124ca Revert "gitlab-ci: disable a630 tests as mesa-cheza is down"
This reverts commit f38851d84c

Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3687>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3687>
2020-02-03 21:45:16 +00:00
Eric Engestrom f38851d84c gitlab-ci: disable a630 tests as mesa-cheza is down
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3677>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3677>
2020-02-03 10:08:25 +00:00
Eric Anholt f6e59911e5 ci: Enable -Werror on the meson-i386 build.
I find warnings to be very disruptive to my workflow (using emacs's "go to
next error" feature), and I periodically have to go clean up other
people's drivers to get back to finding my own warnings in the noise.  I
know I'm not the only one doing something like this.

We don't want to enable -Werror by default in builds, since it means that
end users will have builds spuriously fail based on what compiler version
and opt flags they have compared to what the devs are using.  However, it
is quite easy to have CI ensure that we at least don't introduce warnings
on the compiler version that it uses.

For now I've just enabled it on meson-i386 to cover a bunch of Mesa core
and get us started on ratcheting up warnings-cleanliness in the tree,
without me having to fix up all the drivers at once.

Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3539>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3539>
2020-01-28 12:31:07 -08:00
Michel Dänzer 5a6a88f58c gitlab-ci: Use single if for manual job rules entry
I thought multiple ifs would all need to match, but looks like only the
last one (or either one?) does.

This should prevent a manual pipeline from getting created after merging
changes which can't affect the pipeline.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3474>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3474>
2020-01-22 16:42:11 +00:00
Michel Dänzer 2dd0cc60f1 gitlab-ci: Set GIT_STRATEGY to none for the dummy job
It doesn't need anything from the Git repository.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3474>
2020-01-22 16:42:11 +00:00
Matt Turner 17c9ec94f5 gitlab-ci: Skip ext_timer_query/time-elapsed
This test's result is unpredictable, so it may occasionally pass when we
expect it to fail, thus causing the CI pipeline to fail.

Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3498>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3498>
2020-01-22 00:53:48 +00:00
Michel Dänzer 8775b742ea gitlab-ci: Stop using manual jobs for merge requests
They were causing trouble with Marge Bot: The project settings require
that the pipeline succeeds before a merge request (MR) can be merged,
otherwise Marge doesn't wait for the pipeline to succeed before merging
an MR assigned to her. But Marge can't start manual jobs, so she would
always time out waiting for pipelines with manual jobs.

To avoid this, use these rules:
* Run the pipeline by default for MRs and main project branches changing
  any files affecting it.
* For other MRs, run a single dummy job which always succeeds.
* Don't run any jobs for main project branch changes (e.g. from an MR
  having been merged) not affecting the pipeline.
* Allow jobs to be started manually on branches of forked projects, as
  before.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3361>
2020-01-15 10:31:01 +00:00
Tomeu Vizoso 22d976454f gitlab-ci: Consolidate container and build stages for LAVA
Use the normal build job to also prepare the artifacts for LAVA jobs.

For that, the build container needs to also build the test suites,
kernel, ramdisk, etc.

Then the build job will place the just-built Mesa in the ramdisk and the
test job can generate a LAVA job and point to those artifacts.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3295>
2020-01-14 13:17:24 +00:00
Michel Dänzer 5f0ff004ca gitlab-ci: Test against LLVM / clang 9 on x86
They're not available for Debian buster yet, so we have to use upstream
snapshot packages again.

In contrast to earlier, we now store the LLVM APT repository key in Git
instead of re-downloading it every time.
2020-01-07 11:00:16 +01:00
Tomeu Vizoso d62dd8b0cb gitlab-ci: Switch LAVA jobs to use shared dEQP runner
Take one step towards sharing code between the LAVA and non-LAVA jobs,
with the goals of reducing maintenance burden and use of computational
resources.

The env var DEQP_NO_SAVE_RESULTS allows us to skip the procesing of the
XML result files, which can take a long time and is not useful in the
LAVA case as we are not uploading artifacts anywhere at the moment.

Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2020-01-06 14:27:36 +01:00
Dave Airlie e6b2af56cb llvmpipe: switch to NIR by default
Add LP_DEBUG=tgsi_ir (tgsi already taken) to fallback to TGSI paths.

Disable NIR_VALIDATE in CI (Michel/Eric acked)

Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2303>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2303>
2019-12-21 13:07:17 +10:00
Samuel Pitoiset ff2e11b210 gitlab-ci: set RADV_DEBUG=checkir for RADV test jobs
This is used to validate if the driver emits correct LLVM IR.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-12-11 15:44:40 +00:00
Michel Dänzer dadd609664 gitlab-ci: Don't exclude any piglit quick_shader tests
Now that we're running these with process isolation enabled, their
results will hopefully be stable.

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-12-10 11:19:11 +00:00
Samuel Pitoiset c1a362722f gitlab-ci: add a job that runs Vulkan CTS with RADV conditionally
Only Polaris10 is tested at the moment, and I disabled a TON of
tests to keep a CTS run within 5 minutes because my local runner
is a bit slow. A full CTS run takes more than 1h, which means it
will hit the timeout.

RADV CI can only be triggered manually on personal branches to
avoid breaking the world because one runner is definitely not
enough. This will allow us to test it until it's stable enough
to be enabled by default.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:58:03 +01:00
Samuel Pitoiset 40c6a56751 gitlab-ci: build RADV in meson-testing
This requires to bump LLVM to 8 because it's the minimum supported
version by RADV.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:58:00 +01:00
Samuel Pitoiset 16b999b7d1 gitlab-ci: allow to run dEQP Vulkan with DEQP_VER
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:57:55 +01:00
Samuel Pitoiset 0b246d3558 gitlab-ci: add a new base test job for VK
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:57:54 +01:00
Samuel Pitoiset f2a594f384 gitlab-ci: add a new job that builds a base test image for VK
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:57:48 +01:00
Samuel Pitoiset 520a77d486 gitlab-ci: add a gl suffix to the x86 test image and all test jobs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-06 10:57:46 +01:00
Michel Dänzer 41797a1fed gitlab-ci: Overhaul job run policy
Use new rules: instead of only:

For container stage jobs:

* In the main Mesa project, run them by default.

* In merge requests, run them by default if any files affecting pipeline
  results are changed.

* In all other cases (in particular branches in personal projects),
  don't run them by default but allow triggering them manually.

build & test stage jobs are left at the default (when: on_success), so
they will run automatically once all their dependencies are satisified.
(Using the same rules as above would require these jobs to be manually
triggered as well, which is only possible once all dependency jobs have
passed) Please be considerate of CI runner resources and cancel unneeded
jobs on personal branches with no corresponding merge requests (this can
be done before the jobs start running).

In summary: No more special branch names. Unnecessary job runs are
avoided by default, but jobs which don't run by default can be triggered
manually.

v2:
* Split out LAVA changes to separate commit
* Clarify commit log a little, in particular WRT build/test stage jobs

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> # v1
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> # v1
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> # v1
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-12-06 10:02:01 +01:00
Eric Anholt 3097efe5f0 ci: Disable egl_ext_device_drm tests in piglit.
If the runner has a HW device that would be supported, even without
/dev/dri forwarded into the container, it will be enumerated and the tests
on llvmpipe fail with (for example):

libEGL warning: Not allowed to force software rendering when API explicitly selects a hardware device.
libEGL warning: MESA-LOADER: failed to open i965 (search paths /builds/anholt/mesa/install/lib/dri)

Given that we can't necessarily control the DRI devices present on the
runners (particularly for developers bringing their own runners to reduce
the demands on fd.o's shared resources), just skip these tests in CI.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-12-05 18:06:10 +00:00
Tapani Pälli 2d26cc077d gitlab-ci: bump piglit checkout commit
Commit also updates the Piglit quick_gl.txt, list modifications happened
due to following Piglit commits: c248bf201,c acff58ca, 5603e2e60.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2019-12-04 15:27:41 +00:00
Michel Dänzer 5585b8eadd gitlab-ci: Run piglit glslparser & quick_shader tests separately
And only use --process-isolation false for the quick_gl tests.

This will hopefully avoid variance in the test results that we've been
seeing lately. But even if it doesn't, it should at least help narrow
down the cause of the variance.

Tested-by: Vasily Khoruzhick <anarsoul@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
2019-12-04 10:36:33 +01:00
Rob Clark b31637c453 gitlab-ci: disable junit results for deqp
They don't seem to be hugely useful, and seem to be bogging down gitlab.

Signed-off-by: Rob Clark <robdclark@chromium.org>
2019-12-03 08:46:39 -08:00
Michel Dänzer 0c88d5952a gitlab-ci: Update to current ci-templates master
Fixes skopeo copy failures.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
2019-12-03 16:03:31 +01:00
Michel Dänzer a3b3d3bfcc gitlab-ci: Put HTML summary in artifacts for failed piglit jobs
This will make it easier to look at details of failed / skipped tests.

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-27 10:20:31 +01:00
Michel Dänzer c9cdb7cef0 gitlab-ci: Stop reporting piglit test results via JUnit
It was basically useless in this form, and processing the JUnit data in
the GitLab backend was pretty expensive.

Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-27 10:18:33 +01:00
Rob Clark 9f422cbe1c gitlab-ci/deqp: generate junit results
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-22 13:48:29 -08:00
Rob Clark 8af7551a9e gitlab-ci: bump arm test container
To pick up updated cts_runner and netcat for the flake reporting.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-22 13:48:29 -08:00
Rob Clark fdaf777076 gitlab-ci/deqp: detect and report flakes
If there are a small number of fails, re-run to determine if they are
flakes, and optionally (if `$FLAKES_CHANNEL` configured) report the
flakes.

This way flakes don't interfere with developers working on other
drivers, but get logged so that the developers working on the flaking
driver can monitor the situation.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-22 13:48:29 -08:00
Samuel Pitoiset 6f7ec6ee39 gitlab-ci: reduce the number of scons build
It seems overkill to me to build scons 7x for every pipeline.
Scons is now build with the oldest llvm version in scons-old-llvm
and with the newest llvm version in scons.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-22 10:39:21 +01:00
Eric Anholt dd76a6f198 ci: Move freedreno's parallelism to the runner instead of gitlab-ci jobs.
I set the runners to concurrency=1, so they serve only one gitlab-ci job
at at time.  Swap over to using the parallel runner now to keep the
runners busy, more efficiently than spawning many docker containers and
downloading artifacts multiple times, and producing easier-to-understand
results for browsing on the web.

This bumps the a306 runners to 4x parallel instead of 2x like before, but
cheza gles3 drops from 6 to 4.  Current rough timings of the jobs (if no
container download):

db410c-gles2: 5:00
a630-gles2: 1:30
a630-gles3: 6:00
a630-gles31: 5:30

a630-gles3 is a bit longer than I like, but it should come back down once
I can sort out the NIR algebraic rewinding.
2019-11-21 05:48:17 -08:00
Michel Dänzer 32618ee719 gitlab-ci: Directly use host-mapped directory for ccache
Use hardcoded /cache/mesa/ccache for the cache, so it will be shared by
all jobs of all Mesa projects running on the same runner host. This
should increase the hit rate and decrease the worst case storage used.

Further benefits of directly using a host-mapped directory:

* Saves up to ~1 minute per job for restoring and saving the cache
  contents via the GitLab CI cache mechanism
* Cache contents generated by failed jobs are no longer lost
* Jobs running in parallel on the same runner host can get hits from
  each other

Also enable compression, so the default maximum cache size of 5G might
be sufficient.

v2:
* Move CCACHE_DIR variable to the .build-linux template

Suggested-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Anholt <eric@anholt.net> # v1
2019-11-21 10:13:43 +01:00
Samuel Pitoiset 0d1085ac4a gitlab-ci: remove now useless meson-swr-glvnd build job
All things are already part of meson-main.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset 7362176cfe gitlab-ci: build GLVND in meson-clang
Building GLVND in meson-main doesn't work because this disables
libEGL and it's needed for running shader-db.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset e6d26d77a3 gitlab-ci: build swr in meson-main
Now that debugoptimized isn't set and that all test jobs depend on
meson-testing, enabling swr shouldn't slowdown the CI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset 6cf9b53fa2 gitlab-ci: do not build with debugoptimized for meson-main
This should reduce compile time because optimizations are costly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:05 +01:00
Samuel Pitoiset 66b5627074 gitlab-ci: add a job that only build things needed for testing
For turnip and RADV testing, we will need a debugoptimized build
without UBSAN. This introduces meson-testing which builds only the
things that are needed by the test stage.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:35:04 +01:00
Samuel Pitoiset eab328fbe9 gitlab-ci: fix ldd check for Vulkan drivers
The 'dri' directory isn't created when building Vulkan drivers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-21 09:34:08 +01:00
Timothy Arceri 530d3b2900 gitlab-ci: bump piglit checkout commit 2019-11-20 05:05:56 +00:00
Michel Dänzer c6c7652753 gitlab-ci: Organize images using new REPO_SUFFIX templates feature
Two benefits:

Most docker image related environment variables can now be defined in
the jobs where they're used instead of globally. The DEBIAN_TAG values
are propagated to other jobs via YAML anchors.

Images on https://gitlab.freedesktop.org/mesa/mesa/container_registry
are now organized in separate repositories with a suffix matching the
name of the job which makes sure the image is there.

Acked-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:23:22 +01:00
Michel Dänzer 506e9d5fc7 gitlab-ci: Rename container install scripts to match job names (better)
Cleans up .gitlab-ci/ a little, and allows using a single DEBIAN_EXEC
line for all container jobs.

v2:
* Use lava_arm.sh instead of arm_lava.sh for consistency with v2 of the
  previous change

Reviewed-by: Eric Anholt <eric@anholt.net> # v1
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:21:10 +01:00
Michel Dänzer 3a48f4565e gitlab-ci: Use functional container job names
This makes it easier to tell which job is which in a pipeline.

v2:
* Use lava_arm{64,hf} instead of arm{64,hf}_lava to keep these jobs
  together in pipeline overviews

Reviewed-by: Eric Anholt <eric@anholt.net> # v1
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:20:16 +01:00
Michel Dänzer 670277846d gitlab-ci: Document that ci-templates refs must be in sync
Otherwise there can be weird breakage.

(Removing the include from .gitlab-ci/lava-gitlab-ci.yml doesn't seem
possible unfortunately:
https://gitlab.freedesktop.org/daenzer/mesa/pipelines/79458)

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-15 16:06:54 +01:00
Tapani Pälli 7a893a0d57 gitlab-ci: update Piglit commit, update skips
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-15 12:06:15 +02:00
Eric Engestrom ac78ca4b39 gitlab-ci: auto-cancel CI runs when a newer commit is pushed to the same branch
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-14 17:32:31 +00:00
Samuel Pitoiset 47ba227448 gitlab-ci: build RADV on ARM64
The ARMHF LLVM package is LLVM 7 but RADV requires LLVM 8.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-13 10:52:10 +01:00
Samuel Pitoiset cb19f69ff0 gitlab-ci: build a specific libdrm version for ARM64
RADV requires libdrm-2.4.100 but the distrib package is too old.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-13 10:52:08 +01:00
Eric Anholt 52843ec5d3 ci: Enable all of GLES3/3.1 testing for softpipe.
Now that we're not using so many job slots, it's easy to get these
jobs run in a reasonable amount of time (gles3 took 10 minutes for 4
cores, and gles31 was 15 minutes for 4 cores).

Acked-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-12 12:54:04 -08:00
Eric Anholt f08c810028 ci: Use cts_runner for our dEQP runs.
This runner is a little project by Bas, written in C++, that spawns
threads that then loop grabbing chunks of the (randomly shuffled but
consistently so) test list and hand it to a dEQP instance.  As the
remaining list gets shorter, so do the chunks, so hopefully the
threads all complete effectively at once.  It also handles restarting
after crashes automatically.  I've extended the runner a bit to do
what I was doing in the bash scripts before, like the skip list and
expected failures handling.  This project should also be a good
baseline for extending to handle retesting of intermittent failures.

By switching to it, we can have the swrast tests just take up one job
slot on the shared runners and keep their allotment of CPUs busy,
instead of taking up job slots with single-threaded dEQP jobs.  It
will also let us (eventually, once I reprovision) switch the freedreno
runners over to threading within the job instead of running concurrent
jobs, so that memory scribbles in one pipeline don't affect unrelated
pipelines, and I can experiment with their parallelism (particularly
on a306 where we are frequently backed up) without trashing other
people's jobs.

What we lose in this process is per-test output in the log (not a big
loss, I think, since we summarize fails at the end and reducing log
length keeps chrome from choking on our logs so badly).  We also drop
the renderer sanity checking, since it's not saving qpa files for us
to go poke through.  Given that all the drivers involved have fail
lists, if we got the wrong renderer somehow, we'd get a job failure
anyway.

v2: Rebase on droppong of the autoscale cluster and the arm64
    build/test split.  Use a script to deduplicate the cts-runner
    build.
v3: Rebase on the amd64 build/test container split.

Acked-by: Daniel Stone <daniels@collabora.com> (v1)
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com> (v2)
2019-11-12 12:54:04 -08:00
Eric Anholt 66719e0242 ci: Use several debian buster packages instead of hand-building.
This helps cut down our container build time.  I've left a few that
we're likely to rev more frequently or I was less confident in
dropping.

v2: Rebase on the build/test container split, now bumps the build
    container tag in this commit.

Acked-by: Eric Engestrom <eric.engestrom@intel.com> (v1)
Acked-by: Daniel Stone <daniels@collabora.com> (v1)
2019-11-12 12:54:04 -08:00
Eric Engestrom 06347989a0 gitlab-ci: build libdrm using meson instead of autotools
Autotools was deprecated for a while and has now been removed, so let's
start using meson here so that we won't have any issues next time we
update libdrm.

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-11-12 17:08:02 +00:00
Michel Dänzer aebf43dcc1 gitlab-ci: Use separate docker images for x86 build/test jobs
Same as was done for the ARM images before.

This should make it less painful to update to newer dEQP / piglit as
well as to make changes to the build/test environment.

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:17:21 +01:00
Michel Dänzer 576f7b6ea5 gitlab-ci: Run piglit tests with llvmpipe
One job for the quick_gl profile, one for the glslparser & quick_shader
profiles (doing these together takes hardly any more time than
quick_shader alone).

v2:
* Don't break lava tests
v3:
* Remove piglit test artifacts paths:
* Exclude some quick_shader tests again:
  - Test whose result flips between pass/fail/skip
  - *@vs_in tests, as not the same one of these gets picked every time
v4:
* Do not list passing tests in .gitlab-ci/piglit/*.txt (Eric Anholt)
* Include the test number summary in .gitlab-ci/piglit/*.txt
* Completely disable generating any vs_in tests in the piglit build.
* Remove some more unneded files from the piglit build tree.
* Exclude quick_gl arb_gpu_shader5 tests; they were all skipped anyway,
  as llvmpipe doesn't support this extension yet, but occasionally they
  would spuriously fail instead.
v5:
* Set LD_LIBRARY_PATH, so we actually test the Mesa build from the
  pipeline...
* Verify that wflinfo reports the expected Mesa version
* Pass -noreset to Xvfb
v6:
* Don't use autoscale runners, run piglit with -j4 (Eric Anholt)

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:16:23 +01:00
Michel Dänzer df26e18b9f gitlab-ci: Share dEQP build process between x86 & ARM test image scripts
See https://gitlab.freedesktop.org/mesa/mesa/issues/2056

v2:
* Rename .gitlab-ci/deqp-build.sh => .gitlab-ci/build-deqp.sh
  (Eric Anholt)

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:14:49 +01:00
Michel Dänzer 59fcb019d0 gitlab-ci: Move artifact preparation to separate script
It's currently only needed for the meson-main and meson-arm64 jobs, not
the other meson build jobs.

Also remove MESON_SHADERDB, just run .gitlab-ci/run-shader-db.sh
directly from the meson-main job.

v2:
* Also run prepare-artifacts.sh in meson-arm64 script
v3:
* Move tarball creation into the new script as well, as it prevented
  ccache --show-stats from running in after_script

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com> # v1
Reviewed-by: Eric Anholt <eric@anholt.net>
2019-11-12 10:14:26 +01:00
Eric Anholt cb655d2554 Revert "ci: Switch over to an autoscaling GKE cluster for builds."
This reverts commit c9df92bf79.

It turns out that gitlab-runner uses kubernetes all wrong, spawning Pods
and sshing into them to run the script instead of Jobs containing the
script to run.  This means that when anything goes wrong with the pod
(autoscale, preemption, VM maintenance, cluster reconfiguration), the job
fails and only sometimes gets handled as a runner system failure.  Even
worse, due to bugs in either the runner or k8s itself, some classes of
timeout-related failure end up not being reported as failures, and the job
will incorrectly report success!

Disable using the "autoscale" cluster until we can do something else
(docker-machine instead of k8s, or the custom third-party k8s-native
runner).

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Acked-by: Daniel Stone <daniels@collabora.com>
2019-11-06 11:38:07 -08:00
Michel Dänzer b4d3ae2269 gitlab-ci: Use separate arm64 build/test docker images
The image used for test jobs is only about 1/6 as big as before, which
may help avoid some issues with some of the test boards.

Inspired by https://gitlab.freedesktop.org/mesa/mesa/issues/2046 .

v2:
* Leave LIBDRM_VERSION at 2.4.99 (Daniel Stone)
* Delete more build artifacts from dEQP tree (Daniel Stone)
v3:
* Set LD_LIBRARY_PATH for ldd

Acked-by: Daniel Stone <daniels@collabora.com> # v2
Reviewed-by: Eric Anholt <eric@anholt.net> # Except for the ldd line
2019-11-06 13:10:07 +01:00
Michel Dänzer d80dece065 gitlab-ci: Set arm job CCACHE_DIR properly
$PWD doesn't work for variables:, it ended up as "/ccache", always
starting with an empty cache.

v2:
* Use relative path and realpath
v3:
* Use $CI_PROJECT_DIR (Eric Anholt)
* Clear ccache stats in before_script if the cache is in $CI_PROJECT_DIR

Fixes: c9df92bf79 "ci: Switch over to an autoscaling GKE cluster for
                     builds."
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-11-05 09:27:32 +01:00
Eric Anholt c9df92bf79 ci: Switch over to an autoscaling GKE cluster for builds.
The GKE pool we're using is 1-3 32-core VMs, preemptible (to keep
costs down), with 8 jobs concurrent per system.  We have plenty of
memory (4G/core), so we run make -j8 to try to keep the cores busy even
when one job is in a single-threaded step (docker image download, git
clone, artifacts processing, etc.)  When all jobs are generating work
for all the cores, they'll be scheduled fairly.

The nodes in the pool have 300GB boot disks (over-provisioned in space
to provide enough iops and throughput) mounted to /ccache, and
CACHE_DIR set pointing to them.  This means that once a new
autoscaled-up node has run some jobs, it should have a hot ccache from
then on (instead of having to rely on the docker container cache
having our ccache laying around and not getting wiped out by some
other fd.o job).  Local SSDs would provide higher performance, but
unfortunately are not supported with the cluster autoscaler.

For now, the softpipe/llvmpipe test runs are still on the shared
runners, until I can get them ported onto Bas's runner so they can be
parallelized in a single job.

Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-10-31 11:19:43 -07:00
Eric Engestrom 9b58ab803d gitlab-ci: build a recent enough version of GLVND (ie. 1.2.0)
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
2019-10-31 17:09:59 +00:00
Michel Dänzer aaf1b09270 gitlab-ci: Disable meson-windows job for the time being
It needs a CI runner carrying the mesa-windows tag, but there's none
available currently.
2019-10-30 09:38:20 +01:00
Michel Dänzer 2a38fc1027 gitlab-ci: Only run the pipeline if any files affecting it have changed
E.g. documentation-only changes cannot affect the outcome of the
pipeline, so don't waste resources on running it.

The thing we need to be careful about here is that the container stage
jobs must always run if any later stage jobs using the corresponding
docker images run. We're currently using the same .ci-run-policy
template for all jobs, so this is trivially true.

v2:
* Add bin/ and common.py (Eric Engestrom)

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> # v1
Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-29 15:09:56 +00:00
Alexandros Frantzis 1257d06ba7 gitlab-ci: Update required libdrm version
Commit 9edcce2a32 bumped the required libdrm-amdgpu version to
2.4.100. Update the version we use in our CI scripts to avoid CI
build failures.

Also bump the debian image name for this change to take effect.
Note that amdgpu is only built with the debian-buster image,
so only this image requires an update.

Fixes: 9edcce2a ("ac: get tcc_harvested from the kernel")
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
2019-10-29 09:50:09 +00:00
Erik Faye-Lund 15e7f94278 gitlab-ci: fixup debian tags
When resolving a merge-conflict, I accidentally only updated the
ARM64-tag tag. Let's correct this.

Fixes: 3d529c1739 ("gitlab-ci: also build Zink on CI")
2019-10-28 12:07:30 +01:00
Erik Faye-Lund 3d529c1739 gitlab-ci: also build Zink on CI
This prevents accidentally breaking the driver-build while working on
other drivers.

Signed-off-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
2019-10-28 08:51:48 +00:00
Dylan Baker 19851c9ad6 gitlab-ci: Add a job for meson on windows
This adds a new CI job that runs on windows with MSVC. It currently
builds softpipe and osmesa, and runs the related unit tests. It does
rely on meson's wraps for zlib, but I've set up caching of the wrap
dependencies so hopefully that wont be a problem.

I really wanted to user powershell for this, but there just isn't an
easy way to do that, it's much easier to use batch scripts, so thats
what I used.

The leading `/` for .gitlab-ci/lava... must be removed because windows
doesn't understand it, and when it reads the file the job ends in error.

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-10-25 22:47:32 +00:00
Dylan Baker 06e4647cb0 gitlab-ci: refactor out some common stuff for Windows and Linux
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-10-25 22:47:32 +00:00
Dylan Baker 54053bc8d0 scons: Print a deprecation warning about using scons on not windows
At this point meson should be able to handle all of the non-windows
platforms just fine; we'd like to be able to stop maintaining scons for
those platforms sooner than later.

Reviewed-by: Eric Anholt <eric@anholt.net>
2019-10-24 18:33:50 +00:00
Michel Dänzer 75cc8c0b82 gitlab-ci: Enable UBSan for the meson-vulkan job
It doesn't report any errors now.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
2019-10-24 16:21:48 +02:00
Michel Dänzer 793f6b30d9 gitlab-ci: Enable llvmpipe in ARM build jobs
v2:
* Use LLVM 8 from buster-backports
v3:
* Use LLVM 7 again for armhf, llvmpipe is still broken there with LLVM 8

Acked-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-22 10:26:29 +00:00
Michel Dänzer 163ec5d808 gitlab-ci: Use native aarch64 runner for ARM build jobs
This allows running the regression tests.

One downside is that we can't easily build the Vulkan overlay layer,
because only x86 binaries of the glslang validator are available. If
that's important, we could either use those binaries via qemu, or build
it from source.

v2:
* Add :amd64 suffix to existing debian-9/10 job names (Eric Engestrom)

Acked-by: Eric Engestrom <eric.engestrom@intel.com> # v1
2019-10-22 10:26:29 +00:00
Michel Dänzer c5aa2711a4 gitlab-ci: Explicitly list debian-10 in needs: for .deqp-test template
Apparently needs: in a definition overwrites inherited ones. So
.deqp-test effectively didn't declare needs: for debian-10, which means
any jobs based on .deqp-test could spuriously run after the debian-10
job failed or was cancelled.
2019-10-22 10:26:29 +00:00
Dylan Baker 449f831088 Revert "gitlab-ci: Disable meson-mingw32-x86_64 job again for now"
This reverts commit d60b8679a4.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-16 23:26:09 +00:00
Dylan Baker 6e375ff1aa gitlab-ci: Add a pkg-config for mingw
The one debian provides is broken in buster+, so I've just written my
own. This allows meson to find the installed zlib and prevents it from
falling back to wraps.

Reviewed-by: Eric Engestrom <eric.engestrom@intel.com>
2019-10-16 23:26:09 +00:00
Eric Engestrom 3bcd54f3fc gitlab-ci: set a common job parent for test stage
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
2019-10-15 17:42:39 +01:00