I found the C++ runner hard to develop on, and we had stability issues and
outstanding feature needs that made me want something I felt good about
hacking on. Thus, Rewrite It In Rust of the deqp runner.
The new runner includes:
- Skip lists don't reshuffle the test list.
- Known-flake handling without resorting to skip lists (fixing our main CI
reliability issue on a3xx right now).
- Per-thread Vulkan shader caches should speed up VK CI runtime.
- Tracking of crashes separate from fails (so we can see progress on that
front).
- Logging of deqp stderr spam (particularly assertion failures!) in the CI
log.
- Integrated QPA filtering so we don't have bash perf issues for it.
- Logging of what caselist to go look at for a given error report (in red,
so it's easier to find in your CI log).
- The code is 1/3 unit tests, and easy to extend for more coverage.
- Non-LAVA CI runs create a failures.csv in artifacts that you can check
in as your deqp-*-fails.txt file.
- Test runtime is included in results.csv so you can debug how to speed up
your CI job.
- Pretty summary at the end of the run of slow/flaky/failed tests.
Since this is a new runner with a different RNG, the test groups are
shuffled one more time. This seems to result in some panfrost T720
stability issues (See its new deqp-panfrost-t720-flakes.txt), and one new
flake in freedreno a630.
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7434>
This makes sure that we keep executing the tests so that we can get our
alerts in IRC and know whether the tests are still flaking. It also keeps
us from having adjustments to the skip list causing failures/flakes to
move to different tests (as seen with a530 having to move some xfails
around after changing the skip list)
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6392>
These fields (TILE_ALL and MIN_LAYERSZ) seem to be the same on a5xx as
a6xx, having looked at some UBWC vs non-UBWC texturator cases. Setting
MIN_LAYERSZ does fix the 3D fail we see in the CTS.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5127>
Now that we don't get scheduled to any 19mhz CPUs, the old GLES3 job went
from 12 minutes of deqp-runner runtime to 54s. Increase how much of the
testsuite we cover in exchange, still keeping the runtime at 3-6 min
(compared to previous 10-17 min). Since the tests we're running changed,
reset the xfails list.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5115>
Following on from the db410c conversion to baremetal testing, reuse the
same scripts in the same rack to run 7 db820c boards (#4/8 is failing in
the bootloader for unknown reasons).
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4177>