Commit Graph

12 Commits

Author SHA1 Message Date
Emma Anholt 3f8114d1e0 ci/bare-metal: Get rid of servo's serial feed threads.
If the SerialBuffers can just feed the same line queue, then we don't need
the extra threads reading line queues into a new merged line queue.

Less python threading code is always better.  Plus, now we can pass args
to SerialBuffer.lines() for timeout/phase.

Acked-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17096>
2022-06-21 21:38:25 +00:00
Emma Anholt ca453714aa ci/bare-metal: Add per-boot-stage timeouts for fastboot and poe.
This should avoid the 1-hour timeouts if something goes wrong, and just
restart.

Fixes: #6682
Acked-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17096>
2022-06-21 21:38:25 +00:00
Emma Anholt 1e15ec1949 ci/bare-metal: Apply autopep8 to our python scripts.
My editor likes to pep8 as I edit, and I'm tired of carefully not
committing those changes.

Acked-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17096>
2022-06-21 21:38:25 +00:00
Ilia Mirkin eb0b08ea1a gitlab-ci: serial close can leave an active read
So instead cancel the read first, and then close. Make sure the
serial-reading properly detects this cancelled condition under all
circumstances and exits.

Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14033>
2021-12-03 23:26:27 +00:00
Emma Anholt 8f5a0bd9b4 ci/bare-metal: Close serial and join serial threads before exit.
This should fix the intermittent (~1/week) cheza failure where python
complains that a thread tried to do stdio while the main thread has
exited.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13462>
2021-11-10 20:36:57 +00:00
Juan A. Suarez Romero 72b68bd2a6 ci: add testing for VC4 drivers (Raspberry Pi 3)
This tests OpenGL ES 2.0 CTS suite with VC4 drivers, through baremetal
Raspberry Pi 3 devices.

The devices are connected to a switch that supports Power over Ethernet
(PoE), so the devices can be started/stopped through the switch, and
also to a host that runs the GitLab runner through serial-to-USB cables,
to monitor the devices to know when the testing finishes.

The Raspberries uses a network boot, using NFS and TFTP. For the root
filesystem, they use the one created in the armhf container. For the
kernel/modules case, this is handled externally. Currently it is using
the same kernel/modules that come with the Raspberry Pi OS. In future we
could build them in the same armhf container.

At this moment we only test armhf architecture, as this is the default
one suggested by the Raspberry Pi Foundation. In future we could also
add testing for arm64 architecture.

Finally, for the very rare ocassions where the Raspberry Pi 3 device is
booted but no data is received, it retries the testing for a second
time, powering off and on the device in the process.

v2:
 - Remove commit that exists capture devcoredump (Eric)
 - Squash remaining commits in one (Andres)

v3:
 - Add missing boot timeout check (Juan)

v4:
 - Use locks when running the PoE on/off script (Eric)
 - Use a timeout for serial read (Eric)

v5:
 - Rename stage to "raspberrypi" (Eric)
 - Bump up arm64_test tag (Eric)

v6:
 - Make serial buffer timeout optional (Juan)

Reviewed-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7628>
2020-12-04 19:05:19 +01:00
Eric Anholt bf29daa1b5 ci/deqp: Switch to a new dEQP runner written in Rust.
I found the C++ runner hard to develop on, and we had stability issues and
outstanding feature needs that made me want something I felt good about
hacking on.  Thus, Rewrite It In Rust of the deqp runner.

The new runner includes:

- Skip lists don't reshuffle the test list.
- Known-flake handling without resorting to skip lists (fixing our main CI
  reliability issue on a3xx right now).
- Per-thread Vulkan shader caches should speed up VK CI runtime.
- Tracking of crashes separate from fails (so we can see progress on that
  front).
- Logging of deqp stderr spam (particularly assertion failures!) in the CI
  log.
- Integrated QPA filtering so we don't have bash perf issues for it.
- Logging of what caselist to go look at for a given error report (in red,
  so it's easier to find in your CI log).
- The code is 1/3 unit tests, and easy to extend for more coverage.
- Non-LAVA CI runs create a failures.csv in artifacts that you can check
  in as your deqp-*-fails.txt file.
- Test runtime is included in results.csv so you can debug how to speed up
  your CI job.
- Pretty summary at the end of the run of slow/flaky/failed tests.

Since this is a new runner with a different RNG, the test groups are
shuffled one more time.  This seems to result in some panfrost T720
stability issues (See its new deqp-panfrost-t720-flakes.txt), and one new
flake in freedreno a630.

Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7434>
2020-11-06 19:48:39 +00:00
Eric Anholt fe61230b38 ci/bare-metal: Reset colors at the end of a line of serial output.
We don't want the next line of our timestamp and other context to inherit
colors set by the serial command (visible with the new dEQP runner)

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7434>
2020-11-06 19:48:39 +00:00
Eric Anholt ff6741728d ci/bare-metal: Apply autopep8 to the bare-metal scripts.
Let's follow proper python formatting (easy now that vscode does it for
me)

Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7434>
2020-11-06 19:48:39 +00:00
Eric Anholt fd2ee49b21 ci/bare-metal: Use python for handling fastboot booting and parsing
Modeling after what I did for cros_servo_run.py, this gives us easy
support for restarting the test run a530 when we detect a spontaneous
reboot.  I had to touch up serial_buffer.py to handle buffering in from a
file instead of a serial device, to support the upcoming etnaviv CI
(tested by running it against a serial log from db410c and seeing it step
to calling "fastboot")

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6529>
2020-09-03 23:22:44 +00:00
Eric Anholt 785d3cace4 ci/bare-metal: Include a timestamp in our serial reads.
gitlab CI doesn't include timestamps in its logs by default, but it's
really useful for finding delays in our CI so stuff one in on the lines
coming in from serial and being output to the gitlab log.  The artifacts
file is still the raw serial output.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6529>
2020-09-03 23:22:42 +00:00
Eric Anholt b4374080d7 ci/bare-metal: Use a new serial buffer tool.
This one uses python threads to move some of our logic from shell
pipelines to python, and opens the door to doing better serial output
tracking in the future (the SerialBuffer.lines() method)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6398>
2020-08-21 20:10:18 +00:00