Write tracie results to the 'results/junit.xml' file using the JUnit XML
format. Among other uses, this file can be picked up by gitlab to
display more useful information to the user.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6598>
Provide some sensible values for required environment variables to allow
tests to run properly in any environment.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6598>
As drivers have been tested with more and more traces, the yml file is
becoming a bit unwieldy. As more drivers are going to be tested with
traces, and more traces will be used, split them in per-driver files so
the size stays manageable.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Reviewed-By: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6185>
As these credentials are valid only for 15 minutes, generate them closer
to when they are going to be used.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6124>
Developers can see how the rendering differed from the executed value.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6113>
After a trace succeeds, check if the rendered image already exists in
the repository of reference images, and upload it if it doesn't.
This image will be used for comparing with failed retraces.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6113>
I'm seeing occasional unexpected 403 errors when uploading artifacts.
Print the response in case MinIO is telling us why.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6101>
Put artifacts in a per-job folder, because if a job is retried then it
will try to upload a file to the same key and fail with the following
error:
403 Client Error: Forbidden for url:
https://minio-packet.freedesktop.org/artifacts/daenzer/mesa/180609/gl-panfrost-t860/results.yml
Also, to prevent in the future similar clashes if several trace files
share the same name, upload the images with their checksums as their
names. This will also make it easier to fetch images for comparison with
the references.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6029>
Upload failed images and the results.yml file to MinIO, to facilitate
debugging.
Also, fix version checking when git is installed as Mesa is going to
output a different renderer string if git is installed.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-By: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5472>
Downloading the traces directly from git causes very high egress from
GCE, which is expensive.
So we can expand trace testing further, we are going to keep a cache in
freedesktop.org's MinIO instance. This commit implements downloading
from it.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-By: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5472>
v2: Python's nitpick (Andres)
In case of an empty list of traces, the results.yml will contain an empty
curly braces. In YAML, an associative array can also be specified by text
enclosed in curly braces ({}),
This commit also adds the corresponding test to check the behavior of
tracie when no traces are added in the traces.yml file.
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4916>
test_tracie_skips_traces_without_checksum does the logic previous to
the commit 8546d1dd78. The traces.yml includes
several traces, only the one without checksum is ignored by tracie.
As a complementary action, this change adds an new test
(test_tracie_only_traces_without_checksum) to verify the behavior for
cases where the traces.yml only contains traces without checksum.
Finally, test_tracie_skips_traces_without_checksum is renamed as
test_tracie_traces_with_and_without_checksum
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4916>
v2: Verbatim translation from the original shell script
Make the corrections visible in explicit commits (Andres)
Remove redundant code (Alexandros)
Code style nitpick (Rohan)
Reimplementation of the tracie's self-tests using a pythonic test suit
(pytest).
The new tracie/test.py module is almost a direct translation of the
tests defined in the tracie/test.sh. This new implementation of the
test provides a more common framework where define the tests.
Also allows a better introspection for the tests results and/or
resulting errors.
This patch also adds python3-pytest as dependency for the built images
and adapts the tracie-runner scripts to run the self-test using pytest.
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com> [v1]
Reviewed-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com> [v1]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4916>
Change the main function to receive the args parameter from
sys.argv[1:]. The args parameter will be passed to the
ArgumentParser.parse_args() function as argument.
This change provides an easier main() function signature to use
with pythonic testsuites.
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4916>
RESULTS_PATH and RESULTS_PATH, as variables in the module context, are
resolved one single time, only during the first module loading. If the
the Python code in execution changes the current dir at some point,
those paths are not going to be updated anymore keeping the paths
wrongly pointing to the old working dir.
This change modify the definition of those variables to use simply
relative paths.
Signed-off-by: Pablo Saavedra <psaavedra@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4916>
Otherwise, we will fail when the traces description file doesn't
contain any checksum for the specified device.
Fixes: efbbf8bb81 ("tracie: Print results in a machine readable format")
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4839>
v2:
- Update the default location for the traces when there is no
traces-db entry in the traces definition file (Alexandros).
Fixes: 90a39af5f6 "(ci: Drop the git dependency in tracie)"
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4640>
This change was missing after acf7e73be5 "(gitlab-ci: make explicit
tracie is gitlab specific)".
Fixes: acf7e73be5 "(gitlab-ci: make explicit tracie is gitlab specific)".
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4638>
Tracie main script and traces.yml file talk about repo(sitory) when it
actually means GitLab's project.
Since the script is GitLab's API specific, make it clear.
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4496>
When running tracie in a docker container, renaming files from
inside the container to a bind-mounted folder on the host causes
a invalid cross-device link due to os.rename limitations.
Switching to shutil allows us to overcome this.
Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4377>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4377>
We are currently comparing a hex string representation of the git lfs
OID with a byte array representation of the locally calculated OID,
causing detection of valid cached traces to fail. Ensure we are
comparing compatible representations (in this case hex strings).
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4300>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4300>
Instead of using git, use python and the Gitlab API
to fetch traces. This helps us slim down our ramdisks
in preparation for integrating trace replay on LAVA
devices.
Signed-off-by: Rohan Garg <rohan.garg@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4000>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4000>
In preparation for using apitrace to replay traces in LAVA jobs, build a
newer waffle so apitrace can use the surfaceless EGL platform.
As things were before this commit, Xvfb would have been needed in the
LAVA images.
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4000>
Also, adds an example job for radv.
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
In preparation to having "vk" (Vulkan) along "gl" (OpenGL/ES).
This is so it is clearer which traces belong to which API and also for
the build jobs.
Signed-off-by: Andres Gomez <agomez@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Introduce automated testing of Mesa by replaying traces with Renderdoc
or Apitrace.
For now only LLVMPipe is tested, but other drivers can be tested if
there's runners with the necessary hardware.
Signed-off-by: Alexandros Frantzis <alexandros.frantzis@collabora.com>
Signed-off-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2935>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2935>