Commit Graph

21 Commits

Author SHA1 Message Date
David Heidelberg a04720dc1c ci: move the CI handling scripts into bin/ci
Any change in .gitlab-ci/ directory triggers an entire CI run,
which is unnecessary for just changing the developers' scripts.

Added symlink to `.gitlab-ci/bin` to keep original invocation compatibility.

Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20371>
2022-12-19 16:36:19 +01:00
David Heidelberg 07f2c862e0 ci: ci_run_n_monitor.py: fix args.rev leftover
Fixes: a8b6b2367e ("ci: allow omitting of --rev for ci_run_n_monitor.py")

Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20371>
2022-12-19 16:35:39 +01:00
David Heidelberg a8b6b2367e ci: allow omitting of --rev for ci_run_n_monitor.py
When --rev is omitted, try to get revision automatically.

Acked-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20271>
2022-12-16 14:00:37 +01:00
David Heidelberg f745e86391 ci: ci_run_n_monitor fix Unicode log parsing
Fixes issues as  `...truncated \ xXX escape` while parsing the log.

Acked-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20271>
2022-12-16 14:00:27 +01:00
David Heidelberg c415d8f08b ci: ci_run_n_monitor.py: fail when job is not found
Acked-by: Emma Anholt <emma@anholt.net>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19902>
2022-11-21 23:09:30 +00:00
David Heidelberg cdbb30334c ci/update_traces_checksum.py: make output more clean and implement colors
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19510>
2022-11-15 07:19:20 +00:00
David Heidelberg e1d40d11f5 ci/update_traces_checksum.py: check if checksum is in the array, not it's value
Fixes: 45eda06953 ("ci: introduce update_traces_checksum.py")

Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19510>
2022-11-15 07:19:20 +00:00
Guilherme Gallo be3c46964b ci/bin: Remove whitespace from token files
There was a security problem with some `gitlab_gql.py` scenarios because
of `\r` and `\n` in the token file, which interrupted the requests for
Gitlab endpoints.

Stripping the token file after reading the file content solves the
problem.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18066>
2022-10-14 01:46:52 +00:00
Guilherme Gallo d52d51b24d ci/bin: Fix requirements.txt
Add missing aiohttp and PyYAML packages

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18066>
2022-10-14 01:46:52 +00:00
David Heidelberg 45eda06953 ci: introduce update_traces_checksum.py
Invoking this script takes checksums from all failed jobs and update
them in $driver-traces.yml files.

```
.gitlab-ci/bin/update_traces_checksum.py --rev $(git rev-parse HEAD)
```

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18329>
2022-09-13 09:16:19 +00:00
David Heidelberg de08baade7 ci: split ci_run_n_monitor into script and shared parts
These parts will be shared with the update checksum script.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18329>
2022-09-13 09:16:19 +00:00
David Heidelberg 4a4e5320a1 ci: implement stress testing jobs in ci_run_n_monitor.py
Can be triggered by `--stress`.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18014>
2022-08-20 10:05:57 +00:00
David Heidelberg 607bf3cc41 ci: GraphQL was already implemented in ci_run_n_monitor.py
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18014>
2022-08-20 10:05:57 +00:00
David Heidelberg 160aa603f5 ci/bin: ignore Gitlab GQL cache
Fixes: 266e3a627f ("ci/bin: Cache GQL queries")

Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17885>
2022-08-04 22:00:21 +00:00
Guilherme Gallo fcbfce4e18 ci/bin: Print job needs DAG in ci_run_n_monitor
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17791>
2022-08-03 23:10:37 +00:00
Guilherme Gallo 266e3a627f ci/bin: Cache GQL queries
To avoid abusing fd.o Gitlab instance, let's cache API queries that
should have the same response. This will speedup the use of the client
tools as well.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17791>
2022-08-03 23:10:37 +00:00
Guilherme Gallo 101697ceb3 ci/bin: Add script to expand jobs manifest
Normally, Mesa devs bump into CI errors and need to replicate the CI job
environment locally, but that is not an easy task, since one needs to
find the merged YAML CI file from Gitlab interface.

As it can happen often, here is a script that use Gitlab's GraphQL and
finds all the variables set by it, alongside with the container image
used to run it and the script that it will run.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17791>
2022-08-03 23:10:37 +00:00
Guilherme Gallo 39cdcea251 ci/bin: Make ci_run_n_monitor finds dependencies automatically
Use the GraphQL util to find dependencies accurately saving up time for
both developers and runners.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17791>
2022-08-03 23:10:37 +00:00
Guilherme Gallo caf9900b3d ci/bin: Sort imports from ci_run_n_monitor.py
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17791>
2022-08-03 23:10:37 +00:00
Guilherme Gallo 65b6edee3e ci/bin: Add utility to find jobs dependencies
Use GraphQL API from Gitlab to find jobs dependencies in a pipeline.
E.g: Find all dependencies for jobs starting with "iris-"

```sh
.gitlab-ci/bin/gitlab_gql.py --sha $(git -C ../mesa-fast-fix rev-parse HEAD) --print-dag --regex "iris-.*"
```

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17791>
2022-08-03 23:10:37 +00:00
David Heidelberg 6676349c5d ci: add script for manage Mesa CI
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16891>
2022-07-10 00:00:39 +00:00