KonstantinSeurer
/
mesa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

gitlab-ci: Make the test job fail when bugs are unexpectedly fixed. 

If people fix bugs without updating the expected-fails list, then we
end up with a lack of coverage of those failures in the future.  Also,
some day down the line another developer ends up trying to figure out
if the bug was actually fixed or their environment is just failing to
reproduce it.

Suggested-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Acked-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
This commit is contained in:
Eric Anholt 2019-09-12 12:34:50 -07:00
parent 1190c81f8e
commit 7859eb1390
1 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
.gitlab-ci/deqp-runner.sh
View File

@ -123,3 +123,14 @@ if [ -s /tmp/new-fails.txt ]; then
else
echo "No new failures"
fi
sort /tmp/case-list.txt > /tmp/sorted-case-list.txt
comm -12 /tmp/sorted-case-list.txt /tmp/expected-fails.txt > /tmp/expected-fails-in-caselist.txt
comm -13 $RESULTS/fails.txt /tmp/expected-fails-in-caselist.txt > /tmp/new-passes.txt
if [ -s /tmp/new-passes.txt ]; then
echo "Unexpected passes, please update $DEQP_EXPECTED_FAILS (or add flaky tests to $DEQP_SKIPS):"
cat /tmp/new-passes.txt
exit 1
else
echo "No new passes"
fi