102 lines
4.0 KiB
ReStructuredText
102 lines
4.0 KiB
ReStructuredText
skqp
|
|
====
|
|
|
|
`skqp <https://skia.org/docs/dev/testing/skqp/>`_ stands for SKIA Quality
|
|
Program conformance tests. Basically, it has sets of rendering tests and unit
|
|
tests to ensure that `SKIA <https://skia.org/>`_ is meeting its design specifications on a specific
|
|
device.
|
|
|
|
The rendering tests have support for GL, GLES and Vulkan backends and test some
|
|
rendering scenarios.
|
|
And the unit tests check the GPU behavior without rendering images.
|
|
|
|
Tests
|
|
-----
|
|
|
|
Render tests design
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
It is worth noting that `rendertests.txt` can bring some detail about each test
|
|
expectation, so each test can have a max pixel error count, to tell skqp that it
|
|
is OK to have at most that number of errors for that test. See also:
|
|
https://github.com/google/skia/blob/c29454d1c9ebed4758a54a69798869fa2e7a36e0/tools/skqp/README_ALGORITHM.md
|
|
|
|
.. _test-location:
|
|
|
|
Location
|
|
^^^^^^^^
|
|
|
|
Each `rendertests.txt` and `unittest.txt` file must be located inside a specific
|
|
subdirectory inside skqp assets directory.
|
|
|
|
+--------------+--------------------------------------------+
|
|
| Test type | Location |
|
|
+==============+============================================+
|
|
| Render tests | `${SKQP_ASSETS_DIR}/skqp/rendertests.txt` |
|
|
+--------------+--------------------------------------------+
|
|
| Unit tests | `${SKQP_ASSETS_DIR}/skqp/unittests.txt` |
|
|
+--------------+--------------------------------------------+
|
|
|
|
The `skqp-runner.sh` script will make the necessary modifications to separate
|
|
`rendertests.txt` for each backend-driver combination. As long as the test files are located in the expected place:
|
|
|
|
+--------------+----------------------------------------------------------------------------------------------+
|
|
| Test type | Location |
|
|
+==============+==============================================================================================+
|
|
| Render tests | `${MESA_REPOSITORY_DIR}/src/${GPU_DRIVER}/ci/${GPU_VERSION}-${SKQP_BACKEND}_rendertests.txt` |
|
|
+--------------+----------------------------------------------------------------------------------------------+
|
|
| Unit tests | `${MESA_REPOSITORY_DIR}/src/${GPU_DRIVER}/ci/${GPU_VERSION}_unittests.txt` |
|
|
+--------------+----------------------------------------------------------------------------------------------+
|
|
|
|
Where `SKQP_BACKEND` can be:
|
|
|
|
- gl: for GL backend
|
|
- gles: for GLES backend
|
|
- vk: for Vulkan backend
|
|
|
|
Example file
|
|
""""""""""""
|
|
|
|
.. code-block:: console
|
|
|
|
src/freedreno/ci/freedreno-a630-skqp-gl_rendertests.txt
|
|
|
|
- GPU_DRIVER: `freedreno`
|
|
- GPU_VERSION: `freedreno-a630`
|
|
- SKQP_BACKEND: `gl`
|
|
|
|
.. _rendertests-design:
|
|
|
|
skqp reports
|
|
------------
|
|
|
|
skqp generates reports after finishing its execution, they are located at the job
|
|
artifacts results directory and are divided in subdirectories by rendering tests
|
|
backends and unit
|
|
tests. The job log has links to every generated report in order to facilitate
|
|
the skqp debugging.
|
|
|
|
Maintaining skqp on Mesa CI
|
|
---------------------------
|
|
|
|
skqp is built alongside with another binary, namely `list_gpu_unit_tests`, it is
|
|
located in the same folder where `skqp` binary is.
|
|
|
|
This binary will generate the expected `unittests.txt` for the target GPU, so
|
|
ideally it should be executed on every skqp update and when a new device
|
|
receives skqp CI jobs.
|
|
|
|
1. Generate target unit tests for the current GPU with :code:`./list_gpu_unit_tests > unittests.txt`
|
|
|
|
2. Run skqp job
|
|
|
|
3. If there is a failing or crashing unit test, remove it from the corresponding `unittests.txt`
|
|
|
|
4. If there is a crashing render test, remove it from the corresponding `rendertests.txt`
|
|
|
|
5. If there is a failing render test, visually inspect the result from the HTML report
|
|
- If the render result is OK, update the max error count for that test
|
|
- Otherwise, or put `-1` in the same threshold, as seen in :ref:`rendertests-design`
|
|
|
|
6. Remember to put the new tests files to the locations cited in :ref:`test-location`
|