Commit Graph

347 Commits

Author SHA1 Message Date
Connor Abbott 49827da6fa tu: Implement VK_EXT_pipeline_creation_feedback
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16562>
2022-05-18 13:14:55 +00:00
Chia-I Wu 2410993ef6 turnip: fix off-by-one in border color bitset
BITSET_FFS reserves 0 for no bit set.  BITSET_CLEAR just below cleared
the wrong bit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16567>
2022-05-17 23:29:15 +00:00
Chia-I Wu 0b7751babf turnip: fix sampledImageIntegerSampleCounts
It seems fine to advertise msaa in sampledImageIntegerSampleCounts.

dEQP-VK.rasterization.rasterization_order_attachment_access.format_integer.*
goes from NotSupported to Pass for more test cases.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16487>
2022-05-16 19:26:46 +00:00
Connor Abbott 05329d7f9a tu: Implement pipeline caching with shared Vulkan cache
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16147>
2022-05-13 17:07:05 +00:00
Connor Abbott 43981f0f58 tu: Include turnip debug flags in pipeline cache UUID
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16147>
2022-05-13 17:07:05 +00:00
Connor Abbott d023ae4686 tu: Rewrite cache UUID based on radv
Switch to using sha1 so that we can add as many other flags as we need
to easily.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16147>
2022-05-13 17:07:05 +00:00
Emma Anholt b282d504a4 turnip: Add a TU_DEBUG=perf debug option.
For doing performance investigation, I often find it useful to have a "are
we tripping over any of our performance TODOs?" flag, so add it and use it
in a few of the TODOs.

This also greatly cleans up the deqp-vk logs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16316>
2022-05-12 01:00:25 +00:00
Danylo Piliaiev db69218cbe tu: Implement VK_EXT_image_view_min_lod
Relevant tests:
 dEQP-VK.texture.mipmap.*.image_view_min_lod.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16292>
2022-05-09 07:53:41 +00:00
Danylo Piliaiev 725ae34458 turnip: Add debug option to print gmem load/store skip stats
TU_DEBUG=log_skip_gmem_ops would print stats about skipped
gmem/load every second.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15974>
2022-04-29 09:29:55 +00:00
Emma Anholt ce15bf19fb turnip: Add TU_DEBUG=layout for dumping image layouts.
This was useful for comparing image allocations between gfxbench
gl_5_normal and vk_5_normal to see if rendering was generally equivalent
(formats, MSAA, UBWC choices, and notably gfxbench vk was choosing DXT5
instead of ASTC on non-android builds!)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15982>
2022-04-19 18:45:29 +00:00
Emma Anholt 835704e669 turnip: Move autotune buffers to suballoc.
Now the ANGLE trex_200 trace replay does a single BO allocation at startup
for autotune results instead of one per frame (~350 for the whole replay).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15038>
2022-04-12 01:01:56 +00:00
Emma Anholt dc3203b087 turnip: Sub-allocate pipelines out of a device-global BO pool.
Allocating a BO for each pipeline meant that for apps with many pipelines
(such as Asphalt9 under ANGLE), we would end up spending too much time in
the kernel tracking the BO references.

Looking at CS:Source on zink, before we had 85 BOs for the pipelines for a
total of 1036 kb, and now we have 7 BOs for a total of 896 kb.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15038>
2022-04-12 01:01:56 +00:00
Danylo Piliaiev dde1623ed2 turnip: Implement VK_EXT_primitives_generated_query
Similar to pipeline statistics but done for a single counter.

We use REG_A6XX_RBBM_PRIMCTR_7 to get generated primitives
and not PRIMCTR_8 because PRIMCTR_7 counts pre-clipped prims
while PRIMCTR_8 counts them after clipping.

OpenGL spec for GL_PRIMITIVES_GENERATED says:
 "Subsequent rendering will increment the counter once for every
  vertex that is emitted from the geometry shader, or from the
  vertex shader if no geometry shader is present."

Passes tests:
 dEQP-VK.transform_feedback.primitives_generated_query.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15746>
2022-04-07 08:01:59 +00:00
Danylo Piliaiev 6c18602164 turnip: Add "unaligned_store" debug option to better test gmem stores
Unaligned store is incredibly rare in CTS, we have to force it to
actually test it.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15532>
2022-04-06 08:44:28 +00:00
Jason Ekstrand bdf52654ac turnip: Enable VK_EXT_debug_utils
It's implemented in common code as long as you use vk_command_buffer.

Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15560>
2022-04-06 01:18:23 +00:00
Connor Abbott b91b90c256 tu: Expose VK_KHR_maintenance4
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15488>
2022-04-05 17:46:35 +00:00
Connor Abbott 7455a7a44c tu: Fill out maxBufferSize
It seems this is really a workaround for silly issues in
GetBufferMemoryRequirements when you ask for a really large buffer. Just
expose the maximum possible size ATM.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15488>
2022-04-05 17:46:35 +00:00
Connor Abbott d1762b7df0 tu: Implement GetDevice*MemoryRequirements()
Based mostly on anv, which is a bit more optimized than radv - we at
allocate the image on the stack.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15488>
2022-04-05 17:46:35 +00:00
Emma Anholt 51b04a7dfb turnip: Add support for VK_KHR_format_feature_flags2.
This reports all of our storage formats as supporting read/write without
format, since we don't have any in-shader format conversions.  Similarly,
shadow comparisons were already supported on all the depth formats.

This extension is required for VK 1.3.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15293>
2022-04-02 19:55:40 +00:00
Boris Brezillon 799a9db24c turnip: Stop using VK_OUTARRAY_MAKE()
We're trying to replace VK_OUTARRAY_MAKE() by VK_OUTARRAY_MAKE_TYPED()
so people don't get tempted to use it and make things incompatible with
MSVC (which doesn't support typeof()).

Suggested-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15522>
2022-03-25 11:00:02 +00:00
Connor Abbott fc381fa1e3 tu: Actually expose VK_EXT_texel_buffer_alignment
Oops...

Fixes: 3d04c435 ("tu: Trivially implement VK_EXT_texel_buffer_alignment")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15451>
2022-03-18 18:30:20 +00:00
Connor Abbott 3d04c43576 tu: Trivially implement VK_EXT_texel_buffer_alignment
The previous alignment of 64 bytes, which we got from the blob,
indicates that single-texel alignment isn't supported. So just do a
trivial no-op implementation that returns the same alignment as before.
This matches what newer blobs that expose this extension do.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15427>
2022-03-17 20:45:19 +00:00
Connor Abbott 072fdcabcd tu: Enable UniformBufferUpdateAfterBind
UBOs are now read at run-time via the preamble so this can be enabled.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13148>
2022-03-17 12:15:45 +00:00
Connor Abbott 9932ca8a3f ir3, turnip: Use ldc.k to push UBOs
This reuses the same UBO analysis to do the pushing in the shader
preamble via the ldc.k instruction instead of in the driver via
CP_LOAD_STATE6. The const_data UBO is exempted as it uses a different
codepath that isn't as critical.

Don't do this on gallium because there are some regressions. Aztec Ruins
in particular regresses a bit, and nothing I've benchmarked benefits.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13148>
2022-03-17 12:15:45 +00:00
Connor Abbott 221a912b8c ir3: Refactor ir3_compiler_create() to take an options struct
This will let us add more options without creating too much churn.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13148>
2022-03-17 12:15:45 +00:00
Emma Anholt 2f25d16653 turnip: Use the DRM or KGSL GPU reset status ioctls to report device loss.
ANGLE-on-venus-on-turnip and zink-on-turnip want real data here for EGL's
reset tests.

This required moving the remaining GPU-reset-causing tests from flakes or
xfails to skips.  Otherwise, the rest of the caselist associated with them
ends up being marked as fails as well.  The alternative would be to put
these tests in their own test groups with tests_per_group = 1, but that
didn't seem worth the effort.  Or, we could finally do something with
https://gitlab.freedesktop.org/anholt/deqp-runner/-/issues/14.

Fixes: #5955
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14839>
2022-03-16 19:28:04 +00:00
Emma Anholt eb9b092001 turnip: Enable VK_EXT_display_control using the common code.
It's all implemented now, so we can turn it back on.  Passes 15/16 tests
when X11 isn't running, and 1/16 when it is, with no failures in either
mode.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15351>
2022-03-15 20:08:58 +00:00
Connor Abbott cdee38a57b tu: Expose subgroup arithmetic
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14107>
2022-03-10 17:15:29 +00:00
Danylo Piliaiev c4703cd846 tu: Implement VK_EXT_depth_clip_control
Since negativeOneToOne is a static property of the pipeline and
viewport state could be dynamic, we have to defer viewport state
emission until negativeOneToOne value is known.

See: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6070

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14363>
2022-03-10 11:08:50 +02:00
Rob Clark 711f0d1df4 turnip: Don't call getenv() directly
I noticed it was using getenv directly when I tried to use 'setprop
mesa.tu.debug ..' on android.  Use os_get_option() instead so we get
sysprop fallback on android.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15289>
2022-03-09 00:22:36 +00:00
Danylo Piliaiev e2fc99b188 turnip: Add "rast_order" debug option to force rast order access
Enables rasterization order attachment access for all pipelines,
see VK_ARM_rasterization_order_attachment_access for details.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15262>
2022-03-07 17:07:18 +00:00
Danylo Piliaiev 549e861dc1 turnip: Implement VK_EXT_physical_device_drm
Copied from ANV and V3DV.

v1. Fix a build error for clang "unannotated fall-through between switch labels"
( Hyunjun Ko <zzoon.ko@igalia.com> )

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6011

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14971>
2022-03-01 07:10:40 +00:00
Danylo Piliaiev ebc23ac963 turnip: Implement VK_ARM_rasterization_order_attachment_access
Trivially implemented by using A6XX_GRAS_SC_CNTL_SINGLE_PRIM_MODE.

This extension is useful for emulators e.g. AetherSX2 PS2 emulator and
could drastically improve performance when blending is emulated.

Relevant tests:
dEQP-VK.rasterization.rasterization_order_attachment_access.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15106>
2022-02-23 11:31:59 +00:00
Emma Anholt 59bc17d57a turnip: Request no implicit sync when we have no implicit-sync WSI BOs.
I chose to implement this as a global flag in the device, because
otherwise we would end up with extra draw overhead trying to avoid it in
the implicit-sync WSI case, and you're probably going to end up needing
implicit sync anyway because you used one of the BOs in any of the
submitted cmdbufs.  To do better than this, we would probably want a
skip-implicit-sync flag on the BOs in the BO list, rather than global on
the submit.

Reports about venus on turnip say that this flag reduces worst-case
QueueSubmit time in a game workload from ~10ms to ~4ms.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14838>
2022-02-22 17:36:05 +00:00
Danylo Piliaiev a814a4f9db turnip: Add a refcount mechanism to BOs
Until now we have lived without a refcount mechanism in the driver
because in Vulkan the user is responsible for handling the life
span of memory allocations for all Vulkan objects, however,
imported BOs are tricky because the kernel doesn't refcount
so user-space needs to make sure that:

1. When importing a BO into the same device used to create it
   (self-importing) it does not double free the same BO.
2. Frees imported BOs that were not allocated through the same
   device.

Our initial implementation always freed BOs when requested,
so we handled 2) correctly but not 1) on drm and we would
double-free self-imported BOs because kernel doesn't return
a unique gem_handle on each import.

Beside this the submit ioctl checks for duplicates in the
BO list and returns an error if there is one.

This fixes the problem for good by adding refcounts to BOs
so that self-imported BOs have a refcnt > 1 and are only freed
when all references are freed.

KGSL on the other hand does not have the same problems,
at least not with ION buffers which are used for exportable
BOs on pre 5.10 android kernels.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5936
Fixes CTS tests: dEQP-VK.drm_format_modifiers.export_import.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15031>
2022-02-19 15:16:55 +00:00
Yiwei Zhang 2a87a741ae turnip: advertise VK_EXT_queue_family_foreign
Both Venus and Android AHB requires this extension.

Turnip ignores VK_SHARING_MODE_EXCLUSIVE so this is a no-op.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Acked-by: Rob Clark <robdclark@chromium.org>
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14836>
2022-02-14 21:27:35 +00:00
Danylo Piliaiev 44bdac9849 tu: Implement VK_AMD_buffer_marker to support Graphics Flight Recorder
Graphics Flight Recorder is:
 "The Graphics Flight Recorder (GFR) is a Vulkan layer to help
  trackdown and identify the cause of GPU hangs and crashes.
  It works by instrumenting command buffers with completion tags."

This is a nice little tool which could help quickly identify the call
which hanged. Or if command buffer is executed for too long.

The tiling nature of our GPU shouldn't be a big issue aside from
lower performance.

For non-segfault case, if:
- Hang happens at the same place in cmdbuf and draw/dispatch is not
  finished at that point - it is likely that there is an infinite
  loop in some of the shaders in this draw.
- Hang happens always in different place - likely there is nothing
  wrong and command buffer just takes too long to execute and you
  should try increasing hangcheck_period_ms. If it doesn't help
  it is likely a synchronization issue.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13553>
2022-02-07 12:53:34 +02:00
Danylo Piliaiev fded7a95c5 turnip: Expose VK_KHR_shader_non_semantic_info
This is entirely implemented in the SPIR-V frontend.

Relevant CTS tests:
dEQP-VK.spirv_assembly.instruction.compute.non_semantic_info.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14829>
2022-02-04 09:24:06 +00:00
Danylo Piliaiev ff059605aa turnip: Implement VK_KHR_zero_initialize_workgroup_memory
Moved nir_lower_compute_system_values to lower
load_local_invocation_index which could be emitted by
nir_zero_initialize_shared_memory.

Relevant CTS tests:
dEQP-VK.compute.zero_initialize_workgroup_memory.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14829>
2022-02-04 09:24:06 +00:00
Danylo Piliaiev c6d1cac6e5 turnip: Expose VK_EXT_image_robustness
VK_EXT_image_robustness is a strict subset of VK_EXT_robustness2
so we could just expose it.

Relevant CTS tests: dEQP-VK.robustness.image_robustness.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14829>
2022-02-04 09:24:06 +00:00
Danylo Piliaiev 03f9deecb8 turnip: Use the shared helpers to expose 1.3 core extensions/limits
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14829>
2022-02-04 09:24:06 +00:00
Connor Abbott 0248644c89 ir3,tu: Enable subgroup shuffles and relative shuffles
We still don't use the fast path for relative shuffles, that's left for
future work.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14412>
2022-02-01 16:27:46 +00:00
Emma Anholt bf289e3123 turnip: Store the computed iova in the tu_image.
Less of a big deal than for buffers, but let's be consistent in how we
handle our bindings.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14816>
2022-02-01 15:30:12 +00:00
Emma Anholt f460fb3f91 turnip: Store the computed iova in the tu_buffer.
We recently had a bug of forgeting to add the buf->bo_offset.  Just make
the easiest field to get be the bo->iova + buf->bo_offset already.  Plus,
a little less work at emit time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14816>
2022-02-01 15:30:12 +00:00
Danylo Piliaiev 803055ccb4 tu: add debug option to force gmem
With autotuner we now want to be able to force gmem rendering,
it will respect existing constraints though.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12128>
2022-01-31 15:26:35 +00:00
Danylo Piliaiev dbae9fa7d8 tu: implement sysmem vs gmem autotuner
The implementation is separate from Freedreno due to multithreading
support.

In Vulkan application may fill command buffer from many threads
and expect no locking to occur. We do introduce the possibility of
locking on renderpass end, however assuming that application
doesn't have a huge amount of slightly different renderpasses,
there would be minimal to none contention.

Other assumptions are:
- Application does submit command buffers soon after their creation.

Breaking the above may lead to some decrease in performance or
autotuner turning itself off.

The heuristic is too simplistic at the moment, to find a proper
one - we should run a bunch of traces with sysmem and gmem, and
build better heuristic from gathered data.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12128>
2022-01-31 15:26:35 +00:00
Danylo Piliaiev f2c53c2a9b turnip/trace: refactor creation and usage of trace flush data
Fixes the case when last cmd buffer in submission doesn't have
tracepoints leading to flush data not being freed.

Added a few comments, renamed things, refactored allocations - now
the data flow should be a bit more clean.

Extracted submission data creation into tu_u_trace_submission_data_create
which would be later used in in tu_kgsl.

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14391>
2022-01-27 18:59:43 +00:00
Danylo Piliaiev cadcbed258 tu: expose VK_KHR_copy_commands2
Relevant CTS tests:
dEQP-VK.api.copy_and_blit.copy_commands2.*

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14623>
2022-01-20 10:43:31 +00:00
Charles Giessen 4e0604279d freedreno, tu: Update LoaderICDInterfaceVersion to v5
With the proper version checking in the common vulkan instance code
(commit 88b9b68) it is now possible to bring the reported interface
version up to v5.

Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14563>
2022-01-20 07:25:07 +00:00
Danylo Piliaiev e4c582ee71 tu: support VK_EXT_primitive_topology_list_restart
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Hyunjun Ko <zzoon@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14556>
2022-01-17 15:21:03 +00:00