Commit Graph

157067 Commits

Author SHA1 Message Date
Timur Kristóf 28db493cdc radv: Copy BO list to ACE internal CS.
This is necessary to make sure the ACE internal cmdbuf
can access the same memory as the GFX cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 5a1cbafd9d radv: Submit internal compute cmdbuf.
Use scheduled dependencies to create two submissions:
first we submit ACE then GFX.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf eeb8366bbd radv: Workaround MEC taskmesh dispatch hang when count buffer has zero.
The DISPATCH_TASKMESH_INDIRECT_MULTI_ACE packet has a firmware bug,
it hangs the GPU when the draw count is zero.

This commit adds a workaround sequence using COND_EXEC packets
which make sure that this indirect packet is never executed when
the draw count is zero.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 7b8f087f6b radv: Support task shaders in secondary cmd buffers.
Special consideration is needed to keep ACE and GFX in sync.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 4c6f83006d radv: Synchronization for task shaders.
Add a separate flush_bits field for tracking cache
flushes in the ACE internal cmdbuf.
In barriers and image transitions we add these flush bits to ACE.

Create a semaphore in the upload BO which makes it possible
for ACE to wait for GFX for the purpose of synchronization.
This is necessary when a barrier needs to block task shaders.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 2479b62869 radv: Implement mesh shading draw calls with task shaders.
This implements NV_mesh_shader draw calls with task shaders.

- On the GFX side:
  DISPATCH_TASKMESH_GFX for all draws
- On the ACE side:
  DISPATCH_TASKMESH_DIRECT_ACE for direct draws
  DISPATCH_TASKMESH_INDIRECT_MULTI_ACE for indirect draws

Additionally, the NV_mesh_shader indirect BO layout is
incompatible with AMD HW, so we add a function that copies
that into a suitable layout.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 99179f8490 radv: Introduce radv_before_taskmesh_draw.
This includes additional code that takes care of
handling the internal ACE cmdbuf.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 79287f5ae6 radv: Flush descriptors and push constants for task shaders.
Task shaders are executed on the internal compute cmdbuf, so they
need special consideration.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf ef07c3272a radv: Create internal cmdbuf when a graphics pipeline needs compute.
This is mainly going to be used by task shaders, because
the HW implementation mismatches the API:

- In the API, task shaders are considered graphics shaders which
  are part of a graphics pipeline and the draws are submitted to
  a graphics queue.
- The HW requires the driver to dispatch task shaders on
  an async compute queue.

When a pipeline is bound that has a task shader, create a
driver-internal ACE (async compute engine) cmdbuf which
we are going to submit to an ACE queue.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 44a71594c8 radv: Add dispatch_initiator_task field to radv_device.
This is going to be used with task shader dispatches.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 822e370390 radv: Allow reusing pipeline compute state emit functions.
We are going to reuse them outside of radv_pipeline.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf bc4012d7ed radv: Minor cleanup of radv_queue_submit_normal.
- Move the uses_perf_counters ternary expression out of
  the loop into a variable called cs_offset.

- Constify cmd_buffer_count.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf d718bea92d radv: Move inline push constants to a new function.
This cleans up radv_flush_constants and also
the new function will be reused later.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf c4d60df380 radv: Refactor view index emit to use a per-stage function.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 99b7521504 radv: Refactor radv_emit_inline_push_consts to work with radeon_cmdbuf.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 19fcf5a4cb radv: Refactor radv_emit_descriptor_pointers to work with radeon_cmdbuf.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf 21e6d596ed radv: Refactor radv_emit_userdata_address to work with radeon_cmdbuf.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf aadb3d69d1 radv: Refactor some CP DMA functions to work with radeon_cmdbuf.
Allow emitting these packets without a radv_cmd_buffer object.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf e10cbb5d98 radv: Refactor predication for compute queues.
Initialize the inverted predication VA only when it is used
for the first time.

This is needed to get conditional rendering work correctly with
task shaders because the internal compute cmdbuf may not exist
yet when conditional rendering starts.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Juan A. Suarez Romero 7131a05f92 v3d/ci: update expected list
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17660>
2022-07-20 18:37:26 +00:00
Mike Blumenkrantz ea92421298 zink: scale number of shader cache-get threads
this should improve startup times for short tests

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17659>
2022-07-20 18:19:10 +00:00
Mike Blumenkrantz 153983328d zink: handle null samplerview/image rebinds more gracefully
fixes a crash in nine

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17656>
2022-07-20 17:48:03 +00:00
Mike Blumenkrantz 3c66617213 zink: move layout-setting for dynamic render attachments after no-op case
this avoids unnecessarily terminating a renderpass to insert unnecessary barriers

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz edab34f68d zink: allow no-op renderpass updates in zink_batch_rp()
in some cases it becomes desirable to "maybe" stop and start the current
renderpass, such as when updates MAY result in layout changes for attachments

for such cases, avoid splitting the renderpass unless it actually needs to
be split

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 4aec761596 zink: improve granularity of renderpass switching
this should ensure that (future) renderpass recalcs will never split
a renderpass unnecessarily

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz f78919d36d zink: add a bits union for zink_render_pass_state comparisons
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 13951385b3 zink: reuse rt layout functions to refactor zink_render_pass_attachment_get_barrier_info()
maybe slightly easier to read?

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz c602e877a3 zink: break out color rt layout calc
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 903060787e zink: break out zs renderpass layout calc
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 1ff18f0879 zink: break out renderpass attachment load op calc
no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 1bc9db6879 zink: split out rp_changed to be more granular for dynamic render
sometimes a state change MAY require a renderpass change, but this change
will not require splitting the current renderpass

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 9b087dcccd zink: add an assert to verify that deferred clears aren't added in renderpass
this would be very, very broken

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz d9a320b62c zink: remove redundant renderpass update check for clears
this should be handled already in the clear code

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 06b04a5c71 zink: unflag ctx->rp_changed after dynamic render update
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz b31a8dfc85 zink: do dynamic render implicit clear updates on rp_changed
this should be flagged correctly now

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:19 +00:00
Mike Blumenkrantz 5ae1c76baa zink: flag renderpass change if renderpass clears change
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:18 +00:00
Mike Blumenkrantz ca3d696f80 zink: don't flag renderpass change on depth write change as often
this is only "sometimes" required and only if dynamic render isn't being
used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:18 +00:00
Mike Blumenkrantz d079e3c9b6 zink: don't try to start a renderpass before CmdClearAttachments
it should be impossible to reach this point if a renderpass is not active

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17640>
2022-07-20 17:30:18 +00:00
Mike Blumenkrantz 644f059a8d llvmpipe: ignore multisample state for alpha_to_one
this is more accurate and fixes usage with lavapipe

cc: mesa-stable

Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17627>
2022-07-20 15:41:14 +00:00
Mike Blumenkrantz a884d1eb0e vulkan/wsi: fix multiple acquires for sw without mit-shm
in this case, lying about having multiple images and then returning the
same image every time doesn't work, so use the busy flag
and return an available image when possible

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17590>
2022-07-20 15:10:15 +00:00
Georg Lehmann 333f056edf radv, aco: Don't lower 16bit isign.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Georg Lehmann 4be6164ac9 radv: Remove ineg from lower_bit_size_callback callback.
It's always lowered to isub.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Georg Lehmann b96126ee95 radv,aco: Don't lower and vectorize 16bit iabs.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Rhys Perry 37dda55f46 aco: remove unnecessary exp_cnt increments
update_counters() already does this.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17257>
2022-07-20 14:04:15 +00:00
Vinson Lee 7384aab3ee aco: Initialize spill_ctx members in constructor.
Fix defects reported by Coverity Scan.

Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member sgpr_spill_slots is not
initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member vgpr_spill_slots is not
initialized in this constructor nor in any functions that it calls.

Fixes: 7d34044908 ("aco: refactor VGPR spill/reload lowering")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17583>
2022-07-20 13:39:12 +00:00
Mike Blumenkrantz 687a5d8690 zink: use gfx_barrier for synchronization when binding buffer descriptors
this will preserve existing shader stages in the access mask and avoid desync
later on

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17639>
2022-07-20 09:21:30 -04:00
Mike Blumenkrantz 268011e8c6 zink: fix viewport count tracking
the number of viewports in use is based on the outputs of the last vertex
stage, not the viewports passed by the state tracker

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17639>
2022-07-20 09:21:30 -04:00
Mike Blumenkrantz eaf11d3cd9 zink: call bind_last_vertex_stage() when binding vs
ensure all the necessary updates are flagged

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17639>
2022-07-20 09:21:30 -04:00
Mike Blumenkrantz 011c02ce3c zink: improve oom error message
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17639>
2022-07-20 09:21:30 -04:00
Mike Blumenkrantz 2aab0a9c16 zink: fix crashing on bo allocation failure
list init occurs after mem allocation, so check this first

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17639>
2022-07-20 09:21:30 -04:00