Commit Graph

278 Commits

Author SHA1 Message Date
Emma Anholt f6c5b1d6c6 nir: Split usub_sat lowering flag from uadd_sat.
Intel vec4 would like to do uadd_sat, but use lowering for usub_sat.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17637>
2022-07-22 17:54:28 +00:00
Mike Blumenkrantz 56e5eaeba1 zink: fix xfb emit check in compiler
nir->info.has_transform_feedback_varyings is set for all stages in the
pipeline when xfb is present, so it can't be used for this

harmless, but this is more correct

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17404>
2022-07-15 17:32:27 +00:00
Mike Blumenkrantz bd2eaaeb7d zink: add a compiler pass to split xfb block outputs
this splits all the members of a struct into separate variables to
improve xfb inlining and reduce the number of locations consumed by
xfb outputs, reducing the chances of running out of shader outputs

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17404>
2022-07-15 17:32:27 +00:00
Mike Blumenkrantz 5245290565 zink: fix xfb array inlining
get_slot_components() returns the total number of output components
for arrays for initial evaluation phase, but during the packed->inlined
conversion the arrayed size must be normalized to the slot's component count
in order to effectively catch and inline the array

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17404>
2022-07-15 17:32:27 +00:00
Mike Blumenkrantz 3a47576687 zink: add a compiler pass to match up tex op dest types
this handles bitsize conversions and depth component splatting to
enable simplifying some of the related ntv code

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17427>
2022-07-15 14:08:56 +00:00
Georg Lehmann e09e04a2c0 zink: Lower uadd_sat/usub_sat.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17468>
2022-07-13 07:34:09 +00:00
SoroushIMG eb6faf329c zink: Fix BO size when it's not aligned to 16 bytes
glsl_get_explicit_size can return non-16 byte aligned sizes.
Therefore, make sure the sure the size isrounded up so that OOB does not happen.

Fixes: ea8a0654f5 ("zink: further improve bo sizing")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17460>
2022-07-12 12:58:02 +00:00
Mike Blumenkrantz 6759320c65 zink: use a std430 interface for bo types
no functional changes but is technically more correct

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 17:03:53 -04:00
Mike Blumenkrantz ab9e423c4f zink: add an optimizer pass to enforce bo bounds
this eliminates (some) out-of-bounds bo access and will ensure that
bo sizing is always accurate by breaking all the cases where it isn't

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz 48c19bb508 zink: pass zink_shader to optimize_nir during final optimize pass
no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz 9dd41991d0 zink: don't run nir_lower_uniforms_to_ubo repeatedly
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz ac2141a5b2 zink: handle bitsizes in get_bo_vars() analysis
this allows it to be run repeatedly

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz 58780b3aa6 zink: move/refactor bo_vars struct creation in compiler
no functional changes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz 7b6020d487 zink: force unsized array in bo creation using the last block member type
if the last member of a block is an unsized array then the bo needs to have
an unsized array

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz ea8a0654f5 zink: further improve bo sizing
using the attribute slot size isn't sufficient in this case, as the layout
rules may have additional effects upon sizing

instead, just use the explicit size

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz e241bbe1ce zink: size uniform_0 ubo accurately
this one is always identifiable from lower_uniforms_to_ubo, so the
info should always be reliable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17239>
2022-07-05 16:01:01 -04:00
Mike Blumenkrantz a530b90cd3 zink: remove swizzle from fbfetch lowering
I had this in at one point to fix something, but now it somehow just
breaks fbfetch instead of fixing anything

cc: mesa-stable

fixes:
dEQP-GLES31.functional.blend_equation_advanced*

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17254>
2022-06-27 20:38:53 +00:00
Mike Blumenkrantz e38b2adb88 zink: use the bigger of the variable type and interface type for bo sizing
this avoids the scenario where the full bo size isn't accounted for because
no variable for the block has been created

cc: mesa-stable

affects:
KHR-GL33.shaders.uniform_block.random.all_per_block_buffers.3

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17217>
2022-06-24 01:21:45 +00:00
Mike Blumenkrantz 1783a261f6 zink: only enable vtn debugging if validation is active
this is otherwise incredibly cumbersome and eats up minutes in ci

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17069>
2022-06-16 02:18:53 +00:00
Mike Blumenkrantz 03d7f31054 zink: don't print VK_EXT_shader_atomic_float warning for nir_intrinsic_image_deref_atomic_exchange
this is covered by base spec and doesn't require the extension

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17010>
2022-06-14 19:21:15 +00:00
Mike Blumenkrantz 44cf6f7564 zink: fix generated tcs deletion
deleting the generated shader on the first loop iteration like this was
broken if the shader was used in multiple programs, so delete at the end

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17010>
2022-06-14 19:21:15 +00:00
Mike Blumenkrantz cf34d3fe05 zink: unset generated tcs pointer from tes on deletion
otherwise this will free the generated tcs multiple times if the tes
is used by multiple programs

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17010>
2022-06-14 19:21:15 +00:00
Mike Blumenkrantz 11e55bce49 zink: cap driver inlining using ssa allocation limit
usually inlining is optimal for cpu drivers since the majority of
time is spent in the shaders, and any amount of reduction to shader code
will be optimal

if, however, the shaders are still really big after inlining, this improvement
will be negated by the insane amount of time spent doing stupid llvm optimizer
passes, so check post-inline size to see whether it exceeds a size threshold

lavapipe release build - 1700% improvement
* spec@arb_tessellation_shader@execution@variable-indexing@tcs-output-array-vec4-index-rd-after-barrier

before: 142.15s user 0.42s system 99% cpu 2:23.14 total

after: 8.60s user 0.07s system 99% cpu 8.677 total

fixes #6647

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16977>
2022-06-13 16:04:01 +00:00
Mike Blumenkrantz 98d7a9a9e4 zink: run copy_prop_vars during optimization
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16973>
2022-06-10 21:10:51 +00:00
Mike Blumenkrantz 5a95c6b328 zink: rewrite atomic ssbo intrinsics as atomic derefs
todo--

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16973>
2022-06-10 21:10:51 +00:00
Mike Blumenkrantz bbe5136658 zink: fix 32bit bo rewriting
this was correct for 64bit loads and manually converted 32bit loads (e.g., bindless),
but it was broken for the case where 64bit was not supported, as the offset wasn't
being correctly adjusted

break out the offset division to hopefully make this a little clearer

Fixes: 150d6ee97e ("zink: move all 64-32bit shader load rewriting to nir pass")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz ea8fc23119 zink: avoid adding more 64bit alus during bo rewriting
nir_unpack_64_2x32 isn't supported, but split is, so use that

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz cdaa601de3 zink: scalarize when rewriting explicit 64bit io
all of ntv requires scalarized io since the offsets are now array indices
instead of byte offsets, so enforce scalarization here to avoid breaking
the universe

Fixes: 150d6ee97e ("zink: move all 64-32bit shader load rewriting to nir pass")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz 5b2f850425 zink: rewrite 64bit shader i/o as 32bit
zink can't use lower_io, so this all has to be done manually and in
excruciating depth and detail

fixes (tu):
KHR-Single-GL46.arrays_of_arrays_gl.InteractionFunctionCalls2
KHR-GL46.gpu_shader_fp64.fp64.named_uniform_blocks
KHR-GL46.gpu_shader_fp64.fp64.varyings
KHR-GL46.vertex_attrib_binding.advanced-bindingUpdate
KHR-Single-GL46.enhanced_layouts.varying_array_components
KHR-Single-GL46.enhanced_layouts.varying_array_locations
KHR-Single-GL46.enhanced_layouts.varying_components
KHR-Single-GL46.enhanced_layouts.varying_locations
KHR-Single-GL46.enhanced_layouts.xfb_explicit_location
dEQP-GLES3.functional.transform_feedback.basic_types.interleaved.lines.highp_mat3x4
dEQP-GLES3.functional.transform_feedback.basic_types.interleaved.lines.mediump_vec3
dEQP-GLES3.functional.transform_feedback.basic_types.interleaved.points.mediump_vec4
dEQP-GLES3.functional.transform_feedback.basic_types.separate.points.highp_vec3

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz 26338c4abb zink: if a variable is flagged as being xfb, treat it as manual xfb output
this allows passes to flag variables as "needs manual action" for further
handling later on

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz fd516b0a9f zink: flag all shader i/o variables as !is_xfb
reclaiming this flag for internal use

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz b59d67431d zink: run more int64 passes during optimization loop if int64 isn't available
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz 7c75ce6204 zink: don't lower pack/unpack intrinsics
this just ends up propagating the bitsize that we don't want, making it
harder to remove them later

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz 3d8131341b zink: always scalarize pack/unpack alu ops
these are tricky

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz b1f684c80b zink: run nir_lower_alu_to_scalar in optimizer loop for 64bit lowering
ensure that pack/unpack ops aren't passing swizzles since those are hard
to implement

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz 02a3a5ff31 zink: flag optimization pass after inlining uniforms
this probably generated some pretty gnarly code

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16669>
2022-06-10 02:35:12 +00:00
Mike Blumenkrantz b818c0862b zink: simplify some compiler bo code
this is a bit easier to read

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16927>
2022-06-09 12:17:45 +00:00
Mike Blumenkrantz e1f46524e3 zink: lower subgroup width to 1 for unsupported subgroup vote stages
this should handle unsupported gfx subgroup vote

cc: mesa-stable

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16757>
2022-05-31 14:45:02 +00:00
Mike Blumenkrantz fe2ba184d8 zink: use descriptor indices in compiler
should be no functional changes, just adding indirection

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16645>
2022-05-25 04:01:56 +00:00
Mike Blumenkrantz e303898258 zink: manually validate shaders in debug builds
VVL is great, but there's actually cases where it doesn't catch critical
spirv errors, so add in our own validation pass to make sure things are
okay

this is especially useful for running on nvidia, as their compiler will
either crash on or silently drop illegal instructions

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16462>
2022-05-17 02:55:20 +00:00
Jason Ekstrand 3c07c3e16d shader_info: Make images_used a bitset
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15988>
2022-05-10 11:23:11 -05:00
Mike Blumenkrantz 269083d94b zink: delete nir_lower_dynamic_bo_access
no longer used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15906>
2022-05-10 05:55:55 +00:00
Mike Blumenkrantz a7327c7cac zink: implement indirect buffer indexing
this compacts all buffers in the shader into an array that can be
used in a single descriptor, thus handling the case of indirect indexing
while also turning constant indexing into indirect (with const offsets)
since there's no sane way to distinguish

a "proper" implementation of this would be to skip gl_nir_lower_buffers
and nir_lower_explicit_io altogether and retain the derefs, but that would
require a ton of legwork of other nir passes which only operate on the
explicit io intrinsics

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15906>
2022-05-10 05:55:55 +00:00
Mike Blumenkrantz c0c69b1be1 zink: semi-handle 1D sparse texture rewrites for drivers that don't support them
nvidia can't do this, but also nothing uses it, so I've gone ahead and
done the bare minimum here to make cts pass

I think the work to do the shader rewrites should be easy, but without a test
case, I see no point in spending the time for it

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16100>
2022-05-10 05:44:56 +00:00
Emma Anholt 536c8ee96d nir/lower_tex: Make the adding a 0 LOD to nir_op_tex in the VS optional.
This controls the whole lowering of "make tex ops with implicit
derivatives on non-implicit-derivative stages be tex ops with an explicit
lod of 0 instead", but it's really hard to describe that in a git commit
summary.

All existing callers get it added except:
- nir_to_tgsi which didn't want it.
- nouveau, which didn't want it (fixes regressions in shadowcube and
  shadow2darray with NIR, since the shading languages don't expose txl of
  those sampler types and thus it's not supported in HW)
- optional lowering passes in mesa/st (lower_rect, YUV lowering, etc)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16156>
2022-04-28 21:26:08 +00:00
Mike Blumenkrantz 306b5f3724 zink: use vulkan memory model shader semantics for tcs barriers
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15959>
2022-04-22 03:35:44 +00:00
Mike Blumenkrantz 12cf9a1544 zink: remove tcs patch slot map
this is illegal, and we'll just have to eat some piglit fails
until indirects are handled

Fixes: f7ade1f188 ("zink: simplify shader i/o assignment")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15976>
2022-04-18 21:32:40 +00:00
Mike Blumenkrantz d275d6c32f zink: clamp max shader images to 32
NO MATTER WHAT.

Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16013>
2022-04-18 17:54:20 +00:00
Mike Blumenkrantz fcd6b2a47a zink: avoid creating ssbo variable types with multiple runtime arrays
this is illegal

affects:
KHR-GL46.shader_storage_buffer_object.advanced-unsizedArrayLength-cs-packed-matC

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15894>
2022-04-13 10:50:22 +00:00
Mike Blumenkrantz c7ae22e4b8 zink: prune unused st-injected pointsize exports
only the last vertex stage needs to keep these, so prune any that aren't
being weirdly passed through

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15853>
2022-04-12 04:49:17 +00:00