Commit Graph

148617 Commits

Author SHA1 Message Date
Marek Olšák d4a1766a5a radeonsi: move the GS copy shader into shader variants
This will allow further optimizations for shader variants that change
GS outputs (affecting the copy shader), and this is mainly about sharing
optimizations with NGG instead of having a totally separate codepath for
legacy GS.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 1caa94f2a5 radeonsi: add into the disk cache key whether cached shaders contain LLVM IR
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 8ed9d38e73 radeonsi: move si_nir_scan_shader into si_shader_info.c
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 70919f30c1 radeonsi: change si_shader_output_values::vertex_stream to a bitmask
to match si_shader_info.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 2c4926dfc8 radeonsi: use nir->scratch_size instead of ac_count_scratch_private_memory
It's the same.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 3fb77ef2e0 radeonsi: do opt_large_constants & lower_indirect_derefs after uniform inlining
because loop unrolling caused by uniform inlining can eliminate large constants
and indirect derefs.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 198ad7e4dc radeonsi: move smoothing to the main shader part to remove 1 live VGPR
The samplemask VGPR that we had to pass to the epilog increased VGPR usage
by 1 for all shaders. Do it in the main function by using the mono key
structure, which causes on-demand compilation and stall, but we'll save
the VGPR.

57794 shaders in 35145 tests
Totals:
SGPRS: 2715856 -> 2716272 (0.02 %)
VGPRS: 1776168 -> 1718432 (-3.25 %)
Spilled SGPRs: 3704 -> 3630 (-2.00 %)
Spilled VGPRs: 1727 -> 1733 (0.35 %)
Private memory VGPRs: 256 -> 256 (0.00 %)
Scratch size: 2008 -> 2016 (0.40 %) dwords per thread
Code Size: 61429584 -> 61393288 (-0.06 %) bytes
Max Waves: 838645 -> 840484 (0.22 %)

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:31 +00:00
Marek Olšák 12b942bd16 radeonsi: pass sample_coverage VGPR index to the PS prolog instead of guessing
The code was correct, but little confusing. This is cleaner.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák 3283df1425 radeonsi: remove unused si_shader::prolog2
This became unused when the GS prolog was removed.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák 054d50904b radeonsi: don't bind the ESGS ring twice, handle the difference in the shader
The other shader is changed to modify the descriptor to get the required
values.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák 18fadc8e6a radeonsi: reorder slots for internal buffers, reuse a slot for GS_QUERY_BUF
GFX10_GS_QUERY_BUF is also renamed.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák 0246580106 radeonsi: simplify compacted_mrt_index in si_export_mrt_color
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák ca5f11ccd3 radeonsi: export mrtz before color exports
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák e762bc751e radeonsi: remove unnecessary code that was used to find the last export
We can do it trivially like this.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák fd14e0afb1 radeonsi: set done=1 for PS exports at the end of si_llvm_build_ps_epilog
so that we can reorder the exports

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák fd02299055 radeonsi: clean up si_export_mrt_color
Remove the intermediate args variable.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák 7f6643ffd0 radeonsi: make get_thread_id_in_tg non-static
for future work

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák 6386b95f0f radeonsi: modifiers can't disable DCC
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák d382ceea2b ac/llvm: remove the num_channels parameter from ac_build_buffer_store_dword
It was used when LLVM didn't support vec3 and we had to pass vec4
with num_channels=3. We no longer need to do that.

This also removes the vec3 splitting or conversion to vec4 in callers.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák e6aac44051 ac/llvm: add vindex into ac_build_buffer_store_dword
for future work

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Marek Olšák bb017d5d16 amd/registers: work around an assertion in parse_kernel_headers.py
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14266>
2022-01-05 12:46:30 +00:00
Iago Toral Quiroga 44fa8304d4 v3dv: add a refcount mechanism to BOs
Until now we have lived without a refcount mechanism in the driver
because in Vulkan the user is responsible for handling the life
span of memory allocations for all Vulkan objects, however,
imported BOs are tricky because the kernel doesn't refcount
so user-space needs to make sure that:

1. When importing a BO into the same device used to create it
   (self-importing) it does not double free the same BO.
2. Frees imported BOs that were not allocated through the same
   device.

Our initial implementation always freed BOs when requested,
so we handled 2) correctly but not 1) and we would double-free
self-imported BOs. We tried to fix that in commit d809d9f3
but that broke 2) and we started to leak BOs for some imports.

This fixes the problem for good by adding refcounts to BOs
so that self-imported BOs have a refcnt > 1 and are only freed
when all references are freed.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5769
Tested-by: Roman Stratiienko <r.stratiienko@gmail.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14392>
2022-01-05 12:22:45 +00:00
Marek Olšák 946bd90a09 radeonsi: decrease the size of si_pm4_state::pm4 except for cs_preamble_state
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák a031cd3c2f radeonsi: replace SI_PM4_MAX_DW with a max_dw field
it will vary

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák 005fc20a34 radeonsi: pack si_pm4_state
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák 3ea5beca1f radv: apply spi_cu_en to CU_EN
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák 955c6de1c1 radv: set COMPUTE_DESTINATION_EN_SEn to spi_cu_en
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák 384014bebe radeonsi: apply spi_cu_en to CU_EN
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák b06b481dfe radeonsi: program COMPUTE_STATIC_THREAD_MGMT_SE4..7 on Arcturus
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák 5406ad93a9 radeonsi: set COMPUTE_DESTINATION_EN_SEn to spi_cu_en
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák 470b61f3a9 ac/gpu_info: add AMD_CU_MASK environment variable to set CU_EN
requested internally

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Marek Olšák a68cb9db8d ac/gpu_info: set cu_mask correctly for Arcturus
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14122>
2022-01-05 01:36:10 -05:00
Emma Anholt b9e8936bfb i915g: Turn off FP16 in the vertex shaders.
This ended up being turned on in gallivm, but since we use nir_to_tgsi on
the VS and TGSI doesn't have FP16, we can't let that happen.

Fixes: f814a2449e ("llvmpipe: enable FP16 and update CL + traces piglit results.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14403>
2022-01-05 03:15:15 +00:00
satmandu d27805753f Fix compilation on armv7l with gcc 11.2.0
Cc: mesa-stable
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12810>
2022-01-05 02:48:29 +00:00
Timothy Arceri d2711f9b61 glsl/glcpp: make sure to expand new token after concatenation
Previously the code was using a hack to change the token type from
INDETIFIER -> OTHER in order to avoid getting in an infinite loop
expanding the tokens. This worked ok until we got to a paste where
the replacement parameters had already had their type changed
to OTHER because the newly created paste token would then
inherit the OTHER type and never get expanded inself.

For example with the follow code:

   #define STEP_ONE() \
	out_Color = vec4(0.0,1.0,0.0,1.0)

   #define GLUE(x,y) x ## _ ## y
   #define EVALUATE(x,y)  GLUE(x,y)
   #define STEP(stepname) EVALUATE(STEP, stepname)()
   #define PERFORM_RAYCASTING_STEP STEP(ONE)

This would get all the way to expanding PERFORM_RAYCASTING_STEP to
STEP_ONE() but because it was created via the paste `x ## _ ## y`
it would never get any further.

To fix this we remove the OTHER hack and instead just track if the
token has already been handled via a bool value `explanding`.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5724

Fixes: 28842c2331 ("glcpp: Implement token pasting for non-function-like macros")

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14101>
2022-01-04 23:36:42 +00:00
Emma Anholt 4dc6cd5933 tgsi/exec: Simplify indirects now that they always use the ADDR file.
This was a lot of extra code in the hot path of getting though
fetch_src_file_channel().  No significant perf difference in softpipe,
though.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14360>
2022-01-04 23:05:41 +00:00
Emma Anholt c00db99e0e gallium: Delete PIPE_CAP_TGSI_ANY_REG_AS_ADDRESS
Softpipe was the only driver still using this feature.  I had enabled it
in ba22f014f9 ("softpipe: Enable PIPE_CAP_TGSI_ANY_REG_AS_ADDRESS;") for
an instr count win, but it's really not important to that driver and it's
not worth keeping the knob around just for that.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14360>
2022-01-04 23:05:41 +00:00
Emma Anholt 4bb9c0a28a nir_to_tgsi: Use the same address reg mappings as GLSL-to-TGSI did.
It turns out r600 has a bunch of expectations about the Dimension being in
ADDR[1].x, and sampler or atomic indirects being in ADDR[2].x.  It's
simpler to just use this static assignment than our dynamic one, anyway.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14360>
2022-01-04 23:05:41 +00:00
Biju Das daf59694ac kmsro: Add 'rcar-du' driver support
rcar-du is used by Renesas R-Car and RZ/G SOCs.
Driver is available in the mainline kernel [1]
[1] https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/rcar-du

Suggested-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: Show Liu <show.liu.yj@renesas.com>
Signed-off-by: Biju Das <biju.das.jz@bp.renesas.com>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14393>
2022-01-04 21:59:16 +00:00
Adam Jackson 69678d50d1 mesa: Remove unused _mesa_get_linear_format_srgb
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14220>
2022-01-04 19:45:55 +00:00
Adam Jackson 267f28e384 mesa: Remove unused _mesa_format_fallback_rgbx_to_rgba
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14220>
2022-01-04 19:45:55 +00:00
Adam Jackson 07d23c207a mesa: Remove unused _mesa_bind_texture
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14220>
2022-01-04 19:45:55 +00:00
Adam Jackson e44ce092be mesa: Remove unused _mesa_AllocTextureStorage_sw
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14220>
2022-01-04 19:45:54 +00:00
Adam Jackson c77da4de4b mesa: Remove unused _mesa_allow_light_in_model
ctx::_ForceEyeCoords is now never not false, so remove and simplify to
match.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14220>
2022-01-04 19:45:54 +00:00
Timur Kristóf d34d0f38e1 radv: Support VRS for mesh shaders.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14193>
2022-01-04 17:46:02 +00:00
Timur Kristóf bc94c2718a aco: Emit VRS rate when it's per-primitive.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14193>
2022-01-04 17:46:02 +00:00
Timur Kristóf 811c001049 radv: Lower primitive shading rate for mesh shaders.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14193>
2022-01-04 17:46:02 +00:00
Timur Kristóf c6becb9574 radv: Note when a mesh shader writes the primitive shading rate.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14193>
2022-01-04 17:46:02 +00:00
Rob Clark 94f465856c clover: Move min image support check
If the gallium driver supports images, but not the minimum image
requirements for CL, then simply don't claim to support images.  This
is better than not claiming to support CL at all.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14310>
2022-01-04 16:56:25 +00:00
Samuel Pitoiset 12ac44378d radv: add UMR markers for the vertex prolog
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13376>
2022-01-04 07:50:07 +00:00