Commit Graph

669 Commits

Author SHA1 Message Date
Marek Olšák 5af124c92c radeonsi: change si_resource::alignment to alignment_log2 for better packing
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10813>
2021-05-25 16:15:44 +00:00
Marek Olšák 36e07198a7 radeonsi: always use the L2 LRU cache policy for faster clears and copies
Waves and CP DMA can finish sooner if L2 doesn't do any evictions, which
is hard to predict.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10813>
2021-05-25 16:15:44 +00:00
Marek Olšák c7e731c737 radeonsi: remove unused SI_IMAGE_ACCESS_AS_BUFFER
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10813>
2021-05-25 16:15:44 +00:00
Marek Olšák 94a1f45e15 ac/llvm: set target features per function instead of per target machine
This is a cleanup that allows the removal of the wave32 target machine and
the wave32 pass manager.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10813>
2021-05-25 16:15:44 +00:00
Marek Olšák b04044b350 radeonsi: stop using u_resource_vtbl::resource_destroy
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10659>
2021-05-21 17:38:04 +00:00
Marek Olšák ec77a2d43a gallium/u_threaded: add callbacks and documentation for resource busy checking
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10662>
2021-05-17 10:37:24 +00:00
Marek Olšák 967757a208 gallium+(u_threaded,r300,r600,radeonsi): move transfer offset into pipe_transfer
Let's use the 4 bytes of unused padding usefully in pipe_transfer.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10527>
2021-05-01 17:38:42 +00:00
Mike Blumenkrantz dae3113c3d gallium: split drawid out of pipe_draw_info and as a separate draw_vbo param
the only case in which this is nonzero is if a multidraw gets split by the frontend,
i.e., mesa core, and in all other cases it can be ignored. the value can also be ignored
for all indirect draws, though it seems many (most?) gallium drivers are not aware of this

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10166>
2021-04-30 03:59:19 +00:00
Mike Blumenkrantz 4566383ae4 gallium: move pipe_draw_info::index_bias to pipe_draw_start_count_bias
this moves index_bias into the multidraw struct, enabling draws where the value
changes to be merged; the draw_info struct member is renamed and moved to the end
of the struct for tc use

u_vbuf still has some checks to split draws if index_bias changes, maybe
this can be removed at some point?

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10166>
2021-04-30 03:59:19 +00:00
Mike Blumenkrantz 4fe6c85526 gallium: rename pipe_draw_start_count -> pipe_draw_start_count_bias
and add an index_bias member

no functional changes yet, just the rename and unused struct member

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10166>
2021-04-30 03:59:19 +00:00
Marek Olšák 804e292440 radeonsi: remove the separate DCC optimization for Stoney
This removes some complexity from the driver.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10343>
2021-04-26 22:53:30 +00:00
Marek Olšák 1f8fa96412 radeonsi: make the gfx9 DCC MSAA clear shader depend on the number of samples
because different DCC equations are used.

Fixes: 3120113ee7 - radeonsi: implement DCC MSAA 4x/8x fast clear using DCC equations on gfx9

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10343>
2021-04-26 22:53:30 +00:00
Simon Ser 4a6b87ceab radeonsi: implement pipe_context.create_video_buffer_with_modifiers
Just pass down the modifier list to vl_video_buffer_create_as_resource,
filtering out DCC modifiers because we don't support these for now.

Signed-off-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10237>
2021-04-22 15:57:29 +00:00
Marek Olšák a1653854f5 radeonsi: fix automatic DCC retiling after compute image stores
Only internal compute shaders use DCC stores, so the TODOs are not
critical yet.

Fixes: 1d64a1045e - radeonsi: enable dcc image stores on gfx10+

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10261>
2021-04-17 02:37:49 +00:00
Marek Olšák f9b527a9a5 radeonsi: unify internal compute with SSBOs in si_launch_grid_internal_ssbos
just deduplicate the code

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák ec60526035 radeonsi: move binding the internal compute shader into si_launch_grid_internal
instead of doing it in each function

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák 3120113ee7 radeonsi: implement DCC MSAA 4x/8x fast clear using DCC equations on gfx9
MSAA 4x and 8x should only clear the first 2 samples because other samples
are uncompressed. The compute shader only clears that subset of DCC.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák 8b95f51ef1 radeonsi: fix and enable full DCC with MSAA 2x on gfx9
This enables fast clear with any clear color (not just 0/1) for bpp >= 32.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák 7e68fae25f ac,radeonsi: rewrite DCC retiling without the DCC retile map
The retile map is removed and replaced by direct DCC address computations
in the retile shader using the new function ac_nir_dcc_addr_from_coord.

The RADV code is disabled.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák 06b6af596c radeonsi: do Z-only or S-only HTILE clear using a compute shader doing RMW
This adds a clear_buffer compute shader that does read-modify-write to
update a subset of bits in HTILE.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák 4dd8d58ad5 radeonsi: clean up some mess around htile_stencil_disabled
Set the final value in si_texture_create_object, so that other places
don't have to derive it redundantly.

The only thing to remember is that HTILE stencil can be enabled when
stencil is not present, and it can be disabled when stencil is present
due to various workarounds.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák bcd1a69f79 radeonsi: parallelize Z/S conversion into TC-compatible with fast color clears
It's not really a fast clear, but it's the next logical step towards doing
HTILE clears here.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák fb72d41b18 radeonsi: implement Z/S fast clear for non-zero mipmap levels
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10003>
2021-04-13 03:17:42 +00:00
Marek Olšák faf10bd49d ac/surface: use named "color and "zs" structures in unions
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10083>
2021-04-12 20:53:45 +00:00
Marek Olšák 468836317b ac/surface: unify htile_* and dcc_* fields as meta_* fields
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10083>
2021-04-12 20:53:45 +00:00
Pierre-Eric Pelloux-Prayer 8c6a64c9b0 radeonsi/rgp: export compute shader programs
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10105>
2021-04-12 14:27:29 +02:00
Pierre-Eric Pelloux-Prayer aa077ba3a2 radeonsi/rgp: export barriers
Wrap the si_cp_wait_mem call to emit RGP_SQTT_MARKER_IDENTIFIER_BARRIER_START and
RGP_SQTT_MARKER_IDENTIFIER_BARRIER_END events.

Only for gfx9+ for now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10105>
2021-04-12 14:27:26 +02:00
Marek Olšák 0580d4c1a2 radeonsi: enable HTILE with mipmapping on gfx9+
Everything seems to be there except fast clears.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 3345e32de7 radeonsi: group and parallelize all clears in si_texture_create_object
This reduces aux_context flushes significantly.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 8cd61d1248 radeonsi: parallelize CMASK and DCC clears
Clearing 8 RTs with both DCC and CMASK caused 16 synchronized clears where
we also did 16 times WAIT_REG_MEM for CB flushes that were 15 times
useless.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák d0f06e5c47 radeonsi: remove si_screen::dcc_msaa_allowed
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 4707dc6a64 radeonsi: determine accurately whether the framebuffer state has DCC MSAA
We only need to check storage samples, which is what affects DCC.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 034c1e4845 radeonsi: decrease the maximum variable block size
to allow packing the block size in 1 user SGPR with 10 bits per component,
so that block sizes such as 512x1x1 fit in there.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák c53261645d radeonsi: add SI_CONTEXT_PFP_SYNC_ME to skip syncing PFP for image operations
DCC/CMASK/HTILE clears will not set this. We could do a better job
at not setting this in other cases too

Image copies also don't set this.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák b1a73ec99b radeonsi: rename and apply SI_OP_CPDMA_SKIP_CACHE_FLUSH to compute as well
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 7e2b5ce722 radeonsi: set compute/cpdma sync flags in the outermost caller
This allows us to control syncing everywhere.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 1af99a28a0 radeonsi: merge CP DMA flags with internal compute flags
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák dd5e9af78f radeonsi: remove unused SI_CP_DMA_SKIP_* definitions
The existing uses had no effect.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 938dc0e291 radeonsi: rename internal compute sync flags
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák 28d065d3e5 radeonsi: don't insert start/stop pipeline stat events if it has no effect
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9795>
2021-04-02 12:05:00 +00:00
Marek Olšák e5ea9a3baa radeonsi: add a fast path for MSAA resolving with RGB -> BGR swizzling
When we encounter a situation when we need to swizzle, which the CB can't
resolve in one pass, swap the channel order on the next clear, so that we
don't have to swizzle.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9615>
2021-03-19 16:05:03 +00:00
Marek Olšák a94bd9033d radeonsi: use pipe_sampler_state::border_color_is_integer to simplify stuff
We don't need the separate integer sampler state if we know the border
color type.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9577>
2021-03-17 22:36:42 +00:00
Marek Olšák 32eb74e1e1 ac/gpu_info: fix more non-coherent RB and GL2 combinations
It ignored non-harvested chips with a non-power-of-two memory bus.

Fixes: abed921ce7 - amd: add support for Navy Flounder

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9568>
2021-03-17 14:40:54 +00:00
Axel Davy 8283ed65cf radeonsi: Limit the size of the in-memory shader cache
The in-memory shader cache can get significantly
huge in some rare cases.
Limit its size to 64MB on 32 bits, and 1GB else.

Signed-off-by: Axel Davy <davyaxel0@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9578>
2021-03-13 21:51:38 +00:00
Marek Olšák e6a0f243ea radeonsi: update pipe_screen::num_contexts
This allows skipping mutex locking. Don't take the aux context into account.

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9356>
2021-03-11 05:05:39 +00:00
Pierre-Eric Pelloux-Prayer c276bde34a radeonsi/sqtt: export shader code to RGP
With these changes the shader code is visible in RGP.

Vk pipeline feature is emulated using si_update_shaders: when shaders are
updated we compute a sha1 of their code and use it as a pipeline hash.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9277>
2021-03-05 13:10:11 +00:00
Marek Olšák c97ebe1461 radeonsi: don't index si_context::shaders with enum gl_shader_stage
Fixes: a8373b3d38 "radeonsi: store si_context::xxx_shader members in union"

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9313>
2021-03-02 01:14:44 +00:00
Pierre-Eric Pelloux-Prayer 1d64a1045e radeonsi: enable dcc image stores on gfx10+
This was implemented in 1d3bffaf9c,
but missing the WRITE_COMPRESS_ENABLE bit, then disabled by
4dc6ed2a59040f04648eadbffeb1522587d00f3.

This commits reimplements it to:
- avoid disabling dcc when uploading FP16 textures
  (see si_use_compute_copy_for_float_formats)
- being able to use compute to upload textures in more cases, rather
  than using the blit path

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8958>
2021-02-17 14:57:26 +01:00
Pierre-Eric Pelloux-Prayer f18bceac72 radeonsi: replace force_cp_dma arg of si_clear_buffer by enum
The new enum has 3 values:
 - SI_CP_DMA_CLEAR_METHOD: equivalent to force_cp_dma = true
 - SI_COMPUTE_CLEAR_METHOD: to force the clear to use compute
 - SI_AUTO_SELECT_CLEAR_METHOD: equivalent to force_cp_dma = false

No functional change yet, but this will be used later.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8958>
2021-02-17 14:57:26 +01:00
Pierre-Eric Pelloux-Prayer bddc0e023c radeonsi: fix read from compute / write from draw sync
A compute dispatch should see the result of a previous draw command.
radeonsi was missing this implicit sync, causing rendering artifacts:
the compute shader was reading from a texture still being written to
by the previous draw.

Framebuffer BOs are marked with RADEON_USAGE_NEEDS_IMPLICIT_SYNC,
so compute jobs will sync.

v2: use RADEON_USAGE_NEEDS_IMPLICIT_SYNC
v3: unconditionally make CB coherent after a flush

Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com> (v3)
Reviewed-by: Marek Olšák <marek.olsak@amd.com> (v3)
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4032
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2878
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/1336
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8869>
2021-02-17 09:11:46 +00:00