Commit Graph

585 Commits

Author SHA1 Message Date
Marek Olšák 3bd9db5be3 r300,r600,radeonsi: inline struct radeon_cmdbuf to remove dereferences
It's straightforward except that the amdgpu winsys had to be cleaned up
to allow this.

radeon_cmdbuf is inlined and optionally the winsys can save the pointer
to it. radeon_cmdbuf::priv points to the winsys cs structure.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7907>
2020-12-05 10:52:17 -05:00
Marek Olšák 8904fcca6d gallium: inline struct u_suballocator to remove dereferences
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7901>
2020-12-03 21:41:19 +00:00
Marek Olšák c7470c1760 radeonsi: don't set DrawID and StartInstance if they are unused
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7721>
2020-12-01 15:33:03 -05:00
Marek Olšák c4ddf67ee1 radeonsi: don't invalidate emitted NUM_INSTANCES for u_blitter
invalidate_draw_sh_constants should invalidate only SGPRs.
invalidate_draw_constants invalidates SGPRs and NUM_INSTANCES.

u_blitter called invalidate_draw_sh_constants, which previously
invalidated NUM_INSTANCES as well. This commit fixes that.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7721>
2020-12-01 15:33:03 -05:00
Marek Olšák 509142876b radeonsi: add AMD_DEBUG=nofastlaunch for debugging
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7721>
2020-12-01 15:33:03 -05:00
Indrajit Kumar Das 5d14562da8 radeonsi/gfx10: fix overflow and primitive queries
This aligns the offsets to match the memory layout of the query buffer
defined by gfx10_sh_query_buffer_mem and calls si_launch_grid_internal
to flush caches and wait for completion of shaders prior to retrieving
results.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7181>
2020-12-01 11:34:16 +00:00
Marek Olšák 1190808eca radeonsi: if VS and TCS have the same number of threads, merge the conditonals
Instead of:
    if (VS) {
	VS;
    }
    if (TCS) {
	TCS;
    }

Do this if the number of threads is the same in VS and TCS:
    exec = enabled_threads;
    VS;
    TCS;

Skipping declare_vb_descriptor_input_sgprs is needed to match the VS return
values.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7623>
2020-11-23 02:22:21 +00:00
Marek Olšák 602d4a78bc radeonsi: handle pipe_draw_info::increment_draw_id
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7441>
2020-11-18 01:41:25 +00:00
Pierre-Eric Pelloux-Prayer 6e7e208867 radeonsi: remove AMD_DEBUG=zerovram flag
The same feature is available by using: radeonsi_zerovram=true

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7525>
2020-11-13 11:19:58 +00:00
Pierre-Eric Pelloux-Prayer b9605f1a74 radeonsi: remove unused NO_RB_PLUS flag
It's not used since https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/1751.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7525>
2020-11-13 11:19:58 +00:00
Bas Nieuwenhuizen d4f7962d48 radeonsi: Add displayable DCC flushing without explicit flushes.
Flushes non-explicit shared textures that need retiling on

* glFlush
* glSync
* glSignalSemaphoreEXT
* DRI fences.
* The first time we create a non-explicit handle for it.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6176>
2020-11-13 03:27:28 +00:00
Marek Olšák a44868beda radeonsi: implement multi_draw for compute-based primitive culling
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7056>
2020-10-31 00:18:11 +00:00
Marek Olšák 0ce68852c1 radeonsi: implement multi_draw but supporting only 1 draw
just adapting to the new interface

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7056>
2020-10-31 00:18:11 +00:00
Marek Olšák 7cc939f7dd radeonsi: add num_draws parameter into si_need_gfx_cs_space
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7056>
2020-10-31 00:18:11 +00:00
Marek Olšák b7501184b9 radeonsi: implement inlinable uniforms
This improves performance for uber shaders.

It must be enabled using the new driconf option.

The driver compiles the specialized shaders in another thread without stalls,
same as all other optimizations.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7057>
2020-10-30 11:07:22 +00:00
Marek Olšák ed3c5fe469 radeonsi: implement GL_INTEL_blackhole_render
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7031>
2020-10-06 15:59:08 +00:00
Marek Olšák 30c3b2c0b6 radeonsi: simplify NGG culling enablement and add radeonsi_shader_culling option
Add a vertex count threshold into si_shader_selector to simplify
the draw_vbo code.

The new option is supposed to be used in 00-mesa-defaults.conf and should be
tweaked for best performance unlike the AMD_DEBUG experimental options.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6948>
2020-10-01 16:29:46 +00:00
Eleni Maria Stea 03af98abe2 radeonsi: support for external buffers (ext_external_objects)
So far, the callback to create a resource from a memory object had code
for importing textures only. Modified it to allow importing buffers too.

Fixes the following piglit tests:
- ext_external_objects/vk-buf-exchange
- ext_external_objects/vk-pix-buf-update-errors
- ext_external_objects/vk-vert-buf-update-errors
- ext_external_objects/vk-vert-buf-reuse

v2: Used si_alloc_buffer_struct instead of CALLOC
v3: Fixed indentation issue, removed free in case of unsuccessful
allocation, joined two if conditions together

Signed-off-by: Eleni Maria Stea <estea@igalia.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6364>
2020-10-01 15:35:07 +00:00
Pierre-Eric Pelloux-Prayer 2c6643546a radeonsi/tmz: add a tmz variant for sctx::eop_bug_scratch
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>
2020-09-24 14:51:16 +00:00
Pierre-Eric Pelloux-Prayer 8e2768bbfb radeonsi/tmz: add tmz variant for sctx::tess_rings
tess_rings must be encrypted when used in a secure job so this commit
introduces a tess_rings_tmz resource.

The cs_preamble_state doesn't contain the tess_rings address anymore since
it can change. The tess_rings related registers go in a separate preamble.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>
2020-09-24 14:51:16 +00:00
Pierre-Eric Pelloux-Prayer 2589888ce9 radeonsi/tmz: add tmz variant of sctx::wait_mem_scratch
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>
2020-09-24 14:51:16 +00:00
Pierre-Eric Pelloux-Prayer 1b0d660cbc radeonsi/tmz: allow secure job if the app made a tmz allocation
This commit makes TMZ always allowed instead of being either off or forced-on
with AMD_DEBUG=tmz.

With this change:
- secure job can be used as soon as the application made a tmz allocation. Driver
  internal allocations are not enough to enable secure jobs (if tmz is supported
  and enabled by the kernel)
- AMD_DEBUG=tmz forces all scanout/depth/stencil buffers to be allocated as TMZ.
  This is useful to test app thats don't explicitely support protected content.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>
2020-09-24 14:51:16 +00:00
Pierre-Eric Pelloux-Prayer 5e4aecec93 radeonsi: introduce SI_RESOURCE_FLAG_INTERNAL / RADEON_FLAG_DRIVER_INTERNAL
Tag allocations as driver internal.
Some of these allocations will need to be doubled to handle TMZ (one secure bo,
one normal bo) but these allocations shouldn't switch the winsys in "the app
is using TMZ".

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>
2020-09-24 14:51:16 +00:00
Marek Olšák 972fb0368c radeonsi: move binning parameters into si_screen
it will be used in the next commit

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6822>
2020-09-24 11:55:06 +00:00
Marek Olšák 40a50e9398 radeonsi: remove KILL_PS_INF_INTERP/CLAMP_DIV_BY_ZERO, use screen::options
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6810>
2020-09-22 15:58:51 +00:00
Bas Nieuwenhuizen 017ca86b22 radeonsi: Move display dcc dirty tracking to framebuffer emission.
To improve performance.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6783>
2020-09-19 03:15:28 -04:00
Bas Nieuwenhuizen c6c1fa9a26 radeonsi: Put retile map in separate buffers.
The retile maps are a software mechanism and hence very suceptible
to change. As such I'd like to avoid making it part of the cross
driver ABI.

Ideally we'd just use the cached tile info + a shader to avoid these
buffers altogether.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6783>
2020-09-19 03:15:25 -04:00
Marek Olšák b23013db0a Revert "radeonsi: set BIG_PAGE fields on gfx10.3"
This reverts commit 430d384c31.

BIT_PAGE can't be set for GTT and we don't know if a buffer has been
evicted to GTT.

Fixes: 430d384c31

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6722>
2020-09-16 02:54:01 +00:00
Marek Olšák cb7bc983ae radeonsi: stop using TGSI_PROPERTY_FS_COLOR0_WRITES_ALL_CBUFS
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6624>
2020-09-07 11:15:40 +00:00
Marek Olšák a407123789 radeonsi: move nir_shader_compiler_options into si_screen
so that they can be different depending on the GPU (for 16-bit support)

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6284>
2020-09-06 14:36:20 +00:00
Marek Olšák 3c54d73e4b radeonsi: change PIPE_SHADER to MESA_SHADER (debug flags)
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6340>
2020-09-02 23:03:00 -04:00
Marek Olšák b1cb72c449 radeonsi: change PIPE_SHADER to MESA_SHADER (si_shader_selector::type)
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6340>
2020-09-02 23:03:00 -04:00
Pierre-Eric Pelloux-Prayer b8445520cb radeonsi,driconf: add clamp_div_by_zero option
Cc: mesa-stable
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6259>
2020-09-02 11:53:16 +02:00
Marek Olšák b8892bc818 radeonsi: don't restore states at the beginning of IBs if they're shadowed
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5798>
2020-07-22 12:08:33 -04:00
Marek Olšák 69014d8c94 radeonsi: implement CP register shadowing
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5798>
2020-07-22 12:08:19 -04:00
Timothy Arceri 4686a95621 r600/radeonsi: silence zero-length-bounds gcc warnings
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5766>
2020-07-08 03:04:03 +00:00
Marek Olšák 50d7553600 radeonsi: add a debug option to enable NGG culling for tessellation
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5524>
2020-06-30 10:56:41 +00:00
Marek Olšák 9049e39804 radeonsi: always use Wave32 for GS fast launch, because Wave64 hangs
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5524>
2020-06-30 10:56:41 +00:00
Marek Olšák 1c1d34a67a radeonsi: rename init_config states to cs_preamble states
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5603>
2020-06-26 07:02:57 +00:00
Marek Olšák 430d384c31 radeonsi: set BIG_PAGE fields on gfx10.3
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5383>
2020-06-09 16:17:36 +00:00
Marek Olšák 85a6bcca61 radeonsi: pass at most 3 images and/or shader buffers via user SGPRs for compute
This should slightly decrease shader lifetime.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5209>
2020-06-02 20:47:49 +00:00
Marek Olšák 7b6b35c6b5 radeonsi: move resetting tracked registers into a new function
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5095>
2020-05-23 03:45:07 -04:00
Marek Olšák 7356144fe4 radeonsi: disable the L2 cache for most CPU mappings of textures
for faster blits over PCIe and no need to flush L2

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4935>
2020-05-15 22:12:35 +00:00
Marek Olšák 2c4c1b0499 radeonsi: rename SI_RESOURCE_FLAG_TRANSFER to FORCE_LINEAR
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4935>
2020-05-15 22:12:35 +00:00
Pierre-Eric Pelloux-Prayer 8873ea0e25 radeonsi: determine secure flag must be set for gfx IB
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4401>
2020-05-11 10:25:53 +02:00
Blaž Tomažič 808eb20186 radeonsi: Fix omitted flush when moving suballocated texture
Fixes: 5e805cc74b "radeonsi: flush the context after resource_copy_region for buffer exports"

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4925>
2020-05-07 17:00:08 -04:00
Marek Olšák 0d83e7f4b9 radeonsi: enable TC-compatible HTILE on demand for best Z/S performance
I haven't measured this, but it can only help.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4866>
2020-05-05 16:27:29 +00:00
Pierre-Eric Pelloux-Prayer 64662dd5ba radeonsi: add workaround for issue 2647
For unknown reasons pixel shaders in KSP game get executed with
infinite interpolation coefficients and this causes an infinite
loop in the shader.

This commit adds a hacky workaround that kills pixel shaders if
invalid interp coeffs are detected and enables it for KSP.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2174
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2647
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4700>
2020-05-05 09:41:14 +00:00
Marek Olšák d6acdbd935 radeonsi: implement and use compute-based DCC decompression on gfx9-10
DCC_DECOMPRESS doesn't work. Instead of trying to figure out why,
use a compute blit where the load is compressed and the store is
uncompressed.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4761>
2020-04-30 22:27:31 +00:00
Marek Olšák d3da73954a radeonsi: add SI_IMAGE_ACCESS_DCC_OFF to ignore DCC for shader images
A shader-based DCC decompress pass will use this.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4761>
2020-04-30 22:27:31 +00:00