The RADEON_GEM_USERPTR_ANONONLY flag is hardcoded here which excludes
shared memory pages. DRM is actually capable of supporting shared file-
backed memory, but only if it's read-only. This mutability intent has to
be conveyed through the stack, so a flags argument is added to the winsys
regime to pass RADEON_FLAG_READ_ONLY.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16115>
It was getting difficult to add more heaps.
This adds more heaps because more flag combinations are legal now.
Invalid flag combinations are also handled better.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16466>
Otherwise the pb_slabs might be freed by another thread in between.
Valgrind example:
==676841== Invalid read of size 1
==676841== at 0x6B0A8B3: get_slab_wasted_size (amdgpu_bo.c:659)
==676841== by 0x6B0AD7D: amdgpu_bo_slab_destroy (amdgpu_bo.c:684)
==676841== by 0x6ACF94F: pb_destroy (pb_buffer.h:259)
==676841== by 0x6ACF94F: pb_reference_with_winsys (pb_buffer.h:282)
==676841== by 0x6ACF94F: radeon_bo_reference (radeon_winsys.h:754)
==676841== by 0x6ACF94F: si_replace_buffer_storage (si_buffer.c:274)
==676841== by 0x6957036: tc_call_replace_buffer_storage (u_threaded_context.c:1554)
[...]
==676841== by 0x4ECCDEE: clone (clone.S:95)
==676841== Address 0x27879945 is 5 bytes inside a block of size 208 free'd
==676841== at 0x48399AB: free (vg_replace_malloc.c:538)
==676841== by 0x6B0E8BD: amdgpu_bo_slab_free (amdgpu_bo.c:863)
==676841== by 0x6B89D4A: pb_slabs_reclaim_locked (pb_slab.c:84)
==676841== by 0x6B89D4A: pb_slab_alloc (pb_slab.c:130)
==676841== by 0x6B0EE7F: amdgpu_bo_create (amdgpu_bo.c:1429)
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4736
Fixes: 965c6445ad ("winsys/amdgpu,radeonsi: add HUD counters for how much memory is wasted by slabs")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11010>
Alignments are always 2^n, so store n = log2(alignment). The next commit
will take advantage of the saved space.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9809>
This will allow removing the winsys pointer from buffers.
The amdgpu winsys adds dummy_ws to get radeon_winsys because there can be
no radeon_winsys around (e.g. while amdgpu_winsys is being destroyed), but
we still need some way to call buffer functions.
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9809>
This will allow removing the winsys pointer from buffers.
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9809>
because amdgpu_bo_wait doesn't wait for active CS jobs using the buffers.
This fixes incorrect buffer reuse of busy buffers.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8895>
It's never true with slab and sparse buffers.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8849>
It's never true with slab and sparse buffers.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8849>
It's never true with slab and sparse buffers.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8849>
It's never used with slab and sparse buffers.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8849>
The initialization of abs_timeout fixes a warning that started appearing
with this commit.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8849>
This decreases the CPU time percentage of amdgpu_cs_add_buffer by 50%
on Ryzen 3900X.
We don't need to call amdgpu_bo_is_referenced_by_any_cs
in amdgpu_bo_can_reclaim. The reclaim function is only called for buffers
that have 0 references.
The only downside is that amdgpu_bo_is_referenced_by_cs might be slower
in some very rare cases. Overall the driver overhead is better.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8849>
Instead of aligning slab allocations to powers of two (e.g. 129K -> 256K),
implement slab allocations with 3/4 of power of two sizes to reduce
overallocation. (e.g. 129K -> 192K)
The limitation is that the alignment must be 1/3rd of the allocation size.
DeusExMD allocates 2.1 GB of VRAM. Without this, slabs waste 194 MB due
to alignment, i.e. 9.2%. This commit reduces the waste to 102 MB, i.e. 4.9%.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8683>
Slabs always allocate the next power of two size from their pools. This
wastes memory if the size is not a power of two.
bo->base.size is overwritten because the default is the allocated power of
two size, but we need the real size to compute the wasted size in
amdgpu_bo_slab_destroy. entry_size is added to the hole in pb_slab_entry
to hold the real entry size.
Like other memory stats, no atomics are used.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8683>
Replace mesa's slightly different container_of() with one more aligned
to the linux kernel's version which takes a type as the 2nd param. This
avoids warnings like:
freedreno_context.c:396:44: warning: variable 'batch' is uninitialized when used within its own initialization [-Wuninitialized]
At the same time, we can add additional build-time type-checking asserts
Signed-off-by: Rob Clark <robdclark@chromium.org>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7941>
It's straightforward except that the amdgpu winsys had to be cleaned up
to allow this.
radeon_cmdbuf is inlined and optionally the winsys can save the pointer
to it. radeon_cmdbuf::priv points to the winsys cs structure.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7907>
This commit makes TMZ always allowed instead of being either off or forced-on
with AMD_DEBUG=tmz.
With this change:
- secure job can be used as soon as the application made a tmz allocation. Driver
internal allocations are not enough to enable secure jobs (if tmz is supported
and enabled by the kernel)
- AMD_DEBUG=tmz forces all scanout/depth/stencil buffers to be allocated as TMZ.
This is useful to test app thats don't explicitely support protected content.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6049>
The indentation is on purpose. The whole file will be reindented to this
code style some other time.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4863>
Jason Volk
Marek Olšák
Pierre-Eric Pelloux-Prayer
Rob Clark