Commit Graph

243 Commits

Author SHA1 Message Date
Rob Clark fb8fe017ee freedreno: Don't try staging blit for non-renderable formats
If a format is not supported as a render target, there is no point in
trying a staging blit, as it will end up in a CPU copy fallback.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11371>
2021-06-15 19:09:24 +00:00
Mike Blumenkrantz 74abd5df0e aux/tc: pass rebind count and rebind bitmask with replace_buffer_storage func
tc already calculates all the rebinding that needs to be done on a given
context, so (some of) this info can be passed on to drivers to enable
optimizations

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11245>
2021-06-14 20:42:47 +00:00
Rob Clark 38dd2e6e41 freedreno: Remove assert
In multi-context scenarios, one context writing to a resource can race
with a pctx->flush_resource() on another context/thread.  Which means
that by the end of flush_resource() we can have a new write_batch.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11200>
2021-06-09 19:09:09 -07:00
Marek Olšák eb74f97769 gallium: split transfer_(un)map into buffer_(un)map and texture_(un)map
The u_resource_vtbl indirection is going to be removed.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10659>
2021-05-21 17:38:04 +00:00
Rob Clark ea112c5104 freedreno: Implement TC resource_busy
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10883>
2021-05-21 16:34:11 +00:00
Rob Clark a6d9a5f55a freedreno: Rename internal resource_busy
So I can re-use the name fd_resource_busy.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10883>
2021-05-21 16:34:11 +00:00
Marek Olšák ec77a2d43a gallium/u_threaded: add callbacks and documentation for resource busy checking
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10662>
2021-05-17 10:37:24 +00:00
Rob Clark bc3f66311d freedreno: Flush resources harder
pctx->flush_resource() has the same expectations that the resource can
be shared with an external client as pctx->flush(), but without the
convenience of a fence to know *when* the resource must be visible to
that external client.  So we need to ensure the batch is flushed all the
way to the kernel so that implicit-sync can do it's job.

Fixes: e9a9ac6f77 ("freedreno/drm: Async submit support")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10530>
2021-05-01 08:46:27 -07:00
Rob Clark 8ab227c373 freedreno/drm: Cleanup bo cpu_prep flags
Also add some STATIC_ASSERT()

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
2021-04-28 15:36:42 +00:00
Rob Clark 7f0abd9048 freedreno/drm: Cleanup bo allocation flags
Most of them were actually unused.  The memory type (KMEM vs SMI) only
applied to very old a2xx era devices that had a small/fast stacked
memory (SMI) vs normal memory (KMEM).  And the cache flags are ignored
(ie. everything is writecombine), but we can add new cache flags later
when they actually do something.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
2021-04-28 15:36:42 +00:00
Rob Clark c64bc095e7 freedreno: Allow resource shadowing for TC
TC_TRANSFER_MAP_NO_INVALIDATE just means we can't invalidate.  It
doesn't preclude the resource shadow path.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
2021-04-28 15:36:42 +00:00
Rob Clark cb058a8922 freedreno: Avoid staging blits with stencil on older gens
This avoids infinite recursion with fallback to
util_resource_copy_region()

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
2021-04-28 15:36:42 +00:00
Rob Clark 224dbd77d5 freedreno: Small indent fix
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
2021-04-28 15:36:42 +00:00
Rob Clark 31782330da freedreno: Add missing foreach macros and update indentation
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10247>
2021-04-14 16:53:26 -07:00
Rob Clark 2d439343ea freedreno: Re-indent
clang-format -fallback-style=none --style=file -i src/gallium/drivers/freedreno/*.[ch] src/gallium/drivers/freedreno/*/*.[ch]

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8883>
2021-04-14 19:52:21 +00:00
Rob Clark 8cb51ba30e freedreno: Add dirty bit for state that needs rsc tracking
aa1ddb6fe3 skipped the tracking for the
!dirty case, but we can do a bit better and track at bind time whether
the state change is one that requires resource tracking.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9581>
2021-03-20 12:13:09 -07:00
Rob Clark 0cb989d71f freedreno: Add helpers to mark dirty state
Doesn't change anything yet, but this will let us more easily add
mapping from dirty gallium state to dirty gen-specific state-groups.

Note that the mapping from shader-state to global state in
fd_context_dirty_shader() optimizes out for release builds.  This
is kind of important, in the next patch we'll want ffs(SOME_CONST)
to optimize away even more.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9581>
2021-03-20 12:13:09 -07:00
Rob Clark 9dbe2405a3 freedreno: threaded_context support
Currently only initialized for a6xx, mostly because that is the easiest
setup for me to test and debug at the moment.  But the couple a6xx changes
should not require counterparts in older gens.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 1a12d682a6 freedreno: Check cb0 in rebind_resource()
Previously we were expecting cb0 to be user_buffer.  (We did in some
cases upload it to a gpu buffer, but this was an internally allocated
buffer and not something subject to rebind.)  But with TC it becomes
a gpu buffer.

(Technically, with pctx->const_uploader, we shouldn't hit the rebind
path for cb0, but better to not try to be overly clever.. sooner or
later that would bite us.)

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark acc2c015b3 freedreno: Add transfer_pool_unsync
With threaded_context, in the TC_TRANSFER_MAP_UNSYNC case, we are
getting called from the frontend thread, rather than driver thread.
So we need a different slab_child_pool for that.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 0c163e0a45 freedreno: Add fd_replace_buffer_storage()
This will be used by threaded_context to avoid stalls in the
DISCARD_WHOLE_RESOURCE case (and DISCARD_RANGE cases that can
be promoted to DISCARD_WHOLE_RESOURCE).

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 63649e4101 freedreno: Extract out helper for transfer-map flag munging
Split out the usage simplification from main part of transfer_map and
handle the threaded-context specific TC_TRANSFER_x flags.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 4f07a24e41 freedreno: Extend threaded_transfer
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 1017dc9f6e freedreno: Extend threaded_resource
No functional change, just big churny

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 5fbaa8033b freedreno: Restructure transfer_map()
Separate the parts that, with threaded_context, can be called from
either driver or frontend thread.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark 39d6343a3e freedreno: Split out batch/resource tracking
For threaded_context, to properly handle replace_buffer_storage, we'll
need to handle multiple "iterations" of a resource using the same
tracking in order to implement transfer_map() correctly.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:16 +00:00
Rob Clark f74ccde2c7 freedreno: Factor out common fd_resource init
Before adding new things that would need initialization in both paths,
refactor out a shared helper.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9323>
2021-03-11 04:42:15 +00:00
Rob Clark f8714b2852 freedreno: Remove dead-cells MBR workaround
With threaded-context we won't have a chance to apply the workaround in
the backend driver.  But the previous commit moves it to a driconf
configured workaround in mesa/st, so we can drop this now.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9316>
2021-03-03 22:47:59 +00:00
Rob Clark 2ed9dfbe6f freedreno: Add macro for duration based warns
Add a macro to do a perf_debug() if a block of code takes longer than a
specified amount of time.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9264>
2021-02-25 20:09:44 +00:00
Rob Clark fd4d759622 freedreno: Add FD_DBG() macro
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9264>
2021-02-25 20:09:44 +00:00
Rob Clark 1b2a35509e freedreno: Fix think-o in fd_resource_wait()
Fixes: dabec19b05 ("freedreno: Add perf_debug logging for bo stalls")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9264>
2021-02-25 20:09:44 +00:00
Rob Clark 9d03dc2537 freedreno: Workaround for UNSYNC+DISCARD_RANGE
Dead-cells (and perhaps others) does MapBufferRange(UNSYNC|DISCARD_RANGE)
to update single buffered VBOs every frame in the opening menu screen,
and because we were considering UNSYNC with higher priority than
DISCARD_RANGE, this would result in the game racing VBO updates between
binning and tile passes, causing visibility stream inconsistency between
the two passes, resulting in "tile flicker".

The letter of the gl spec implies this is undefined behavior (at least
to my reading of it).  But we already hand DISCARD_RANGE in the !UNSYNC
case, so just go down this path instead.  It means we could potentially
end up invalidating (and back-blit) in cases where the app really knows
what it is doing, but oh well.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4337
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9223>
2021-02-24 00:19:33 +00:00
Rob Clark dabec19b05 freedreno: Add perf_debug logging for bo stalls
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9223>
2021-02-24 00:19:33 +00:00
Rob Clark e96cf1ed63 freedreno: Handle InvalidateBufferData() case
We already had all the mechanism, it just wasn't wired up.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9223>
2021-02-24 00:19:33 +00:00
Rob Clark 3f77604623 freedreno: driver-thread annotations
Use clangs thread-safety annotations to implement a virtual lock
protecting context fields that should only be accessed from driver-
thread.  This should let the compiler help us detect problems where
ctx is used unsafely from things that could be called by the fe/st
thread.

This does end up sprinkled far and wide, it would be nice if the
compiler could be a bit smarter about understanding call-graphs
(at least with static fxns), but at least it makes it clear where
things are called from which thread.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9061>
2021-02-18 00:27:49 +00:00
Eric Anholt 75a4386676 freedreno: Make sure that queries are disabled during shadow blits.
They shouldn't count toward occlusion queries.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8789>
2021-02-04 19:31:30 +00:00
Rob Clark 03c28278a7 freedreno: Add perf_warn() for missed UBWC opportunities
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>
2021-01-23 09:57:48 -08:00
Rob Clark 38b073ebfe freedreno: Add fmt/args macros for pipe_resource
Deduplicate some long debug prints before adding more.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8659>
2021-01-23 09:57:48 -08:00
Eric Anholt 239bae7b94 freedreno: Fix warning about uninit size for the size==0 special case.
The size==0 query case would have just used uninitialized stack data for
sizing its BOs.

Fixes: 536ec9d7f5 ("freedreno: Refactor fd_resource_create_with_modifiers() into a helper")
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7664>
2020-11-18 18:15:02 +00:00
Rob Clark 57a2a5db81 freedreno: Drop fd_context_lock() and friends
These were actually just wrappers for the screen->lock, left over from
moving things around a long time ago.  Lets drop them to make things
more explicit (that we are locking the screen, not the context).

Involves a bit of shuffling things around to untangle header deps, but
no functional change.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7342>
2020-11-10 17:58:44 +00:00
Connor Abbott aa2f6bd4f5 freedreno: Use freedreno_dev_info
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7385>
2020-11-02 18:07:05 +00:00
Rob Clark 67238f95b5 freedreno: Disallow tiled if SHARED and not QCOM_COMPRESSED
If the user is not aware of modifiers, and wants to allocate a shared
resource, we shouldn't leave them with tiled.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3678
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7308>
2020-10-26 14:36:24 +00:00
Rob Clark 63bdbfd045 freedreno: Update import/export traces
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7308>
2020-10-26 14:36:24 +00:00
Vinson Lee 0a7bd14dbb freedreno: Move rsc NULL check to before rsc dereferences.
Fix defect reported by Coverity Scan.

Dereference before null check (REVERSE_INULL)
check_after_deref: Null-checking rsc suggests that it may be
null, but it has already been dereferenced on all paths leading
to the check.

Fixes: 6173cc19c4 ("freedreno: gallium driver for adreno")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6903>
2020-10-03 03:39:30 +00:00
Rob Clark 16a3a56321 freedreno: Don't leak LRZ bo's
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6900>
2020-09-29 20:56:54 +00:00
Marek Olšák 22253e6b65 gallium: rename PIPE_TRANSFER_* -> PIPE_MAP_*
Acked-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5749>
2020-09-22 03:20:54 +00:00
Rob Clark aae1e68637 freedreno: Fix missing rsc->seqno updates
There were a couple paths where we weren't getting valid seqno's, which
are supposed to be updated whenever the backing bo is set/changed.  So
wrap that up in a helper to make it harder to mess up.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6575>
2020-09-03 00:06:36 +00:00
Rob Clark 8d1d0c126f freedreno/a6xx: move ubwc clear to blitter
Avoid having to mmap() unnecessarily by moving UBWC clear/init to
blitter.

Because we don't have a context when the bo is allocated, we need to
lazily initialize UBWC data, so hook into the resource_written()
tracking to do this.  Don't bother with resource_read() because that
would be undefined anyways.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6475>
2020-08-28 22:24:25 +00:00
Rob Clark 1fa43a4a8e freedreno: handle case of shadowing current render target
If you have a sequence where there is a single buffer associated with
the current render target, and then you end up shadowing it on the 3d
pipe (u_blitter), because of how we swap the new shadow and rsc before
the back-blit, you could end up confusing things into thinking that
the blitters framebuffer state is the same as the current framebuffer
state.

Re-organizing the sequence to swap after the blit is complicated when
also having to deal with CPU memcpy blit path, and the batch/rsc
accounting.  So instead just detect this case and flush if we need to.

Fixes:
dEQP-GLES31.functional.stencil_texturing.render.depth24_stencil8_clear
dEQP-GLES31.functional.stencil_texturing.render.depth24_stencil8_draw

Cc: mesa-stable@lists.freedesktop.org
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6434>
2020-08-24 20:15:23 +00:00
Rob Clark 743ad7bf6b freedreno: add debug helper to dump buffers
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6434>
2020-08-24 20:15:23 +00:00