This eliminates the need to use container_of in error handling code.
INTEL_MEASURE will need to access the iris context from each batch.
suggested-by: Kenneth Graunke <kenneth@whitecape.org>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7354>
Now that we store shader variants in the objects themselves rather
than a per-context hash table, they are actually global across
contexts. We can enable this feature.
This makes shaders shared across contexts, so apps can compile in
one and use it in another. This has always been allowed by GL,
but in the past we've simply recompiled the shaders in every context,
which is slow and painful.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7668>
We've traditionally stored shader variants in a per-context hash table,
based on a key with many per-stage fields. On older hardware supported
by i965, there were potentially quite a few variants, as many features
had to be emulated in shaders, including things like texture swizzling.
However, on the modern hardware targeted by iris, our NOS dependencies
are much smaller. We almost always guess the correct state when doing
the initial precompile, and so we have maybe 1-3 variants. iris NOS
keys are also dramatically smaller (4 to 24 bytes) than i965's.
Unlike the classic world, Gallium also provides a single kind of object
for API shaders---pipe_shader_state aka iris_uncompiled_shader. We can
simply store a list of shader variants there. This makes it possible
to access shader variants across contexts, rather than compiling them
separately for each context, which better matches how the APIs work.
To look up variants, we simply walk the list and memcmp the keys.
Since the list is almost always singular (and rarely ever long),
and the keys are tiny, this should be quite low overhead.
We continue storing internally generated shaders for BLORP and
passthrough TCS in the per-context hash table, as they don't have
an associated pipe_shader_state / iris_uncompiled_shader object.
(There can also be many BLORP shaders, and the blit keys are large,
so having a hash table rather than a list makes sense there.)
Because iris_uncompiled_shaders are shared across multiple contexts,
we do require locking when accessing this list. Fortunately, this
is a per-shader lock, rather than a global one. Additionally, since
we only append variants to the list, and generate the first one at
precompile time (while only one context has the uncompiled shader),
we can assume that it is safe to access that first entry without
locking the list. This means that we only have to lock when we
have multiple variants, which is relatively uncommon.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7668>
There is a small gap of time where the currently bound uncompiled
shaders, and compiled shader variant, are out of sync. Specifically,
between pipe->bind_*_state() and the next draw.
Currently, shaders variants live entirely within a single context,
and when deleting an iris_uncompiled_shader, we check if any of its
variants are currently bound, and defer deleting those until the next
iris_update_compiled_shaders() hook runs and binds new shaders to
replace them. (This is due to the time gap between binding new
uncompiled shaders, and updating variants at draw time when we have
the required NOS in place.)
This works pretty well in a single context world. But as we move to
share compiled shader variants across multiple contexts, it breaks down.
When deleting a shader, we can't look at all contexts to see if its
variants are bound anywhere. We can't even quantify whether those
contexts will run a future draw any time soon, to update and unbind.
One fairly crazy solution would be to delete the variants anyway, and
leave the stale pointers to dead variants in place. This requires
removing any code that compares old and new variants. Today, we do
that sometimes for seeing if the old/new shaders toggled some feature.
Worse than that, though, we don't just have to avoid dereferences, we'd
have to avoid pointer comparisons. If we free a variant, and quickly
allocate a new variant, malloc may return the same pointer. If it's
for the same shader stage, we may get a new different program that has
the same pointer as a previously bound stale one, causing us to think
nothing had changed when we really needed to do updates. Again, this
is doable, but leaves the code fragile - we'd have to guard against
future patches adding such checks back in.
So, don't do that. Instead, do basic reference counting. When a
variant is bound in a context, up the reference. When it's unbound,
decrement it. When it hits zero, we know it's not bound anywhere and
is safe to delete, with no stale references. This ends up being
reasonably cheap anyway, since the atomic is usually uncontested.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7668>
Now that we're looking at shader info system values rather than
vs_prog_data, there's no reason we have to do this when updating
the shader variants. We can simply check it when binding a new
shader from the API point of view.
Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8759>
brw_compile_vs sets the vs_prog_data fields based on the NIR program's
system values read info field. We can use that directly, enabling more
cleanups in the next patches.
Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8759>
Commits 0278d1fa323cf1f289..b688ea31fcf7e20436 added a new parameter
to set_vertex_buffers(), set_shader_images(), and set_sampler_views()
which specifies a number of trailing slots to unbind. They updated
the iris functions to do the unbinding, but didn't update the code
to mark which things are bound in the bitfields. This meant that
later code would assume those unbound slots were bound, and crash
on a NULL dereference. All that's needed is to add that slot count
when unbinding things in the bitfield.
Fixes: 0278d1fa32 ("gallium: add unbind_num_trailing_slots to set_vertex_buffers")
Fixes: 72ff66c3d7 ("gallium: add unbind_num_trailing_slots to set_shader_images")
Fixes: b688ea31fc ("gallium: add unbind_num_trailing_slots to set_sampler_views")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8758>
There are a few places (mainly u_threaded_context) that do:
set_vertex_buffers(...);
for (i = 0; i < count; i++)
pipe_resource_reference(&buffers[i].resource.buffer, NULL);
set_vertex_buffers increments the reference counts while the loop
decrements them.
This commit eliminates those reference count changes by adding a parameter
into set_vertex_buffers that tells the callee to accept all buffers
without incrementing the reference counts.
AMD Zen benefits from this because it has slow atomics if they come from
different CCXs.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8298>
Instead of calling this functions again to unbind trailing slots,
extend it to do it when binding. This reduces CPU overhead.
A lot of drivers ignore "start" and always unbind all slots after "count".
Such drivers don't need any changes here.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8298>
Instead of calling this function again to unbind trailing slots,
extend it to do it when images are being set. This reduces CPU overhead.
Only st/mesa benefits.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8298>
Instead of calling this functions again to unbind trailing slots,
extend it to do it as part of the call that sets vertex buffers.
This reduces CPU overhead. Only st/mesa benefits from this.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8298>
We often do this:
pipe->set_constant_buffer(pipe, shader, slot, &cb);
pipe_resource_reference(&cb->buffer, NULL);
That results in atomic increment in set_constant_buffer followed by
atomic decrement after set_constant_buffer. This new interface
eliminates those atomics.
For the case above, this should be used instead:
pipe->set_constant_buffer(pipe, shader, slot, true, &cb);
cb->buffer = NULL; // if cb is not a local variable, else do nothing
AMD Zen benefits from this. The perf improvement is ~3% for Viewperf13/Catia.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8298>
Instead of looping over all stages and re-accessing prog_data at the
end, we can just move the URB size check into iris_update_compiled_xs,
in the new != old block, where we know the shader changed - plus where
we're disabling tessellation. We already have the prog_data handy in
these cases, with a bit less pointer chasing.
Improves performance in Piglit's drawoverhead microbenchmark
(#63: DrawArrays, no state change) by 4.07856% +/- 0.540517% (n=850).
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8721>
Reconfiguring the URB partitioning is likely to cause shader stalls,
as the dividing line between each stage's section of memory is moving.
(Technically, 3DSTATE_URB_* are pipelined commands, but that mostly
means that the command streamer doesn't need to stall.) So it should
be beneficial to update the URB configuration less often.
If the previous URB configuration already has enough space for our
current shader's needs, we can just continue using it, assuming we
are able to allocate the maximum number of URB entries per stage.
However, if we ran out of URB space and had to limit the number of
URB entrties for a stage, and the per-entry size is larger than we
need, we should reconfigure it to try and improve concurrency.
So, we begin tracking the last URB configuration in the context,
and compare against that when updating shader variants.
Cuts 36% of the URB reconfigurations (excluding BLORP) from a
Shadow of Mordor trace, and 46% from a GFXBench Manhattan 3.0 trace.
One nice thing is that this removes the need to look at the old
prog_data when updating shaders, which should make it possible to
unbind shader variants without causing spurious URB updates.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8721>
When calculating a URB configuration, we start with a notion of how
much space each stage /wants/ (to achieve the maximum amount of
concurrency), but sometimes fall back to giving it less than that,
because we don't have enough space. (Typically, this happens when
the per-stage size is large, or there are many stages, or both.)
We now output a "constrained" boolean which is true if we weren't
able to satisfy all the "wants" due to a lack of space.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8721>
Disable compression in iris_flush_resource if the resource lacks a
modifier. When a caller wants to prepare such a resource for sharing
(via eglCreateImage for example), this change enables all reference
holders to access the resource in a common manner - without compression.
This fixes misrendering with 3D-accelerated qemu. A piglit test which
reproduces qemu's behavior, ext_image_dma_buf_import-export-tex, is also
enabled to pass.
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2678
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8663>
It is currently a bitset on top of a uint64_t but there are already
more than 64 values. Change to use BITSET to cover all the
SYSTEM_VALUE_MAX bits.
Cc: mesa-stable
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8585>
According to the BSpec page for MEDIA_VFE_STATE, on Gen12 platforms
"if a fused configuration has fewer threads than the native POR
configuration, the scratch space allocation is based on the number of
threads in the base native POR configuration". However we currently
use the subslice count from devinfo->num_subslices[0], which only
includes the subslices currently enabled by the platform fusing. This
leads to scratch space underallocation and occasional hangs.
The problem is likely to affect most Gen12 GPUs with less than 96 EUs.
GFXBench5 Aztec Ruins is able to reproduce the issue fairly reliably.
Fixes: 9e5ce30da7 "intel: fix the gen 12 compute shader scratch IDs"
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8636>
I have never used this to debug anything in iris, and it's been years
since I even thought about using i965's similar functionality. I'm
planning to move a bunch of shaders out of the global hash table, at
which point it'll be much less useful. So, just drop it.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8634>
This tried to de-duplicate identical copies of the same shader
assembly, but in the least efficient way possible: it did a linear
walk through every shader in the entire context memcmp'ing the
final assembly (after going through the effort to compile it).
In the end, all it saved was space and number of BOs, not even
state changes.
This optimization has been mostly replaced by st/mesa's cache
mechanism, which looks for multiple shaders that compile to the
same NIR and go further than this did, and actually reuse the
same pipe shader state. That's even more efficient than this.
This seems to still trigger some times, because the NIR that
st/mesa hashes hasn't quite been finalized and stripped. But
it would be better to improve that, not this.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8634>
The intention of IRIS_DIRTY_{RENDER,COMPUTE}_RESOLVES_AND_FLUSHES
is to avoid considering resolves/flushes on back to back draw calls
where nothing of significance has changed with the resources. When
anything changes that could require a resolve, we must flag those.
Those situations are:
1. Texture/image/framebuffer bindings change
(as the set of images we need to look at is now different)
2. Depth writes are enabled/disabled (the resolve code uses this)
3. The aux state for a currently bound resource changes.
We were missing this last case. In particular, one example where
we missed this was:
1. Bind a texture.
2. Clear that texture (likely blits/copies/teximage would work too)
3. Draw and sample from that texture
Clear-then-Bind would work, as binding would flag resolves as dirty.
But Bind-then-Clear doesn't work, as clear can change the aux state
of the bound texture, but wasn't flagging that anything had changed.
Technically, we could consider whether the resource whose aux state
is changing is bound for compute (and only flag COMPUTE_RESOLVES),
or bound for a 3D stage (and only flag RENDER_RESOLVES), and flag
nothing at all if it isn't bound. But we don't track that well,
and it probably isn't worth bothering. So, flag unconditionally
for now.
This does not appear to impact Piglit's drawoverhead scores.
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3994
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4019
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8603>
For color buffers, conditional fast clears can cause aux-state tracking
to lose information necessary for resolves later on.
For depth buffers, they never actually worked because they occurred
unconditionally. Even if they were conditional, they would suffer from
the same issues as color buffers.
Enables iris to pass the nv_conditional_render-clear-bug piglit test.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3565
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7762>
Apart from an issue with fast clears that will be addressed soon,
aux-state tracking with conditional rendering works because the
aux-state info needed for performing required resolves is never lost.
Add comments explaining how this works. Assertions are omitted to avoid
having to pass render_condition variables into
iris_resource_prepare_access and iris_resource_finish_write.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7762>
In order to handle blitting the stencil aspect of a depth-stencil
resource, use aspect-specific pipe formats in the aspect_mask loop.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8340>
When blitting just the stencil aspect, the source and destination
resources are prepared/setup twice. Move the unconditional resource
setup into the aspect_mask loop to avoid this.
In addition, use the aspect provided by the loop instead of the mask
provided by the info parameter.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8340>
Use iris_resource_prepare_texture in iris_blit to avoid partial resolves
for sRGB <-> linear texture views. This affects a trace of L4D2.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8340>
Instead of having a depth-specific assertion in a generic portion of
iris_blit, move it into the depth-specific cases of
iris_resource_texture_aux_usage. Since iris_blit calls that function,
the test still occurs.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8340>
Before this change, iris_blit would prepare the depth buffer in a
depth-stencil resource even when only the stencil aspect was used for the
blit. Use the aspect mask to prepare the correct resource.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8340>
Drivers aren't allowed to ignore start with user index buffers anymore.
This is required by the new fast path where mesa/main is using pipe_draw_info.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7679>
To remove PIPE_CAP checking in the common code.
It's better if drivers lower multi draws even if the hardware doesn't
support it beause the multi draw loop can be moved deeper into the driver
to remove more overhead.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7679>
Commit 7779b1d71b, disabled clear support
when copying to/from color buffers. According to the performance CI, it
falls within a range of commits that introduced a performance regression
on Bioshock Infinite with Tigerlake. Icelake isn't noticeably affected.
By analyzing a trace of the game, I found a couple cases where that
commit added new partial resolves. Update get_copy_region_aux_settings
to avoid them:
- The trace uploads to R8_UNORM textures. On TGL, these enter the
COMPRESSED_CLEAR state on the upload and are partially resolved before
every subsequent upload. Thankfully, they keep their initial clear
color of all zeroes. Since zeros can survive format reinterpretation,
allow clear support for it.
- The trace copies between RGBA16_FLOAT textures. The ones with zero
clear color are helped by the optimization above. The ones with
non-zero clear color are used as source textures. Thankfully on ICL+,
the clear color used for sampling is in pixel form and can thus be
sampled from with format reinterpretation. Allow clear support for
this case.
I haven't tested the actual performance impact of this change, but it
should be beneficial regardless.
Reported-by: Clayton Craft <clayton.a.craft@intel.com>
Reported-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8262>
Mark Janes
Kenneth Graunke
Marek Olšák
Nanley Chery
Caio Marcelo de Oliveira Filho
Francisco Jerez
Jordan Justen
Yevhenii Kolesnikov
Jason Ekstrand
Christian Gmeiner
Dave Airlie