The unconditional flushing performed by
iris_flush_and_dirty_for_history() is now redundant with the memory
barriers introduced previously in this series, which should be in a
better position to determine from which domain the buffer will
actually be used in the future, and whether an additional flush or
invalidation is required or redundant with other PIPE_CONTROL commands
emitted elsewhere.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15738>
Fast clear with the resource format instead. This is safe to do because
can_fast_clear_color ensures that the clear color generates the same
pixel with either the view format or the resource format.
On SKL, this prevents us from using an invalid surface state. This platform
doesn't support CCS_E with sRGB formats, but prior to this patch we allowed
fast-clearing with this combination. Piglit's fcc-write-after-clear test
can trigger this.
Fixes: 230952c210 ("iris: Don't support sRGB + Y_TILED_CCS on gen9")
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14806>
This makes blits, copies, and (non-fast) clears set the appropriate
BLORP_BATCH_USE_{COMPUTE,BLITTER} flag if their batch is either
IRIS_BATCH_COMPUTE or IRIS_BATCH_BLITTER. We ignore the other
operations for now as those don't support compute or blit yet.
Of course, there is no code to attempt to launch BLORP operations on
either the compute or blitter batches yet, but that will come in time.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14687>
A couple things happen as a result of this:
1. This function is more consistent. It aims to clamp clear color ranges
to format-dependent values, but it didn't clamp the upper ranges of
10-bit and 11-bit floats (64512 and 65024 respectively). Those are
handled now.
2. The clear colors are quantized. The quantization method seems to
differ from what the HW uses for slow (and obviously fast) clears.
Due to this change in quantization method, iris starts failing
dEQP-EGL.functional.image.modify.renderbuffer_rgba4_renderbuffer_clear_color.
That's okay however. That test was removed from the mustpass list
because of threshold issues (see egl-test-issues.txt).
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7740>
INTEL_DEBUG is defined (since 4015e1876a) as:
#define INTEL_DEBUG __builtin_expect(intel_debug, 0)
which unfortunately chops off upper 32 bits from intel_debug
on platforms where sizeof(long) != sizeof(uint64_t) because
__builtin_expect is defined only for the long type.
Fix this by changing the definition of INTEL_DEBUG to be function-like
macro with "flags" argument. New definition returns 0 or 1 when
any of the flags match.
Most of the changes in this commit were generated using:
for c in `git grep INTEL_DEBUG | grep "&" | grep -v i915 | awk -F: '{print $1}' | sort | uniq`; do
perl -pi -e "s/INTEL_DEBUG & ([A-Z0-9a-z_]+)/INTEL_DBG(\1)/" $c
perl -pi -e "s/INTEL_DEBUG & (\([A-Z0-9_ |]+\))/INTEL_DBG\1/" $c
done
but it didn't handle all cases and required minor cleanups (like removal
of round brackets which were not needed anymore).
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13334>
Instead of doing a tile cache flush after slow clears or when the clear
value changes, do it before every fast clear of a HIZ_CCS_WT surface.
This agrees with the Bspec.
Fixes: c85ea824bc ("iris: reduce redundant tile cache flushes")
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11539>
This allows us to delete iris_resource_unfinished_aux_import, which
incorrectly assumed that a CCS-enabled resource needs an aux BO.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12795>
Previously, when importing a resource with modifiers that include
clear color as auxiliary data, we were mapping the clear color BO
on the CPU in order to set res->aux.clear_color to the value stored
there.
We are generally trying to avoid CPU mapping imported buffers, because
in hybrid setups, they could be from a different device, and may not be
mappable. So we'd like to avoid that here.
This CPU-side knowledge of the clear color is only used in a few cases:
1. Avoiding a resolve due to a partial clear with a changing clear color
2. Avoiding disabling CCS when rendering to a texture view of a resource
where the only format difference is sRGB vs. linear and the clear
color is 0/1.
Instead of mapping the clear color BO on the CPU, we instead set a flag
indicating that we don't know the clear color, and reset it to known on
our first clear.
In the first case, the first partial fast clear of a resource would eat
an extra resolve (there is no penalty if we clear the whole resource).
The second case doesn't seem that critical, as it's someone rendering
to an imported BO - when the common case is to texture from it.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11888>
Many places need to know the maximum or minimum possible value for a
given size integer... so everyone just open-codes their favorite
version. There is some potential to hit either undefined or
implementation-defined behavior, so having one version that Just Works
seems beneficial.
v2: Fix copy-and-pasted bug (INT64_MAX instead of INT64_MIN) in
u_intmin. Noticed by CI. Lol. Rename functions
`s/u_(uint|int)(min|max)/u_\1N_\2/g`. Suggested by Jason. Add some
unit tests that would have caught the copy-and-paste bug before wasting
CI time. Change the implementation of u_intN_min to use the same
pattern as stdint.h. This avoids the integer division. Noticed by
Jason.
v3: Add changes to convert_clear_color
(src/gallium/drivers/iris/iris_clear.c). Suggested by Nanley.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Suggested-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12177>
We are flushing tile cache more often than is necessary. In
unified cache mode, tile cache flushing is expensive, evicting all
depth/pixel data from the L3$. This is only need for a handful of
cases, such as: making cpu or gpu changes globally visible
(e.g. map), fast color clears, or slow depth clears. Tile cache
flushing is a gen12+ feature.
Remove blanket flushing of tile cache on all depth/RT flushes.
Replace with selective tile cache flushing.
Improves performance in several workloads:
AztecRuins.ogl-high-offscreen-1440p 1%
UnigineValley.ogl-g2 1%
Dota 2 (replay Jul 2020).ogl-g2 1%
Counter-Strike GO.ogl-g2 1%
Manhattan.ogl-Off-19x10 2%
CarChase.ogl-Off-19x10 1%
Bioshock Infinite.ogl-g2 1%
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10217>
u_threaded_context requires various objects to inherit from a new
threaded_foo base class rather than directly from pipe_foo. This
patch does most of the mechanical changes required for that.
It also initializes the new threaded_resource fields.
Reviewed-by: Zoltán Böszörményi <zboszor@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8964>
Instead of storing a bitfield of which resource levels will be accessed
with HiZ, compute this information on-demand. Makes the iris_resource
struct more generic.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8853>
Before this commit, various functions avoided calling
iris_resource_get_aux_state and iris_resource_set_aux_state for depth
buffers which lacked full HiZ support at certain levels. This was
because:
1. Some callers of prepare/finish neglected to use ISL_AUX_USAGE_NONE
for the levels which lacked full HiZ support.
2. The assertions within the getter and setter were too strict.
Now that both of these issues have been resolved, there's no obvious
reason to try to avoid these function calls. Delete the code for
avoiding them.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8853>
Set dst_aux_usage to ISL_AUX_USAGE_NONE for the depth buffer blit if the
level doesn't fully support HiZ. Enables removing the code that masked
this issue later on.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8853>
For color buffers, conditional fast clears can cause aux-state tracking
to lose information necessary for resolves later on.
For depth buffers, they never actually worked because they occurred
unconditionally. Even if they were conditional, they would suffer from
the same issues as color buffers.
Enables iris to pass the nv_conditional_render-clear-bug piglit test.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3565
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7762>
This function has a number of problems:
1. It performs 24-bit quantization on a format which shouldn't be
quantized at all (PIPE_FORMAT_Z32_FLOAT_S8X24_UINT).
2. The algorithm seems to create a different pixel than HW would in the
absence of this SW conversion. This can cause issues with depth
testing.
Instead of adding more code to deal with these issues, delete the
quantization code.
This code originated from i965
(0ae9ce0f29) and helped to avoid a
regression in Lightsmark 2008. This change continues to avoid that
regression because any new clear value is now casted from double to
float before checking if the resource's clear value has changed.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3783
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7665>
Until recently, the depth value from glClearBufferfv wasn't clamped.
Before then, this patch enabled the driver to fail the clearbuffer-depth
piglit test with INTEL_DEBUG=nofc. This is because convert_depth_value
relies on the assumption that the depth value is clamped.
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7410>
Currently, every modifier that uses CCS also lacks support for
fast-clears. On gen9+, dmabufs may gain fast-cleared blocks through
clear calls. On gen12, fast-clearing can occur during any rendering
operation. Mark when dmabufs gain fast-cleared blocks and flush them
during a context flush operation.
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3425
Tested-by: Simon Ser <contact@emersion.fr>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7384>
Resources with alpha formats that are mapped to R are fast-cleared with
the wrong clear color.
When such resources with are cleared via iris_clear_texture,
isl_color_value_unpack places channel data in the R channel.
convert_fast_clear_color then overwrites the channel with 0.
To avoid zeroing the clear color, move convert_fast_clear_color to the
other callers of clear_color: iris_clear and iris_clear_render_target.
Enables iris to pass the "A" case of the fcc-clear-tex piglit test.
v2. Rename convert_fast_clear_color to convert_clear_color. (Ken)
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3670
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7345>
Don't assert that the size of every channel is greater than zero. This
assert doesn't work for integer formats with less than 4 channels.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7345>
Resources with luminance alpha formats that are mapped to RG are
fast-cleared with the wrong clear color.
When such resources with are cleared via iris_clear_texture,
isl_color_value_unpack places channel data in the R and G channels.
convert_fast_clear_color then overwrites the G channel with R.
Delete the clear color override that's specific to luminance alpha
formats.
Enables iris to pass the "LA" case of the fcc-clear-tex piglit test.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7345>
For 8bpp surfaces on TGL, prevent LOD1+ from being fast-cleared. This
will be relevant once ISL starts allowing CCS for 8bpp surfaces with
more than 2 miplevels. I verified the problem behind this restriction
with a modified version of the fbo-clearmipmap piglit test.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7085>
When support for multi-slice fast-clears was introduced for color
surfaces, an existing optimization for skipping fast-clears was not
updated (this optimization assumed single-slice fast-clears). As a
result, the driver began to skip multi-layer fast-clears if just the
first slice was in the CLEAR state (ignoring the state of the others).
A Civilization VI trace was the only workload I found to make use of
this optimization and it did so for 2D, non-array textures. Therefore,
this fix simply checks that the depth of the clear box is 1. It also
moves the single-slice aux-state query closer to the optimization to
clarify the need for the depth check.
Enables iris to pass a case of the fcc-write-after-clear piglit test,
[fast-clear tracking across layers 0 -> 1 -> (0,1)].
Fixes: 393f659ed8 ("iris: Enable fast clears on other miplevels and layers than 0.")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6973>
If we clear depth only texture via glClearTex(Sub)Image it may cause:
../src/intel/blorp/blorp_genX_exec.h:1554: blorp_emit_surface_states: Assertion `params->depth.enabled || params->stencil.enabled' failed.
due to clear_depth_stencil calling blorp_clear_depth_stencil when
depth is already fast-cleared and there is no stencil.
Fixes piglit test: arb_clear_texture-depth
Fixes: 51638cf18a
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5770>
For texturing and draw calls, HW expects the clear color to be in two
different color spaces after sRGB fast-clears - sRGB in the former and
linear in the latter. Up until now, iris has stored the clear color in
the sRGB color space. Limit the allowable clear colors for sRGB
fast-clears to 0/1 so that both color space requirements are satisfied.
Makes iris pass the sRGB -> sRGB subtest of the fcc-write-after-clear
piglit test on gen9+.
v2:
* Drop iris_context::blend_enables. (Ken)
* Drop some more resolve-related blend-state-tracking code.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4972>
For rendering operations, avoid adding or using fast-cleared blocks if
the render format is incompatible with the clear color interpretation.
Note that the clear color is currently interpreted through the
resource's surface format.
Makes iris pass subtests of the fcc-write-after-clear piglit test:
* UNORM -> SNORM, partial block on gen8+.
* linear -> sRGB, partial block on gen9+.
* UNORM -> SNORM, full block on gen12.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4972>
Dylan Baker
Tapani Pälli
Francisco Jerez
Nanley Chery
Kenneth Graunke
Dave Airlie
Marcin Ślusarz
Jordan Justen
Ian Romanick
Anuj Phogat
Felix DeGrood
Vinson Lee
Andrii Simiklit
Caleb Callaway
Eric Anholt
Danylo Piliaiev