It's meaningful for this intrinsic and so does not add noise to the
lowering pass.
(Although dual-source writes must be to RT 0, depth and stencil
writes, which store_combined_output_pan is also used for, can still be
done with MRT enabled.)
Fixes: 5c168f09eb ("nir: Eliminate store_combined_output_pan BASE")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16685>
Looks like MSVC doesn't like VLAs:
src/compiler/spirv/spirv_to_nir.c(3879): error C2057: expected constant expression
src/compiler/spirv/spirv_to_nir.c(3879): error C2466: cannot allocate an array of constant size 0
src/compiler/spirv/spirv_to_nir.c(3879): error C2133: 'srcs': unknown size
so let's use a static array size.
Fixes: 87d7431198 ("spirv: Use nir_vec_scalars() to simplify matrix transpose.")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16632>
Even with that alone we can't pass the test, as LLVM enables some
extensions based on the SPIR target we choose.
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16479>
This relies too much on the properties of the SPIRV-LLVM-Translator and is
required to load SPIR-Vs found in the OpenCL CTS.
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16479>
This is controlled by spirv_to_nir_options.relaxed_precision_alu, because
some drivers don't want it.
This gets us mostly 16-bit math on turnip in vk-5-normal.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16465>
This should make it clear whether a validation failure happens in RADV or
zink.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12448>
If a user wants to skip printing the shader if no changes were made
without declaring a dummy variable for the progress.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12448>
Some older drivers don't support GLSL versions with the concept of flat
varyings and also don't support integers. Here we add a new setting to
make sure we don't use the optimisation that sets varyings to flat.
This setting helps us avoid marking varyings as flat and therefore
potentially having them changed to ints via varying packing.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6500
Fixes: 7647023f3b ("glsl: enable the use of the nir based varying linker")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16573>
If this == NULL, we'll get a crash which is pretty much the same thing
when it comes to ease of debugging. This fixes a giant pile of warnings
with GCC 12 of the form:
../src/compiler/glsl/ir.h: In member function ‘ir_dereference* ir_instruction::as_dereference()’:
../src/util/macros.h:149:30: warning: ‘nonnull’ argument ‘this’ compared to NULL [-Wnonnull-compare]
149 | #define assume(expr) ((expr) ? ((void) 0) \
| ~~~~~~~~^~~~~~~~~~~~~~
150 | : (assert(!"assumption failed"), \
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
151 | __builtin_unreachable()))
| ~~~~~~~~~~~~~~~~~~~~~~~~~
../src/compiler/glsl/ir.h:150:7: note: in expansion of macro ‘assume’
150 | assume(this != NULL); \
| ^~~~~~
../src/compiler/glsl/ir.h:160:4: note: in expansion of macro ‘AS_BASE’
160 | AS_BASE(dereference)
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16558>
This will allow us to disable the GLSL IR loop unroller in a
following patch and rely on the NIR loop unroller instead.
This allows the piglit test spec@!opengl 2.0@max-samplers border
to pass on the v3d rpi4 driver.
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16543>
Some drivers don't support these indirects and therefore require
loop unrolling if a shader uses a loop induction variable to
access a sampler array.
Here we add a new nir shader compiler option that drivers can set,
this will be the equivalent of the EmitNoIndirectSampler setting
used in the GLSL IR unrolling pass.
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16543>
The single caller of this function (in st_glsl_to_ir.cpp) always passes
false, so this is dead code.
v2: Delete convert_vec_index_to_cond_assign method because all the
callers are deleted too.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16440>
As of ca63a5ed3e ("glsl: fix interpolateAtXxx(some_vec[idx], ...) with
dynamic idx"), this lowering pass does two things. It converts
ir_binop_vector_extract to an if-ladder to select the dynamically
indexed component, and it extracts a ir_binop_vector_extract from the
source of an interpolateAt function and applies to the result instead.
This change adds a flag to disable the former behavior. The latter is
still useful, but NIR has better (and soon even better) ways of doing
the former.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16440>
This was originally part of a series that made other changes to this
file, but all of those changes got dropped. Since the typing was
already done, there's no reason to not fix the formatting.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16440>
v2: Don't lower ffloor@64 to ffract@64 when both ops are
to be lowered. Settle on ffloor in opt_algebraic because
in can be lowered to other ops in lower_double_ops.
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>(v1)
Jason Ekstrand <jason.ekstrand@collabora.com> (v1)
Reviewed-by: Emma Anholt <emma@anholt.net> (v1)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16431>
With a NIR based linker we get better xfb packing, and we no longer
depend on the GLSL IR optimisations to be able to link shaders
with a large amount of dead input/outputs.
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15731>
This is essentially the old GLSL IR packing pass rewritten as a
NIR based pass.
Doing this packing in NIR after we have preformed NIRs
optimisation passes can give us better packing results.
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15731>
These varyings cannot be packed by the GLSL linkers packing pass
so we need to skip this lowering until later when we can properly
handle them.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15731>
Unlike spirv glsl varyings might not have explicit locations set.
nir_shader_gather_info() was once only called at the end of linking
but these days it even gets called in NIR optimisation loops via
nir_opt_phi_precision.
In the following patches we implement a NIR version of the GLSL
varying linker which means we will have varyings with no location
set when nir_shader_gather_info() gets called the first few times,
and temp values set only for the purpose of removing unmatched
varyings between shaders for some calls after that.
Here rather than asserting we simply abort the io info gathering,
when we hit these values.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15731>
These will be used by a new nir based glsl varying linker that will
add varyings directly to the list before the are packed and we lose
the information.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15731>
This will be shared by a new NIR varying linking pass in following patches
but probably fits better here anyway considering its also used by
shader_query.cpp
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15731>
This version takes a set of variables and totally lowers indirects on
any variable in the set. We also rewrite the builtin_uniform version to
use the new helper internally.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16482>
Remove the conditional break statements associated with all
terminators that are associated with a fixed iteration count,
except for the one associated with the limiting terminator.
This logic matches similiar functionality that exists in the
old GLSL IR unrolling code.
This change helps a piglit test pass on the r300 driver once
we switch off the old GLSL IR unrolling code.
Shader-db results IRIS (BDW):
total instructions in shared programs: 17538619 -> 17538595 (<.01%)
instructions in affected programs: 216 -> 192 (-11.11%)
helped: 3
HURT: 0
helped stats (abs) min: 7 max: 10 x̄: 8.00 x̃: 7
helped stats (rel) min: 10.00% max: 12.07% x̄: 11.38% x̃: 12.07%
total cycles in shared programs: 858674910 -> 858672810 (<.01%)
cycles in affected programs: 79540 -> 77440 (-2.64%)
helped: 3
HURT: 0
helped stats (abs) min: 620 max: 800 x̄: 700.00 x̃: 680
helped stats (rel) min: 2.45% max: 2.83% x̄: 2.63% x̃: 2.62%
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16399>
Previously we only cared if this was set for the limiting
terminator. However in the following patch we will make use of this
information on other terminators to decide if we can eliminate them.
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16399>
Task shaders store their output payload to VRAM where mesh
shaders read from. There are two ring buffers:
1. Draw ring: this is where mesh dispatch sizes and
the ready bit are stored.
2. Payload ring: this is where the optional payload
is stored (up to 16K per task workgroup).
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14929>
Otherwise, it would mutate `fneg(fadd(-0, 0))` into `fadd(0, -0)` which
isn't correct since -0 + (+0) = +0 + (-0) = +0.
This fixes the OpenCL contraction tests on Iris.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16041>
This pattern pops up a bunch and the semantics of nir_channels() aren't
very convenient much of the time. Let's add a nir_trim_vector() which
matches nir_pad_vector().
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16309>
Because we pull the RT from the variable location and use that to look
up formats, we need a constant RT index. To deal with arrays (possibly
of arrays), we would either need to handle array derefs (we don't today)
or we need to require the variables to be split into one variable per
RT. Given that we have to lower indirect derefs anyway (to get constant
indices), we may as well require the client to split output variables by
calling nir_lower_io_arrays_to_elements_no_indirect().
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16309>
Some glcts tests have failed due to incorrect processing of `ir_quadop_vector` in
`ir_validation`. e.g:
`GLES31.functional.shaders.builtin_functions.integer.imulextended.int_highp_geometry`
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6461
Fixes: 23cde71b ("glsl: Stop lowering ir_quadop_vector.")
Reviewed-by: Emma Anholt <emma@anholt.net>
Signed-off-by: Mykhailo Skorokhodov <mykhailo.skorokhodov@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16420>
The comment above is correct, but the code to calculate the mask was broken.
No Foz-db changes outside of noise.
Fixes: 0e6581b87d ("nir/algebraic: Reassociate shift-by-constant of shift-by-constant")
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15990>
This isn't a hot path. We don't need to be manually using the
INSIDE_WORD version which will assert if we ever get a bigger texture
index.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15988>
Now that all consumers of GLSL use NIR, make the remaining drivers take
the path that relies on NIR to really do optimization.
nouveau steam shader-db runtime -6.69631% +/- 1.29235% (n=12).
No change on shader-db there.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16364>
This code just got duplicated a lot. There is still more, but the
remaining instances do a bit more than just removing other functions.
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16348>
Now that everybody goes through NIR, glsl_to_nir is happy to handle the
instruction and turn it into nir_op_vec4 instead of going to a temp
variable and back.
No changes on freedreno shader-db.
Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16363>
When moving code into the main block or loop blocks, put the code into
its own :
if(true) { ... }
block so that we avoid break/continue/return issues.
v2: Also take care of the main block with return instructions
v3: Make deletion more obvious with dummy if blocks (Jason)
v4: Fixup assert for loops (Lionel)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 8dfb240b1f ("nir: Add raytracing shader call lowering pass.")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
When moving code from below to the insertion cursor point, if the
cursor points to a jump instruction, don't bother inserting the code.
It would break the break/continue assumptions of NIR and would not be
executed anyway.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 8dfb240b1f ("nir: Add raytracing shader call lowering pass.")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
Stop using nop instructions which are causing issues with
break/continue, instead use a nir_cursor (which brings its share of
pain).
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 8dfb240b1f ("nir: Add raytracing shader call lowering pass.")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
This will be useful to cut code from one location and paste it at
another place and later keep pasting after the previous insertions.
v2: update comment (Jason)
deal with stiching 2 empty blocks (Jason)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
Now that everything goes through NIR, nir_opt_peephole_select has replaced
it.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8044>
Now that we have only GLSL->NIR as a path in the frontend, we can rely on
the NIR linking support.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8044>
All UBO-supporting drivers now go through the NIR path, which does a
better job of it.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8044>
From the GL 4.6 spec: "If the value of any non-ignored component of the
offset vector operand is outside implementation-dependent limits, the
results of the texture lookup are undefined." We shouldn't assertion
fail, then.
GLSL-to-NIR shouldn't be enforcing limits on TG4 offsets, since it doesn't
for non-TG4 tex_src_offset either. Leave it up to the driver to handle
it.
Fixes a crash in a piglit test on nouveau that supplies a negative random
number up to 10,000 as the first coordinate for some reason. Other NIR
drivers lowered TG4 explicit offsets to tex_src_offset, so they weren't
affected.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16261>
That's a read, not a write. Fixes optimizations getting disabled for fragment
shaders when linked with a shader producing transform feedback varyings.
Fixes: 85a723975b ("nir: add and gather shader_info::writes_memory")
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16285>
Avoids nir validation assertion failures, and it's not like backend
drivers would want to see definitely-out-of-bounds read/writes either.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16066>
Casts shouldn't change the bit pattern of the deref and you have to cast
again after you're done with the ALU anyway so we can ignore casts on
ALU sources. This means we can actually start constant folding NULL
checks even if there are annoying casts in the way.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15673>
This controls the whole lowering of "make tex ops with implicit
derivatives on non-implicit-derivative stages be tex ops with an explicit
lod of 0 instead", but it's really hard to describe that in a git commit
summary.
All existing callers get it added except:
- nir_to_tgsi which didn't want it.
- nouveau, which didn't want it (fixes regressions in shadowcube and
shadow2darray with NIR, since the shading languages don't expose txl of
those sampler types and thus it's not supported in HW)
- optional lowering passes in mesa/st (lower_rect, YUV lowering, etc)
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16156>
u_worklist is nir_block_worklist, suitably generalized. All we need to do is
define the macros to translate between the APIs.
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16046>
XFB varyings are considered as always active IO to prevent them to
be removed or compacted. Though, if the NIR linker doesn't mark XFB
varyings as unmoveable it still possible to remap other varyings to
the same location/component.
Fixes KHR-Single-GL46.enhanced_layouts.xfb_override_qualifiers_with_api
with Zink and a bunch of other dEQP XFB tests.
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6301
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16092>
Components need to be handled, otherwise if a shader has two XFB
varyings at the same location, only one will be marked as always active.
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16092>
pass_flags is only initialized for grouped loads, so change the order
Fixes: 33b4eb149e - nir: add new SSA instruction scheduler grouping loads into indirection groups
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16090>
NTT can't convert local 64 variables of type vec3 or vec4, therefore,
they need to be split into vec2 + double or vec2 + vec2.
At the same time deal splitting the phi nodes.
The pass goes into the global namespace because it is also useful
for r600.
v2: only lower function_temps (Emma) and handle array of arrays (Jason)
v3: - remove bool parameter in function to merge
vec3 and vec4
- simplify load/store_deref_(array|var)
- use a pointer hash table
- simplify PHI splitting (all Emma)
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15945>
util_cpu_detect is an anti-pattern: it relies on callers high up in the call
chain initializing a local implementation detail. As a real example, I added:
...a Mali compiler unit test
...that called bi_imm_f16() to construct an FP16 immediate
...that calls _mesa_float_to_half internally
...that calls util_get_cpu_caps internally, but only on x86_64!
...that relies on util_cpu_detect having been called before.
As a consequence, this unit test:
...crashes on x86_64 with USE_X86_64_ASM set
...passes on every other architecture
...works on my local arm64 workstation and on my test board
...failed CI which runs on x86_64
...needed to have a random util_cpu_detect() call sprinkled in.
This is a bad design decision. It pollutes the tree with magic, it causes
mysterious CI failures especially for non-x86_64 developers, and it is not
justified by a micro-optimization.
Instead, let's call util_cpu_detect directly from util_get_cpu_caps, avoiding
the footgun where it fails to be called. This cleans up Mesa's design,
simplifies the tree, and avoids a class of a (possibly platform-specific)
failures. To mitigate the added overhead, wrap it all in a (fast) atomic
load check and declare the whole thing as ATTRIBUTE_CONST so the
compiler will CSE calls to util_cpu_detect.
Co-authored-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15580>
The BITFIELD_MASK() macro is intended for using with actual bitfields,
not with nir_component_mask_t. This means we do some extra work to
handle values that are invalid for nir_component_mask_t in the first
place.
This eliminates some warnings on Clang, where the compiler complains
about casting UINT32_MAX to UINT16_MAX.
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15547>
Here we just make sure we match the interpolation type on both
sides of the shader interface. Drivers like d3d12 are expecting
this.
Fixes: 9401990e6f ("nir/linker: set varying from uniform as flat")
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16003>
It's quite likely that the source of the f2i32 was already an integer, in
which case we can skip the ftrunc (particularly useful on the int-to-float
class of hardware that's unlikely to just have a native trunc opcode!).
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15870>
Just like the other make-the-float-an-integer opcodes. Noticed in a
gallium nine shader run through TGSI-to-NIR, where the array index had
been floored by the user, but got implicitly rounded by DX9 array
indexing.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15870>
When we put NIR in the compiler stack for r300, indirect addressing broke
for gallium nine. DX's array indirects round the float value, so the DX
shader gets mapped to a TGSI "ARR ADDR[0] src.x" instruction. Translating
that to NIR maps to r0[f2i32(fround(src.x))]. While we might hope that in
translation back using nir-to-tgsi after optimization we would recognize
the construct and emit ARR again, that's going to be error prone (think
"what if src.x is in a NIR register?") so we need a fallback plan. r300
will be able to handle this lowering, so get it in place first to fix the
regression.
Fixes: #6297
Fixes: 7d2ea9b0ed ("r300: Request NIR shaders from mesa/st and use NIR-to-TGSI.")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15870>
This prevents more use-after-free errors. Passing them around using
std::unique_ptr ensures that the LLVMContext gets destroyed but doesn't
ensure destruction order. Declaring it on the stack ensures that the
context doesn't get destroyed until right before the the function
returns which is after any other LLVM stuff is destroyed.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Icecream95 <ixn@disroot.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15937>
First, separate out the LLVM context logging to make it take a
clc_logger instead of passing in a string stream. Currently, the LLVM
context may outlive the string stream which we assign which may lead to
use-after-free errors. Second, use a separate string stream for clang
diagnosticl logging which we intentionally declare before the compiler
so the compiler can't outlive it.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Icecream95 <ixn@disroot.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15937>
This just calls some of the LLVM init functions in a common place
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Icecream95 <ixn@disroot.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15937>
The type of the spv_position_t components can differ across platforms,
it's simpler to just let C++ overloading handle it.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15437>
These are the same as the normal ones, but they take an unsigned 32-bit
offset in BASE and another unsigned 32-bit offset in the last source.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14124>
Helpful when lost in a sea of NIR :)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15887>
Here we remove the outer arrays on geom and tess shaders where
needed. Without this the pass can sometimes attempt to pack a
varying on only one side of the shader interface where it is not
actually needed. The result can be mismatching varying types.
Fixes: d6b9202873 ("glsl: disable varying packing when its not safe")
Tested-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15761>
VK-GL-CTS causes tons of these due to a bug in glslang, to the point where
it's hard to find actual issues in test logs. Disable the warning for
now, with a link to the issue we're waiting on being resolved.
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15332>
Designated initializers require C++20, which is a bit easier said than
done to support well across meson versions. Let's avoid using them
for now instead.
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15706>
DG2 can only do sample_d and sample_d_c on 1D and 2D surfaces. Cube
maps and 3D surfaces were already handled, but 1D array and 2D array
surfaces were not.
Fixes the following Vulkan CTS failures on DG2:
dEQP-VK.glsl.texture_functions.texturegradclamp.isampler1darray_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.isampler2darray_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.sampler1darray_fixed_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.sampler1darray_float_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.sampler2darray_fixed_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.sampler2darray_float_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.usampler1darray_fragment
dEQP-VK.glsl.texture_functions.texturegradclamp.usampler2darray_fragment
The Fixes: tag below is a bit misleading. This commit adds another
lowering, similar to the one in the Fixes: commit, that probably should
have been added at the same time. I just want to make sure this commit
gets applied everywhere that commit was also applied.
Fixes: 635ed58e52 ("intel/compiler: Lower txd for 3D samplers on XeHP.")
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15681>
This was last used with Mesa classic, in _mesa_ir_link_shader().
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15623>
This was last used with i915c, now lower_fpow covers this class of
lowering.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15623>
The define of snprintf in nir_lower_atomics_to_ssbo.c is duplicated,
so remove it from this file
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14014>
shader_storage_blocks_write_access was computed using the buffer indices
in the program but ShaderStorageBlocksWriteAccess is used with the shader
buffers.
So if a VS had 3 SSBOs and a FS had 4, the mask for VS was 0x3 (correct) but
the mask for the FS was 0x78 instead of 0x15.
Fix this by substracting the index of the first shader buffer in the program's
buffers.
Fixes: 79127f8d5b ("glsl: set ShaderStorageBlocksWriteAccess in the nir linker")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6184
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15552>
- load_reloc_const is just an immediate constant load, it's convergent.
- nir_intrinsic_load_global_const_block_intel should be convergent,
it says the address must be uniform, and we uniformize the predicate
- Lowered image intrinsics: image_deref_load_param_intel just reads
information about an image, as long as the image variable is
convergent it should be too. load_raw_intel...if the address we
come up with is convergent, it ought to be as well.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15484>
nir_shader_get_entrypoint(..) should not modify the passed nir_shader
object. Enforce this by marking shader paramenter as const.
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15362>
This is not a valid optimization.
Fixes: fb29cef8dd ("nir: add many passes that lower and optimize 16-bit input/outputs and samplers")
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14895>
As this function doesn't check for any control-flow
dependence, it only returns true for statically
(or globally) uniform values.
The same holds true for is_binding_dynamically_uniform()
in nir_opt_gcm().
Rename to better reflect that property.
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14994>
load_smem_gcn is similar to load_global/load_global_constant, but it's
guaranteed to use SMEM and it's much easier to utilize the format's 32-bit
offset source.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12773>
After running divergence analysis, we include "div" or "con" for each
SSA def's divergence/convergence status:
vec1 32 div ssa_35 = fddy ssa_34
vec1 32 con ssa_36 = fddy ssa_6.x
We omit this before the first time divergence analysis has been run.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15445>
Flat varying can save some rasterization compute cost
and register needed by shader.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15341>
This fixes performance regression for Specviewperf/Energy
on AMD GPU. Other GPUs passing varying by memory may choose
to re-enable it as need.
Fixes: 2604625043 ("nir/linker: support uniform when optimizing varying")
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15341>
The way they're handled is that deref->modes is treated as a bitfield of
possible modes. Variables are required to have a specific mode and
derefs with deref_type_var are as well.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13171>
the specification stipulates that this is a bool value, so don't load it as an int
or else nir_validate explodes
Fixes: f17b41ab4f ("nir: add lowering pass for helperInvocationEXT()")
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15402>
These will be used to implement the ir3-specific shader preamble
lowering in NIR. shps is conceptually similar to getone (although it
technically can't be duplicated) and shpe is similar to other barriers,
since it has to happen after any stores to the constant file in the
preamble. Add NIR intrinsics and plumbs them through ir3.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13148>
This pass tries to move computations that are uniform for the entire
draw to the preamble. There's also an API for backends to insert their
own instructions into the preamble, for porting existing UBO pushing
passes.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13148>
These are functions that run before the entrypoint at least once per
draw and write their results via store_preamble, and then are loaded in
the rest of the shader via load_preamble.
We will add users in the following commits.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13148>
On V3D reading UBOs from uniform addresses uses a more efficient
mechanism with lower latency. On other platforms there may be
simular scenarios.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15276>
This has been pending for a long time. It is not very consistent to
add a significant delay for textures and not do it for UBOs, etc
The reason we have not been doing this so far is the accumulated effect
on register pressure for V3D as shown by shader-db results below, but
from the point of view of a generic scheduler it makes sense to do this.
Later patches will address V3D specific issues with register pressure
derived from this by letting the driver control its instruction delay
settings.
total instructions in shared programs: 12662138 -> 13126587 (3.67%)
instructions in affected programs: 1813091 -> 2277540 (25.62%)
helped: 2410
HURT: 10499
total threads in shared programs: 415858 -> 407208 (-2.08%)
threads in affected programs: 17348 -> 8698 (-49.86%)
helped: 8
HURT: 4333
total uniforms in shared programs: 3711483 -> 3812698 (2.73%)
uniforms in affected programs: 128012 -> 229227 (79.07%)
helped: 3474
HURT: 2143
total max-temps in shared programs: 2138763 -> 2318430 (8.40%)
max-temps in affected programs: 318780 -> 498447 (56.36%)
helped: 588
HURT: 11997
total spills in shared programs: 3860 -> 49086 (1171.66%)
spills in affected programs: 709 -> 45935 (6378.84%)
helped: 23
HURT: 1595
total fills in shared programs: 5573 -> 55810 (901.44%)
fills in affected programs: 1067 -> 51304 (4708.25%)
helped: 23
HURT: 1595
LOST: 3
GAINED: 0
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15276>
We can get a generic nir_intrinsic_memory_barrier to represent a
barrier involving multiple semantics (instead of getting individual
specific barriers for each semantic). This means that we need to
consider these as potentially affecting shared memory access as well.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15276>
The workgroup_index is intended for situations when a 3 dimensional
workgroup_id is not available on the HW, but a 1 dimensional index is.
In this case, we can use lower the 3D ID to use this.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15103>
This is to match other NIR terminology.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15103>
PRIMITIVE_INDICES is a flat array in NV_mesh_shader,
not a proper arrayed output, as opposed to D3D-style
mesh shaders where it's addressed by the primitive index.
Prevent assigning several slots to primitive indices,
to avoid issues.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15160>
Like txf_ms, these take integers not floats.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15242>
These instructions have AMD hardware equivalent and they will be used
to lower fragment shader outputs in NIR.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15231>
vecN instructions which are only used by other ALU
will now get duplicate channels removed.
i915g:
total instructions in shared programs: 396309 -> 396294 (<.01%)
instructions in affected programs: 186 -> 171 (-8.06%)
r300:
total instructions in shared programs: 1165059 -> 1164354 (-0.06%)
instructions in affected programs: 35884 -> 35179 (-1.96%)
total temps in shared programs: 165497 -> 165326 (-0.10%)
temps in affected programs: 2990 -> 2819 (-5.72%)
softpipe:
total instructions in shared programs: 2860028 -> 2859084 (-0.03%)
instructions in affected programs: 55539 -> 54595 (-1.70%)
total temps in shared programs: 516939 -> 516546 (-0.08%)
temps in affected programs: 6623 -> 6230 (-5.93%)
Acked-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12468>
Marcin Ślusarz
Timur Kristóf
Icecream95
Jason Ekstrand
Boris Brezillon
Karol Herbst
Emma Anholt
Rhys Perry
Timothy Arceri
Mike Blumenkrantz
Ian Romanick
Gert Wollny
Georg Lehmann
Konstantin Seurer
Michael Skorokhodov
Jordan Justen
Lionel Landwerlin
Eric Anholt
Alyssa Rosenzweig
Yevhenii Kolesnikov
Samuel Pitoiset
Marek Olšák
Erik Faye-Lund
Alexey Bozhenko
Daniel Schürmann
Dave Airlie
Yonggang Luo
Pierre-Eric Pelloux-Prayer
Kenneth Graunke
Christian Gmeiner
Jason Ekstrand
Qiang Yu
Connor Abbott
Iago Toral Quiroga