This should help get correct math for ARB_fp/vp after the NTT transition,
and will be used for wine nine shortly.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16176>
The callback allows to request different vectorization factors
per instruction depending on e.g. bitsize or opcode.
This patch also removes using the vectorize_vec2_16bit option
from nir_opt_vectorize().
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13080>
NIR already has the necessary lowering, and the GLSL lowering violates
GLSL IR validation rules. Once quadop lowering was turned off, the IR
validation at the end of the compile path on DEBUG builds caught the
problem.
In order to move the lowering to NIR, though, we need to make sure that
drivers supporting these functions actually have the lowering flag set.
xfails added for t860, where apparently this tickles a variety of existing
64-bit bugs in the backend.
Fixes: #6461
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Mykhailo Skorokhodov <mykhailo.skorokhodov@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16437>
This is set to true for all drivers that have a GLSL level
of support lower than 4.00. This matches the rule for setting the
GLSL IR option EmitNoIndirectSampler.
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16543>
I'm working on switching mesa/st to no longer produce TGSI on its own, and
so we need a way to test SVGA against that future.
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Charmaine Lee <charmainel@vmware.com>
Reviewed-by: Neha Bhende <bhenden@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14160>
Because for fragment shaders we still use the variables, and
lower_io_to_vector may leave dead variables that duplicate inputs
that are now vectorized, we have to call this pass, because otherwise
we will may hit the assertion
src/gallium/auxiliary/tgsi/tgsi_ureg.c:318:
ureg_DECL_fs_input_centroid_layout:
Assertion `(ureg->input[i].usage_mask & usage_mask) == 0'
This is relevant for
spec@arb_enhanced_layouts@execution@component-layout@*
on r600/ntt
Fixes: a4840e15ab
r600: Use nir-to-tgsi instead of TGSI when the NIR debug opt is disabled
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16130>
We can't declare input arrays because mesa/st lowers NIR VS output
declarations to elements no matter what, and virgl has depended on
matching array sizes of declarations between producers and consumers. So,
we have to lower it away (which is fine because hardware drivers will
generally be lowering anyway).
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13295>
I had some workarounds in ALU op emits trying to fix up when we were asked
to store to unsupported channels when the ALU op had 64bit srcs (so only
vec2 supported) but a 32-bit dest with a >vec2 writemask.
Those workarounds had some bugs breaking 64-bit uniform initializer tests
on virgl, and also set up too wide of a writemask such that they triggered
assertion failures on nvc0. We can avoid the need for those workarounds
at emit time by just having nir_lower_vec_to_movs not generate unsupported
writemasks in the first place.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15934>
On virglrenderer it is of interest to not re-use temporaries when we
want to handle precise, invariant, and highp/mediump with better
possibility for optimization.
v2: Force optimized RA if the number of registers is too large
(Emma: only 16 bit signed int are reserved for register indices)
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16051>
r600 would end up looking for it past the end of its array of inputs
(which expected 1:1 ordering from declarations to driver locations).
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16043>
virglrenderer emits GLSL referencing all the swizzles, even if the write
mask doesn't contain them. This is a problem when the output is
TessLevelInner, which has only 2 elements.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16043>
Same splitting method as store_output. Fixes regressions in virgl
with nir-to-tgsi.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16043>
Found in virgl, where a glslparsertest accidentally gets its inputs
lowered to undefs, and 64-bit undefs don't get split by the normal
alu/intrinsic splitter (and would be hard to split because other passes
would see reconstruction of the vec4 from undefs and turn it back into
vec3/vec4 undef).
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16043>
I've tried to keep virglrenderer workarounds out of ntt, but this one
would be bothersome to do with tgsi_translate and TG4 is pretty low-stakes
for NTT consumers.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16043>
If we made a copy deref, then we need to do dead-write elimination for the
pervious writes or we'll just emit the same copy deref again next time
around. And, at the end of the opt loop, we need to lower copy derefs
because later passes (locals_to_regs, notably) depend on it.
Fixes infinite opt loop on fs-function-inout-array with virgl on NTT.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15899>
I missed these in the previous fix to mimic GLSL-to-TGSI address reg
behavior, which r600 relies on.
Fixes: 4bb9c0a28a ("nir_to_tgsi: Use the same address reg mappings as GLSL-to-TGSI did.")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15824>
virglrenderer maps atomic accesses to atomic counter declarations using
the .Index field. We were previously emitting a .Index of 0 for array
accesses, so virglrenderer would emit
atomicIncrement(first_counter[counter_offset+array_index]). This would
mostly work because hardware doesn't care about the bounds of counter
declarations, but if the first counter was a non-array, then the [] GLSL
emit gets dropped (can't array access a scalar!) and you'd access the
non-array first_counter instead.
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15824>
It's always in .w, rather than being the next component after the
x/y/array index.
Fixes: c6d3fd8c21 ("gallium/ntt: Emit sample index when necessary for image load/store.")
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15825>
Make sure to take all the array sizes into account when generating the TGSI.
Makes the `piglit.spec@arb_arrays_of_arrays@execution@ubo@fs-const-explicit-binding`
test pass
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15713>
In some cases like when using `NIR_DEBUG=serialize`, impl->num_blocks is 0
which leads to assertions error in the blocklist. Make sure to require the
num_blocks to be populated.
Fixes: 74c02d99b2
Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15640>
This should reduce follow-on optimization work to copy-propagate and
dead-code away the movs generated in construction of vectors.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14865>
virglrenderer makes invalid shaders when faced with vector 64-bit
instructions, which GLSL-to-TGSI never produced. While this doesn't fix
everything, it does get more tests running, and virgl probably the primary
consumer of 64-bit TGSI. virgl may be deprecating its host 64-bit
support, at which point we can drop this workaround.
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15014>
Allocating NIR registers ends up being required for drivers like r600 and
nv30, which don't do their own allocation (except in some cases on r600
where sb is used).
Rather than add a NIR register liveness impl
(https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14158), switch
from NIR-based liveness to just doing the same channel-based liveness
logic that the NIR registers needed at the TGSI level. The actual
liveness code here basically comes straight out of
brw_vec4_live_variables.cpp.
Since we do the liveness in TGSI now, it also means we don't need to be
careful about not reading SSA values from later TGSI instructions (which
may be useful for doing some greedy instruction selection in generating
TGSI instructions).
i915g:
total instructions in shared programs: 400719 -> 380730 (-4.99%)
instructions in affected programs: 284760 -> 264771 (-7.02%)
total tex_indirect in shared programs: 12289 -> 12290 (<.01%)
tex_indirect in affected programs: 4 -> 5 (25.00%)
total temps in shared programs: 32172 -> 22086 (-31.35%)
temps in affected programs: 30647 -> 20561 (-32.91%)
LOST: 0
GAINED: 148
r300:
total instructions in shared programs: 1472463 -> 1459286 (-0.89%)
instructions in affected programs: 507009 -> 493832 (-2.60%)
total temps in shared programs: 212143 -> 201678 (-4.93%)
temps in affected programs: 78007 -> 67542 (-13.42%)
softpipe:
total temps in shared programs: 517071 -> 294387 (-43.07%)
temps in affected programs: 509324 -> 286640 (-43.72%)
Acked-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14404>
To do register allocation well, we want to have a point before
ureg_insn_emit() to look at the liveness of the values and allocate them
to TGSI temporaries. In order to do that, we have to switch from
ureg_OPCODE() emitting TGSI tokens directly to a new ntt_OPCODE() that
stores the ureg args in a block structure.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14404>
Saves instructions if the same fabs value is used multiple times.
i915g:
total instructions in shared programs: 397005 -> 396525 (-0.12%)
instructions in affected programs: 11061 -> 10581 (-4.34%)
LOST: 0
GAINED: 22
r300 (not r500):
total instructions in shared programs: 180286 -> 179767 (-0.29%)
instructions in affected programs: 27102 -> 26583 (-1.91%)
total temps in shared programs: 29692 -> 29638 (-0.18%)
temps in affected programs: 356 -> 302 (-15.17%)
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14938>
Given that our fcsels are on float-bools, we can emit the LRP directly and
save the backend having to emit a SLT to turn the CMP src[0] into a bool.
This required passing a codegen flags struct for nir-to-tgsi. I think
this is a good way forward for it, as the alternative I think has mostly
been adding flags to nir_shader_compiler_options (since adding
PIPE_SHADER_CAPs is an unreasonable amount of pain).
r300 shader-db:
total instructions in shared programs: 1484320 -> 1472463 (-0.80%)
instructions in affected programs: 243588 -> 231731 (-4.87%)
total temps in shared programs: 212485 -> 212143 (-0.16%)
temps in affected programs: 3845 -> 3503 (-8.89%)
Acked-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14886>
This was identified by Coverity. 4bb9c0a28a added uses of a third
address register, but the arrays for tracking address registers only
have two slots.
Add back a version of the assertion from before 4bb9c0a28a to help
prevent future problems. I don't think any drivers that would hit
this path use NIR-to-TGSI yet, so it may be moot.
Reviewed-by: Matt Turner <mattst88@gmail.com>
CID: 1496942
CID: 1496944
Fixes: 4bb9c0a28a ("nir_to_tgsi: Use the same address reg mappings as GLSL-to-TGSI did.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14487>
This helps non-native-integers hardware where relative addressing of UBOs
has a constant offset field, and having addressing math (particularly for
D3D9) emitted as ALU ops ends up running us out of constants. For
native-integers drivers (such as softpipe), the possible-overflow check
typically triggers and we end up not folding.
r300:
total instructions in shared programs: 1279167 -> 1278731 (-0.03%)
instructions in affected programs: 50834 -> 50398 (-0.86%)
total temps in shared programs: 213736 -> 213687 (-0.02%)
temps in affected programs: 598 -> 549 (-8.19%)
total consts in shared programs: 952973 -> 952850 (-0.01%)
consts in affected programs: 26776 -> 26653 (-0.46%)
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14309>
This lets nir-to-tgsi fold the constant offset of addressing calculations
into the CONST[] reference, which is important for D3D9-era compatibility:
HW of that age has limited uniform space, and if we do the addressing math
as math in the shader for dynamic indexing, the nir_load_consts end up
taking up uniforms we don't have available.
r300:
total instructions in shared programs: 1279699 -> 1279167 (-0.04%)
instructions in affected programs: 134796 -> 134264 (-0.39%)
total instructions in shared programs: 1279699 -> 1279167 (-0.04%)
instructions in affected programs: 134796 -> 134264 (-0.39%)
total temps in shared programs: 213912 -> 213736 (-0.08%)
temps in affected programs: 2166 -> 1990 (-8.13%)
total consts in shared programs: 953237 -> 952973 (-0.03%)
consts in affected programs: 45980 -> 45716 (-0.57%)
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14309>
TGSI reserves 2 components for the coord in the first operand vector, even
for 1D. Fixes r600 failure with shadow1d.
Fixes: 390a3fcdc4 ("nir_to_tgsi: Add support for TXP.")
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14322>
This is used by virglrenderer to create the correct shaders on the
host. Fixes:
dEQP-GLES31.functional.primitive_bounding_box.triangles.tessellation_set_per_primitive.vertex_tessellation_fragment.fbo
when using ntt with virgl.
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14423>
That's how the TGSI math opcodes work.
This lets lower_vec_to_regs coalesce the DP output into the .yzw channels,
giving an impressive shader-db win on softpipe:
total instructions in shared programs: 2929840 -> 2794036 (-4.64%)
instructions in affected programs: 1651438 -> 1515634 (-8.22%)
total temps in shared programs: 372730 -> 332744 (-10.73%)
temps in affected programs: 118151 -> 78165 (-33.84%)
and a minor one on r300:
total instructions in shared programs: 51238 -> 51149 (-0.17%)
instructions in affected programs: 2621 -> 2532 (-3.40%)
total vinst in shared programs: 15655 -> 15618 (-0.24%)
vinst in affected programs: 468 -> 431 (-7.91%)
total temps in shared programs: 9838 -> 9828 (-0.10%)
temps in affected programs: 59 -> 49 (-16.95%)
and a bigger one on i915g:
total instructions in shared programs: 398064 -> 395901 (-0.54%)
instructions in affected programs: 29271 -> 27108 (-7.39%)
total tex_indirect in shared programs: 12261 -> 12233 (-0.23%)
tex_indirect in affected programs: 98 -> 70 (-28.57%)
LOST: 0
GAINED: 5
The r300 change is less impressive because it does some backend copy-prop,
but also because intermediate storage of DPs now takes a vec4 instead of a
scalar.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14200>
Softpipe was the only driver still using this feature. I had enabled it
in ba22f014f9 ("softpipe: Enable PIPE_CAP_TGSI_ANY_REG_AS_ADDRESS;") for
an instr count win, but it's really not important to that driver and it's
not worth keeping the knob around just for that.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14360>
It turns out r600 has a bunch of expectations about the Dimension being in
ADDR[1].x, and sampler or atomic indirects being in ADDR[2].x. It's
simpler to just use this static assignment than our dynamic one, anyway.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14360>
This flag is used by the nv50, r600, and svga backends for instruction
exactness. It was easier to plumb it in as an override in tgsi_ureg than
to make all of ALU instruction emit do it.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14201>
This moves some ops down to when they're needed, generally reducing the
number of temps in use. It's not always a win -- sometimes you can end up
moving a generator of a component used by a nir_op_vec down, which means
that op's sources stay live while the vec (whose register likely gets
coalesced with the ops creating it) is also live. But it's generally
good.
softpipe results:
temps in affected programs: 18115 -> 18026 (-0.49%)
imm in affected programs: 19 -> 22 (15.79%)
r300 results:
instructions in affected programs: 174 -> 178 (2.30%)
vinst in affected programs: 156 -> 160 (2.56%)
sinst in affected programs: 54 -> 50 (-7.41%)
temps in affected programs: 2634 -> 2169 (-17.65%)
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14096>
Move all the NIR related debug environmental variables in a single
NIR_DEBUG one.
Use NIR_DEBUG=help to print all the available options.
v2:
- Use a macro to simplify (Marcin, Jason)
- Remove wrong changes (Marcin)
v3 (Marcin):
- Remove rendundant NIR mentioning in option descriptions.
- Unwrap option descriptions.
- Ensure the constant is unsigned.
- Use extern array to remove switch.
v4:
- Add missing kernel shader (Jason).
- Add unlikely() (Marcin).
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13840>
Use nir_foreach_image_variable for images so we survive the coming
refactor where they get their own mode.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4743>
Emma Anholt
Georg Lehmann
Daniel Schürmann
Timothy Arceri
Gert Wollny
Corentin Noël
Ian Romanick
Juan A. Suarez Romero
Jason Ekstrand