I missed this if statement so atomic counters weren't getting bindings
and, when you have more than one of them, that meant they were all
getting combined into one.
Fixes: 3584cb09bc15 "spirv: Give atomic counters their own variable mode"
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6060>
Track them via pending_flush_bits. Previously WFI was only tracked in
flush_bits and WAIT_FOR_ME was emitted directly. This means that we don't
emit WAIT_FOR_ME or WAIT_FOR_IDLE if there wasn't a cache flush or other
write by the GPU. Also split up host writes from sysmem writes, as only
the former require WFI/WAIT_FOR_ME.
Along the way, I also realized that we were missing proper handling of
transform feedback counter writes which require WAIT_MEM_WRITES. Plumb
that through as well. And CmdDrawIndirectByteCountEXT needs a
WAIT_FOR_ME as it does not wait for WFI internally.
As an example of what this does, a typical barrier for transform
feedback with srcAccess = VK_TRANSFORM_FEEDBACK_WRITE_COUNTER_BIT_EXT
and dstAccess = VK_ACCESS_INDIRECT_COMMAND_READ_BIT used to emit on
A650:
- WAIT_FOR_IDLE
and now we emit:
- WAIT_MEM_WRITES
- WAIT_FOR_ME
So we've eliminated a useless WFI and added some necessary waits.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6007>
These have an indirect count which is loaded from an iova, and the
minimum is taken between the indirect and direct counts. Note, I also
had to fix gen_header.py to deal with the extra-long names we get.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6007>
Now that we clear the PITCHALIGN" field when filling GMEM input attachment
descriptors, we can get rid of the extra tile width alignment on a630/a640.
With the "block_align_shift" value change, this brings down the default
gmem_align from 16k to 4k on a630/a640 and down from 24k to 12k on a650,
to match the gallium driver.
Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5528>
No functional changes - fixes the following assert:
nir_lower_io_impl: Assertion `!(modes & ~supported_modes)' failed.
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5997>
To be able to constify all radv_pipeline_generate_*() helpers.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5837>
For consistency regarding how the graphics pipeline is built.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5837>
While we are at it, remove one useless field in radv_multisample_state.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5837>
v3d_compile is now split out into a helper function that gets called a
second time if compilation fails the first time with the result
reporting the register allocation failed. The second time it is run with
the fallback scheduler to try and increase the chances of successfully
allocating the registers.
v2: Add a performance debug message when using the fallback scheduler.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
Instead of just having a bool status for the failure, there is now an
enum so that the compilation can report a more detailed status.
Currently this is only used to report whether the failure was due to
failed register allocation. The “failed” bool doesn’t seem to actually
have been used anywhere so this doesn’t really change a lot.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
The current scheduling algorithm favors parallelism a bit too
aggressively and sometimes generates shaders that fail register
allocation. This happens even if the threshold is set to zero to force
it to always use the CSR instruction choosing algorithm.
This patch adds an option to use an even more aggressive fallback that
just always picks the instruction with the shortest maximum delay in the
hope that that will generate the least register pressure. The intention
is to use this as a last resort after register allocation fails in order
to at least have a working shader.
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Acked-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
The input primitive ID is read from the VPM in the same memory segment
as the outputs. This means that writing the GS header to VPM location 0
needs to be done after reading the primitive ID. This patch adds a
dependency between the load_primitive_id intrinsic and the store_output
intrinsic for location 0 to stop the scheduler from reordering them.
v2: Use an enum for the dependency class number.
v3: Add "GS" to the class enum name.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
Adds a callback function to nir_schedule_options to give the backend a
chance to add custom dependencies between certain intrinsics. The
callback can assign a class number to the intrinsic and then set a read
or write dependency on that class.
v2: Use a linked-list of schedule nodes for the dependency classes
instead of a fixed-sized array.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
Instead of copying the individual members of nir_schedule_options into
the scoreboard, it now just keeps a pointer to the options. This avoids
the duplicated comments and makes it easier to add more options later.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
nir_schedule already has a struct for options which makes it more than
just a function declaration. Later patches intend to add more structs to
complement these options. In order to make the code easier to manage,
this moves the nir_scheduler-related parts out of nir.h to their own
header.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
Previously, objects of type OpTypeImage or OpTypeSampler were treated as
vtn_pointers and objects of type OpTypeSampledImage were a special-use
vtn_sampled_image struct. This commit changes that so that all of those
objects are stored in vtn_ssa_values. Each of images, samplers, and
sampled images, are stored as a scalar or vector nir_ssa_def whose
components are NIR deref values. We now use vtn_type_get_nir_type to
re-resolve those as-needed into GLSL sampler types for NIR.
This simplification has a number of benefits:
1. We can git rid of the rest of our special-cases for handling images
and samplers in function arguments. Now that they're treated as
structs at the glsl_type level, the generic paths can handle images
and samplers.
2. We can now construct composite values containing images and samplers
internally. It's unclear from the SPIR-V spec whether or not this
is allowed and it's not a pattern that GLSLang currently generates
thanks to GLSL rules. However, if we do start seeing SPIR-V that
contains such composites, we should now be able to handle it.
3. SPIR-V OpNull and OpUndef instructions can now create samplers,
images, and sampled images. The NIR generated won't likely be fully
valid but, given a NIR pass to do something sensible, it should be a
thing we can compile.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
We're about to make the SPIR-V -> NIR path generate a bit more complex
SSA chains for certain derefs. This will ensure we don't regress anyone
when we start making vec2's of derefs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
Connor Abbott
Jason Ekstrand
Jonathan Marek
Christian Gmeiner
Samuel Pitoiset
Eric Engestrom
Neil Roberts