Commit Graph

141563 Commits

Author SHA1 Message Date
Emma Anholt a9e3ddfc63 i915g: Fix GL_ARB_copy_buffer assertion fails.
The i915_texture() asserts that the resource is not a buffer, so check for
the buffer fallback first.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11540>
2021-06-22 21:32:36 +00:00
Lionel Landwerlin 7ed0aaced7 nir: use a more fitting index for btd_stack_push_intel
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin 423c47de99 nir: drop the btd_resume_intel intrinsic
This is now 100% equivalent to the new rt_resume intrinsic.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin 4d9fcf2799 intel/rt: switch to common pass for shader calls lowering
v2: rename for new indices

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin 1dacea10f3 anv: implement caching for ray tracing pipelines
v2: Turn a bunch of pointer checks into checks against NULL (Jason)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin fed7ac932f anv: move trivial return shader to device
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin 7c852f78c4 anv: store more RT shader data in pipeline_stage object
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin 045f4600b1 anv: cache raytracing trampoline shader
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Lionel Landwerlin ab77aeb488 blorp: add blorp string in shader keys
Upon looking at caching the raytracing shader (in particular the
trampoline one) I kind of got afraid that some of the keys used for
blorp would end up matching other keys. This is because blorp keys are
fairly simple. There is no SPIRV module hash included.

This change includes a "blorp" string at the beginning of the queue to
ensure we don't collide with other keys.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 7479fe6ae0 anv: Implement vkCmdTraceRays and vkCmdTraceRaysIndirect
v2: Fix anv_cmd_state::binding_tables array size (Lionel)

v2: Fix anv_cmd_state::samplers array size (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand ac6d7a1758 anv: Make anv_address::offset 64-bit
This allows us to convert a 64-bit address to an anv_address which is
useful for working with device addresses.

v2: switch to int64_t to keep state pool relative relocation working
    on non-softpin platforms

v3: Update assert to reflect relative offsets (Jason)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand a67d7c9dee anv: Allow _anv_combine_address with a NULL batch
This is required in order to be able to use GenXML pack functions for
structs with addresses when you're not packing into a batch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand f68d64dac0 anv: Add support for vkCmdSetRayTracingPipelineStackSizeKHR
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 02f7964a13 anv: Compute scratch sizes for ray-tracing pipelines and shader groups
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand c3ac9afca3 anv: Create and return ray-tracing pipeline SBT handles
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand b66d3e627a intel/fs: Don't pull CS push constants if uses_inline_data
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 79dc25d867 anv: Compile trivial return and trampoline shaders
These don't necessarily go in any group but are required for dispatch to
work properly.  The trampoline is a compute shader that is the initial
start point for the trace.  It's in charge of invoking the actual
ray-gen shader.  The trivial return shader is used whenever another
shader is missing and it does no work except the minimum required to do
a stack return.

v2: Rebase on upstream changes (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand e104555851 anv: Compile ray-tracing shaders
This doesn't look too different from other compile functions we have in
anv_pipeline.c.  The primary difference is that ray-tracing pipelines
have this weird two-stage thing where you have "stages" which are
individual shaders and "groups" which are sort of mini pipelines that
are used to handle hits.  For any given ray intersection, only the hit
and intersection shaders from the same group get used together.  You
can't have an intersection shader from group A used with an any-hit from
group B.  This results in a weird two-step compile.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 379b9bb7b0 anv: Support fetching descriptor addresses from push constants
Bindless shaders don't have binding tables so they have to get at the
descriptor sets via a different mechanism.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 937ffb1af0 nir/apply_pipeline_layout: Handle bindless shaders
They don't have binding tables so they have to use A64 descriptor set
access and everything has to be bindless all the time.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 770d331285 anv: Disallow UBO pushing for bindless shaders
They don't really have push constants.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand c92fd35848 intel/rt: Use reloc constants for the resume SBT
It's going to be attached to the end of the shader binary, not an
arbitrary table somewhere in memory.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 705395344d intel/fs: Add support for compiling bindless shaders with resume shaders
Instead of depending on the driver to compile each resume shader
separately, we compile them all in one go in the back-end and build an
SBT as part of the shader program.  Shader relocs are used to make the
entries in the SBT point point to the correct resume shader.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand d055ac9bdf intel/compiler: Add a U32 reloc type
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 55508bbe66 intel/compiler: Generalize shader relocations a bit
This commit adds a delta to be added to the relocated value as well as
the possibility of multiple types of relocations.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand f7668d6fe5 anv,iris: Move the SHADER_RELOC enums to brw_compiler.h
They're common between the two drivers and we want to add a couple more
that get emitted from code in src/intel/compiler.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand d95d5b1465 anv: Add support for binding ray-tracing pipelines
This includes not only vkCmdBindPipeline but also descriptor sets and
push constants.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 8cc5080580 anv: Add a ray-tracing pipeline object
This just adds the core data structure which we'll build on going
forward.

v2: Add VK_EXT_pipeline_creation_cache_control handling (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand dc05daf0e5 anv: Get ready for more pipeline stages
This makes a bunch of loops use ARRAY_SIZE instead of MESA_SHADER_STAGES,
extends a few arrays, and adds a bunch of array length asserts.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand f366f6a071 anv: Add minimal boilerplate for VK_KHR_ray_tracing_pipeline
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand d258b0bf0e anv: Add support for binding acceleration structures
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 4664c92478 anv: Add stub support for acceleration structures
This just adds a base struct and trivial implementations of all the
create/destroy/bind functions.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand e97002ebcf anv: Add minimal boilerplate for VK_KHR_acceleration_structure
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Jason Ekstrand 72437f6d54 spirv: Create acceleration structure and shader record variables
spirv_to_nir now requires NIR variables to be created for everything.

Fixes: 10b3eecd36 "spirv: Don't remove variables used by resource..."
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8637>
2021-06-22 21:09:25 +00:00
Mike Blumenkrantz 5cecf826d3 zink: don't clear batch resources on fence finish
this eliminates the need for locking here and has no noticeable
increase in overhead, so the reduction of complexity seems worthwhile

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz b315aa43f1 zink: move some end-of-batch stuff to submit thread
no reason to do this on the driver thread when it could be async

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz d8dc03c213 zink: flag scanout updates to batch state, not resource
now that submission is serialized better, it's not actually the resource that should be
tagged for scanout sync, it's the batch state, as multiple contexts might reuse the same
resource, thus requiring synchronization on every submit

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz c1d06c402a zink: move fence reset to submit thread
try to reduce job queue latency

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 97f8249336 zink: move sparse buffer commit to screen queue
all queue submission must be serialized with the dispatch thread

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 478f129ee7 zink: move queue submit thread to screen
this needs to be a screen-based queue in order to serialize properly
with multiple contexts

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz d4159963e3 zink: enforce multi-context waiting for unflushed resources on foreign batches
this doesn't seem to be a real issue now that tc doesn't break makeCurrent
anymore, but if such a thing were to once again become a problem, at least
there will be handling for it

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz ef418cfc59 zink: make init_mem_range() a public function
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 18c22f5024 zink: break out offset alignment calculation into helper
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz d574b0a51e zink: apply zink_resource_object::offset for memory flush/invalidates
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz d80d9e1c93 zink: queue v3.0
this uses a pointer to a batch state substruct for timeline tracking,
which provides a few nice benefits:
* explicit ability to detect unflushed batches (even on other contexts)
* the context doesn't need to have a "current" timeline id
* timeline (batch) ids can be distributed during submit, not when recording begins
* an abstracted api which can be more easily changed under the hood

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 28496f6ff2 zink: remove no-longer-used resource helper functions
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 59ec1794f6 zink: use batch usage api for resource helper function
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 1226628eb6 zink: add and use fencing functions which take batch usage structs
this hides the exact mechanics of fencing based on batch usage and makes
the usage a bit more explicit

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 982a2dfbd4 zink: improve tc fence disambiguation
instead of storing separate batch_id values onto the tc fence for matching:
* use a fence pointer on the main context to indicate the current deferred fence
* add a monotonic counter to each batch state which double-increments on each use,
  thereby enabling direct determination by the tc fence as to whether this is the same
  batch state it was created with

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00
Mike Blumenkrantz 618784732a zink: only update last_finished during batch reset if the batch was used
0 is never a valid batch_id, so don't try to update this

minor cosmetic change

Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11437>
2021-06-22 20:57:33 +00:00