This is, unfortunately, a large flag-day mega-commit. However, any
other approach would likely be fragile and involve a lot more churn as
we try to plumb the new vk_fence and vk_semaphore primitives into ANV's
submit code before we delete it all. Instead, we do it all in one go
and accept the consequences.
While this should be mostly functionally equivalent to the previous
code, there is one potential perf-affecting change. The command buffer
chaining optimization no longer works across VkSubmitInfo structs.
Within a single VkSubmitInfo, we will attempt to chain all the command
buffers together but we no longer try to chain across a VkSubmitInfo
boundary. Hopefully, this isn't a significant perf problem. If it ever
is, we'll have to teach the core runtime code how to combine two or more
VkSubmitInfos into a single vk_queue_submit.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>
This gets rid of the bespoke vfunc interface in the WSI code.
Eventually, I'd love to get rid of wsi_display_fence entirely but
this at least makes it a lot more palatable.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13427>
We need to guarantee that when vkQueueSubmit() returns the application
can actually wait on a signaled semaphore/syncobj.
When using a thread to do the submission to i915, this gets a bit
tricky in the following case :
A syncobj is used both as a wait & signal semaphore and has been
signaled once already. It contains a fence before entering
vkQueueSubmit().
This means we need to reset the syncobj to ensure when we return
from vkQueueSubmit(), the syncobj contains no stale fence.
Currently in the Anv, the submission thread is in charge of putting
the new fence in the syncobj and also picks up the wait fence
directly from the syncobj. This means we can't reset the syncobj
from vkQueueSubmit().
The solution to this has been pointed by Bas & Jason :
In vkQueueSubmit(), clone the wait syncobj fence into a new
temporary syncobj that will be destroy after submission and use
this temporary syncobj as a wait fence for i915. This allows us to
reset the original syncobj in vkQueueSubmit().
For this to work with wait_before_signal behavior, we also need to
do a wait-on-materialize on binary semaphores from vkQueueSubmit().
Otherwise the application thread calling vkQueueSubmit() could race
the submission thread and pick up the wrong fence when cloing.
v2: Use copy semantic for clone_syncobj_dma_fence() (Jason)
Do the cloning prior to adding the syncobj to anv_queue_submit so
that if the cloning fails don't have an invalid syncobj in
anv_queue_submit (Jason)
v3: Fix another syncobj leak (Jason)
v4: Fix invalid argument order (Lionel)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4945
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11474>
Starting with 3b363d5b55 ("anv: Assume syncobj support"), we assume
syncobj support and no longer use the execbuf sync_file API directly so
there's no point in checking for it. For the one physical device check
this deletes, we can assume has_exec_fence is always true because every
kernel with syncobj support also has sync_file.
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13610>
The code is correct, but compiler can't see it. Initialize the value
to NULL and assert on it if the function succeeds. It both helps the
compiler and make the code slightly more robust.
```
../src/intel/vulkan/anv_queue.c: In function ‘anv_QueueSubmit2KHR’:
../src/intel/vulkan/anv_queue.c:932:16: warning: ‘impl’ may be used uninitialized in this function [-Wmaybe-uninitialized]
932 | result = anv_queue_submit_add_timeline_wait(queue, submit,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
933 | &impl->timeline,
| ~~~~~~~~~~~~~~~~
934 | value);
| ~~~~~~
../src/intel/vulkan/anv_queue.c:899:31: note: ‘impl’ was declared here
899 | struct anv_semaphore_impl *impl;
| ^~~~
```
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13638>
v2: Use u_foreach_bit64() (Samuel)
v3: Add missing handling of VkMemoryBarrier2KHR in pNext of
VkSubpassDependency2KHR (Samuel)
v4: Remove unused ANV_PIPELINE_STAGE_PIPELINED_BITS (Ivan)
v5: fix missing anv_measure_submit() (Jason)
constify anv_pipeline_stage_pipelined_bits (Jason)
v6: Split flushes & invalidation emissions on
vkCmdSetEvent2KHR()/vkCmdWaitEvents2KHR() (Jason)
v7: Only apply flushes once on events (Jason)
v8: Drop split flushes for this patch
v9: Add comment about ignore some fields of VkMemoryBarrier2 in
VkSubpassDependency2KHR (Jason)
Drop spurious PIPE_CONTROL change s/,/;/ (Jason)
v10: Fix build issue on Android (Lionel)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9045>
This commit does several things:
* Unify code common to several drivers by evaluating INTEL_NO_HW within
intel_get_device_info_from_fd (suggested by Jordan).
* For drivers that keep a copy of the intel_device_info struct, a
separate copy of the no_hw field is now unnecessary. Remove them.
* Minimize kernel queries when INTEL_NO_HW is true. This is done for
code simplification, but we may find reason to undo this later on.
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12007>
If we have 2 command buffers back to back, one with a query pool, one
without, we don't want to retain the second query pool value (NULL).
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 0a7224f3ff ("anv: group as many command buffers into a single execbuf")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12107>
The only reason we had to refcount semaphores was for the ancient
sync_file semaphores which we used for pre-syncobj kernels. Now that we
assume syncobj and that code is gone, we don't need reference counting
anymore either.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9777>
We've only considered the perf query pool change previously. But we
also need to pay attention to the pass index.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 0a7224f3ff ("anv: group as many command buffers into a single execbuf")
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10301>
One exception is src/amd/addrlib/, for which -Wimplicit-fallthrough is
explicitly disabled.
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10220>
Reported by Coverity.
Fixes: 0a7224f3ff ("anv: group as many command buffers into a single execbuf")
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9596>
We also switch from using __alignof__ to alignof() in util/macros.h
which works on MSVC with the one unfortunate downside of requiring an
actual type and not a value.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9511>
This patch renames functions, structures, enums etc. with "gen_"
prefix defined in common code.
Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9413>
When semaphores are not involved, try to batch things up as much as
possible across VkSubmitInfo and also batch command buffers within a
VkSubmitInfo.
v2: Reuse anv_cmd_buffer_is_chainable()
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2371>
v2: Fixup crash spotted by Mark about missing alloc vfuncs
v3: Fixup double iteration over device->memory_objects (that ought to
be expensive...) (Ken)
v4: Add more asserts for non-softpin cases (Ken)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2371>
Before we introduced the submission thread in 829699ba63, once we
returned from vkQueueSubmit, all signaled syncobj would have a
i915_request/dma-fence waiting to be signaled by some work that would
submitted to HW by i915.
After this submission thread that is no longer the case. We added a
few checks in places like vkQueuePresentKHR() to wait for the binary
semaphores to materialize before we would hand things over to the WSI
code.
Unfortunately 829699ba63 forgot to reset the signaled binary
semaphore.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 829699ba63 ("anv: implement shareable timeline semaphores")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4276
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9188>
I'm pretty sure this doesn't fix anything because the WSI code only
use a single VkSubmitInfo, but better be safe.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: ccb7d606f1 ("anv: Use submit-time implicit sync instead of allocate-time")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8934>
We want more control over batching things up when possible to minimize
the round trips to i915. This breaking up of one big internal function
into helpers will be useful later.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8934>
This may vary based on the newer kernel engines based contexts.
v2 (Jason Ekstrand):
- Initialize anv_queue::exec_flags in anv_queue_init
- Don't conflate this with refactors to get_reset_stats
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>
This is modeled on anv_memory_type and anv_memory_heap which we already
use for managing memory types. Each anv_queue_family contains some data
which is returned by vkGetPhysicalDeviceQueueFamilyProperties() verbatim
as well as some internal book-keeping bits. An array of queue families
along with a count is stored in the physical device. Each anv_queue
then contains a pointer to the anv_queue_family to which it belongs.
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>
By moving vk_object_base_finish() to the end and putting the thread
clean-up in an if block we both better mimic anv_queue_init() and have a
more correct object destruction order. It comes at the cost of a level
of indentation but that seems to actually make the function more clear.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8667>
Fix defect reported by Coverity Scan.
Argument cannot be negative (NEGATIVE_RETURNS)
negative_returns: fd is passed to a parameter that cannot be negative
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6643>
This implements timeline semaphores using a new type of dma-fence
stored into drm-syncobjs. We use a thread to implement delayed
submissions.
v2: Drop cloning of temporary semaphores and just transfer their ownership (Jason)
Drain queue when dealing with binary semaphore
Ensure we don't submit to the thread as long as we don't need to
v3: Use __u64 not uintptr_t for kernel pointers
Fix commented code for INTEL_DEBUG=bat
Set DRM_I915_GEM_EXECBUFFER_EXT_TIMELINE_FENCES in timeline fence execbuf extension
Add new anv_queue_set_lost()
Drop multi queue stuff meant for the fake multi queue patch
Rework temporary syncobj handling
Don't use syncobj when not available (DeviceWaitIdle/CreateDevice)
Use ANV_MULTIALLOC
And a few more tweaks...
v4: Drop drained condition helper (Lionel)
Fix missing EXEC_OBJECT_WRITE on BOs we want to wait on (Jason)
v5: Add missing device->lost_reported in _anv_device_report_lost (Lionel)
Fix missing free on submit->simple_bo (Lionel)
Don't drop setting the device in lost state on QueueSubmit error (Jason)
Store submit->fence_bos as an array of uintptr_t (Jason)
v6: condition device->has_thread_submit to i915 & core DRM support (Jason)
v7: Fix submit->in_fence leakage on error (Jason)
Keep dummy semaphore with no thread submission (Jason)
v8: Move ownership of submit->out_fence to submit (Jason)
v9: Don't forget to read the VkFence's syncobj binary payload (Lionel)
v10: Take the mutex lock on anv_gem_close() (Jason/Lionel)
v11: Fix void* -> u64 cast on 32bit (Lionel)
v12: Rebase after BO backed timeline semaphore (Lionel)
v13: Fix missing snippets lost after rebase (Lionel)
v14: Drop update_binary usage (Lionel)
v15: Use ANV_MULTIALLOC (Lionel)
v16: Fix some realloc issues (Ivan)
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> (v8)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2901>
If fd==-1 is passed in ImportSemaphoreFdKHR, instead of importing the
fd, in creates an already signaled syncobj.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6427>
We don't want to leak in case of failure.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reported-by: Ivan Briano <ivan.briano@intel.com>
Fixes: 246261f0ad ("anv: prepare the driver for delayed submissions")
Fixes: 34f32a6d66 ("anv: implement VK_KHR_timeline_semaphore")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6200>
Lionel Landwerlin
Jason Ekstrand
Caio Marcelo de Oliveira Filho
Yevhenii Kolesnikov
Nanley Chery
Michel Dänzer
Timur Kristóf
Marcin Ślusarz
Anuj Phogat
Jason Ekstrand
Mark Janes
Jordan Justen
Vinson Lee
Ricardo Quesada