These are pretty straightforward but there's a lot of details to keep
straight. In the IR, we keep a general logical comparator and types
separately; in the hardware, the type gets fused with a (much more)
limited number of comparators. So there's a fair bit of code here to
account for these differences, fusing in the type information, and
changing up argument order as necessary to make it actually correct.
Anything to save a bit!
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
This way we always have regular csel conditions instead of a weird
.always special case for 3-src CSEL mode.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
This gets us part of the way there to packing lo/hi separately. A little
more work is needed to do this "properly", but hey.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
When generating csel instructions, we can peak to see what condition is
being used. If we're using a "nice" condition, we can fuse it in with
the csel itself, ideally letting the condition itself be DCE'd away.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
We would like to use this routine opportunistically when fusing
conditions into csels and branches, so let's add a mode where we don't
abort.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
We need to make the semantics of BI_VECTOR a bit more precise -
vectorize only the first argument, not all of them. This is enough for
current and future users, as far as I know.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
We rewrite BIR_INDEX_CONSTANT (and _ZERO) to preassigned constant ports
when assign uniform_const for the bundle. There are a lot of issues
raised here, unfortunately, and the implementation here is woefully
incomplete with a nasty hack for loads... nevertheless, it's somewhere
to start.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
We're not totally sure what's up with this but Connor says if you
violate it Bad Things happen in your shader. I think this might be an
issue affecting early Bifrost (G71, ..?); when we know more we can look
into patching in a fix.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
We need ALU mostly scalarized, but we get vector moves created from
lower_vec_to_mov so let's scalarize that ourselves rather than bother
NIR.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
We split off MOV from FMOV since the canonical move on Bifrost doesn't
accept modifiers. (We can still do fmov, but with something like add-0.)
This will also make copyprop a little nicer, I think. Anyway, the
non-modifier version we can implement as-is for FMA.
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4276>
This reworks the data structure a bit and, in my view, simplifies it.
Instead of each node having a header which has the node level in it, we
use the bottom 6 bits of the pointer for that. This requires us to
allocate with the os_malloc/free_aligned helpers (which call into
posix_memalign on Linux) but cache-line aligning our allocations is
actually probably a good thing given that we're doing atomics on them.
The primary advantages to doing this is that it changes the number of
memory accesses per tree level from 2 to 1 when walking the tree because
we no longer have to look at node->level.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4228>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4228>
As soon as I switch to using the allocation helpers in os_memory.h,
these tests start blowing up on the Windows build in GitLab CI. As far
as I can tell, the issue is something with the combination of the debug
allocator in u_debug_memory.c and the mutex implementation in the
version of Wine running in CI. The tests don't fail on real windows nor
do they fail with newer versions of Wine.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4228>
We use this value several times. It's probably best to encourage the
compiler to only read it once. I have no proof that this actually makes
any performance improvement whatsoever.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4228>
Overwrite function for this type was missing and I needed it for my project.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Signed-off-by: Mark Menzynski <mmenzyns@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3903>
This is preparation for the next commit where we may need different
swap_r_b flags for pixel and texel formats.
Reviewed-by: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4241>
According to the SPIR-V specification, these operations require
integer-types. When bit_size is 1, we use booleans, which makes us emit
illegal code.
So let's fix the emitting to check if the first source is one bit wide.
For inot we can take a short-cut, and check the destination instead.
This doesn't work for ieq and ine, so let's not bother to do this
BINOP_LOG.
Tested-by: Marge Bot <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4036>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4036>
Alyssa Rosenzweig
Jonathan Marek
Marek Olšák
Eduardo Lima Mitev
Jason Ekstrand
Eric Anholt
Mark Menzynski
Vasily Khoruzhick
Erik Faye-Lund
Michel Dänzer
Rhys Perry