KonstantinSeurer
/
mesa
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
mesa/src/intel/compiler/brw_fs_generator.cpp

2476 lines
88 KiB
C++
Raw Normal View History

i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
/*
* Copyright © 2010 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
i965: Rename brw_{fs,vec4}_emit.cpp to brw_{fs,vec4}_generator.cpp. The previous names were really confusing to talk about: - brw_fs_visitor() contained methods named emit_whatever(). - brw_fs_generator() contained methods named generate_whatever(), but lived in brw_fs_emit.cpp. So when someone said "the emit layer", or "emit code", we weren't sure whether they meant the visitor's emit() functions or the generator in brw_fs_emit.cpp. By renaming these files, the method names, class names, and file names all match, which is much less confusing. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Paul Berry <stereotype441@gmail.com> Acked-by: Eric Anholt <eric@anholt.net>
2013-09-18 07:32:10 +01:00
/** @file brw_fs_generator.cpp
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
*
i965: Rename brw_{fs,vec4}_emit.cpp to brw_{fs,vec4}_generator.cpp. The previous names were really confusing to talk about: - brw_fs_visitor() contained methods named emit_whatever(). - brw_fs_generator() contained methods named generate_whatever(), but lived in brw_fs_emit.cpp. So when someone said "the emit layer", or "emit code", we weren't sure whether they meant the visitor's emit() functions or the generator in brw_fs_emit.cpp. By renaming these files, the method names, class names, and file names all match, which is much less confusing. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Paul Berry <stereotype441@gmail.com> Acked-by: Eric Anholt <eric@anholt.net>
2013-09-18 07:32:10 +01:00
* This file supports generating code from the FS LIR to the actual
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
* native instructions.
*/
#include "brw_eu.h"
#include "brw_fs.h"
i965: Move brw_fs_cfg.* to brw_cfg.*. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-10-03 21:03:12 +01:00
#include "brw_cfg.h"
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965: Add and use enum brw_reg_file. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 21:11:44 +01:00
static enum brw_reg_file
brw_file_from_reg(fs_reg *reg)
i965/fs: Move brw_file_from_reg() higher in the file. This was supposed to be part of the previous commit.
2014-12-05 17:53:11 +00:00
{
switch (reg->file) {
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-27 00:52:57 +00:00
case ARF:
return BRW_ARCHITECTURE_REGISTER_FILE;
case FIXED_GRF:
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-27 00:09:25 +00:00
case VGRF:
i965/fs: Move brw_file_from_reg() higher in the file. This was supposed to be part of the previous commit.
2014-12-05 17:53:11 +00:00
return BRW_GENERAL_REGISTER_FILE;
case MRF:
return BRW_MESSAGE_REGISTER_FILE;
case IMM:
return BRW_IMMEDIATE_VALUE;
i965: Replace default case with list of enum values. If we add a new file type, we'd like to get warnings if it's not handled. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-10-26 13:58:56 +00:00
case BAD_FILE:
case ATTR:
case UNIFORM:
i965/fs: Move brw_file_from_reg() higher in the file. This was supposed to be part of the previous commit.
2014-12-05 17:53:11 +00:00
unreachable("not reached");
}
i965: Add and use enum brw_reg_file. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 21:11:44 +01:00
return BRW_ARCHITECTURE_REGISTER_FILE;
i965/fs: Move brw_file_from_reg() higher in the file. This was supposed to be part of the previous commit.
2014-12-05 17:53:11 +00:00
}
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
static struct brw_reg
i965/fs: double regioning parameters and execsize for DF in IVB/BYT In IVB and BYT, both regioning parameters and execution sizes are measured as 32-bits element size. So when we have something like: mov(8) g2<1>DF g3<4,4,1>DF We are not actually moving 8 doubles (our intention), but 4 doubles. We need to double the parameters to cope with this issue. However, horizontal strides don't behave as they're supposed to on IVB for DF regions, they will cause each 32-bit half of DF sources to be strided individually, and doubling the value won't make any difference. v2: - Use devinfo directly (Matt). - Use Baytrail instead of Valleview (Matt). - Use IvyBridge instead of Ivy (Matt) - Double the exec_size in code emission (Curro) v3: - Change hstride doubling by an assert and fix commit log (Curro). - Substitute remaining compiler->devinfo by devinfo (Curro). v4: - Fix comment (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-07-18 08:27:56 +01:00
brw_reg_from_fs_reg(const struct gen_device_info *devinfo, fs_inst *inst,
fs_reg *reg, bool compressed)
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
{
struct brw_reg brw_reg;
switch (reg->file) {
case MRF:
i965/fs: double regioning parameters and execsize for DF in IVB/BYT In IVB and BYT, both regioning parameters and execution sizes are measured as 32-bits element size. So when we have something like: mov(8) g2<1>DF g3<4,4,1>DF We are not actually moving 8 doubles (our intention), but 4 doubles. We need to double the parameters to cope with this issue. However, horizontal strides don't behave as they're supposed to on IVB for DF regions, they will cause each 32-bit half of DF sources to be strided individually, and doubling the value won't make any difference. v2: - Use devinfo directly (Matt). - Use Baytrail instead of Valleview (Matt). - Use IvyBridge instead of Ivy (Matt) - Double the exec_size in code emission (Curro) v3: - Change hstride doubling by an assert and fix commit log (Curro). - Substitute remaining compiler->devinfo by devinfo (Curro). v4: - Fix comment (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-07-18 08:27:56 +01:00
assert((reg->nr & ~BRW_MRF_COMPR4) < BRW_MAX_MRF(devinfo->gen));
i965: Move MRF register asserts out of brw_reg.h In a later patch we will make BRW_MAX_MRF return a different value depending on the hardware generation, but it is inconvenient to add a gen parameter to the brw_reg functions only for the assertions, so move these to places where we have the hardware generation available. Ken suggested to add the asserts to brw_set_src0 and brw_set_dest since that would make sure that we catch all uses of MRF registers, even those coming from modules that generate native code directly, like blorp. Unfortunately, this is very late in the process which can make things harder to debug, so add asserts to the generator as well. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-09-16 08:08:19 +01:00
/* Fallthrough */
i965: Rename GRF to VGRF. The 2-bit hardware register file field is ARF, GRF, MRF, IMM. Rename GRF to VGRF (virtual GRF) so that we can reuse the GRF name to mean an assigned general purpose register. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-27 00:09:25 +00:00
case VGRF:
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
if (reg->stride == 0) {
i965: Use brw_reg's nr field to store register number. In addition to combining another field, we get replace silliness like "reg.reg" with something that actually makes sense, "reg.nr"; and no one will ever wonder again why dst.reg isn't a dst_reg. Moving the now 16-bit nr field to a 16-bit boundary decreases code size by about 3k. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-26 11:35:14 +00:00
brw_reg = brw_vec1_reg(brw_file_from_reg(reg), reg->nr, 0);
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
} else {
/* From the Haswell PRM:
*
i965/fs: Extend region width calculation to allow arbitrary execution sizes. Instead of just halving the execution size when the instruction is compressed hoping that it will give a legal source region width, we can calculate the maximum legal width value in closed form from the component size and stride. This makes sure that brw_reg_from_fs_reg() always returns a valid hardware region even for virtual 32-wide instructions (e.g. send-like instructions) that would seem to exceed the hardware region width limit after halving. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 05:43:48 +01:00
* "VertStride must be used to cross GRF register boundaries. This
* rule implies that elements within a 'Width' cannot cross GRF
* boundaries."
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
*
i965/fs: Extend region width calculation to allow arbitrary execution sizes. Instead of just halving the execution size when the instruction is compressed hoping that it will give a legal source region width, we can calculate the maximum legal width value in closed form from the component size and stride. This makes sure that brw_reg_from_fs_reg() always returns a valid hardware region even for virtual 32-wide instructions (e.g. send-like instructions) that would seem to exceed the hardware region width limit after halving. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 05:43:48 +01:00
* The maximum width value that could satisfy this restriction is:
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
*/
i965/fs: Extend region width calculation to allow arbitrary execution sizes. Instead of just halving the execution size when the instruction is compressed hoping that it will give a legal source region width, we can calculate the maximum legal width value in closed form from the component size and stride. This makes sure that brw_reg_from_fs_reg() always returns a valid hardware region even for virtual 32-wide instructions (e.g. send-like instructions) that would seem to exceed the hardware region width limit after halving. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 05:43:48 +01:00
const unsigned reg_width = REG_SIZE / (reg->stride * type_sz(reg->type));
/* Because the hardware can only split source regions at a whole
* multiple of width during decompression (i.e. vertically), clamp
* the value obtained above to the physical execution size of a
* single decompressed chunk of the instruction:
*/
const unsigned phys_width = compressed ? inst->exec_size / 2 :
inst->exec_size;
/* XXX - The equation above is strictly speaking not correct on
* hardware that supports unbalanced GRF writes -- On Gen9+
* each decompressed chunk of the instruction may have a
* different execution size when the number of components
* written to each destination GRF is not the same.
*/
Revert "intel/fs: Use a pure vertical stride for large register strides" This reverts commit e8c9e65185de3e821e1e482e77906d1d51efa3ec. With the actual bug fixed (by commit 6ac2d1690192), this is not necessary. I'm doubtful of its correctness in any case.
2017-11-14 19:24:08 +00:00
const unsigned width = MIN2(reg_width, phys_width);
brw_reg = brw_vecn_reg(width, brw_file_from_reg(reg), reg->nr, 0);
brw_reg = stride(brw_reg, width * reg->stride, width, reg->stride);
i965/fs: fix dst stride in IVB/BYT type conversions When converting a DF to 32-bit conversions, we set dst stride to 2, to fulfill alignment restrictions because the upper Dword of every Qword will be written with undefined value. But in IVB/BYT, this is not necessary, as each DF conversion already writes 2, the first one the real value, and the second one a 0. That is, IVB/BYT already set stride = 2 implicitly, so we must set it to 1 explicitly to avoid ending up with stride = 4. v2: - Fix typo (Matt) v3: - Fix stride in the destination's brw_reg, don't modity IR (Curro) v4: - Remove 'is_dst' argument of brw_reg_from_fs_reg() (Curro) - Fix comment (Curro). - Relax hstride assert (Curro) Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com> Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> [ Francisco Jerez: Minor spelling fixes. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-01-20 07:50:50 +00:00
if (devinfo->gen == 7 && !devinfo->is_haswell) {
/* From the IvyBridge PRM (EU Changes by Processor Generation, page 13):
* "Each DF (Double Float) operand uses an element size of 4 rather
* than 8 and all regioning parameters are twice what the values
* would be based on the true element size: ExecSize, Width,
* HorzStride, and VertStride. Each DF operand uses a pair of
* channels and all masking and swizzing should be adjusted
* appropriately."
*
* From the IvyBridge PRM (Special Requirements for Handling Double
* Precision Data Types, page 71):
* "In Align1 mode, all regioning parameters like stride, execution
* size, and width must use the syntax of a pair of packed
* floats. The offsets for these data types must be 64-bit
* aligned. The execution size and regioning parameters are in terms
* of floats."
*
* Summarized: when handling DF-typed arguments, ExecSize,
* VertStride, and Width must be doubled.
*
* It applies to BayTrail too.
*/
if (type_sz(reg->type) == 8) {
brw_reg.width++;
if (brw_reg.vstride > 0)
brw_reg.vstride++;
assert(brw_reg.hstride == BRW_HORIZONTAL_STRIDE_1);
}
/* When converting from DF->F, we set the destination stride to 2
* because each d2f conversion implicitly writes 2 floats, being
* the first one the converted value. IVB/BYT actually writes two
* F components per SIMD channel, and every other component is
* filled with garbage.
*/
if (reg == &inst->dst && get_exec_type_size(inst) == 8 &&
type_sz(inst->dst.type) < 8) {
assert(brw_reg.hstride > BRW_HORIZONTAL_STRIDE_1);
brw_reg.hstride--;
}
i965/fs: double regioning parameters and execsize for DF in IVB/BYT In IVB and BYT, both regioning parameters and execution sizes are measured as 32-bits element size. So when we have something like: mov(8) g2<1>DF g3<4,4,1>DF We are not actually moving 8 doubles (our intention), but 4 doubles. We need to double the parameters to cope with this issue. However, horizontal strides don't behave as they're supposed to on IVB for DF regions, they will cause each 32-bit half of DF sources to be strided individually, and doubling the value won't make any difference. v2: - Use devinfo directly (Matt). - Use Baytrail instead of Valleview (Matt). - Use IvyBridge instead of Ivy (Matt) - Double the exec_size in code emission (Curro) v3: - Change hstride doubling by an assert and fix commit log (Curro). - Substitute remaining compiler->devinfo by devinfo (Curro). v4: - Fix comment (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-07-18 08:27:56 +01:00
}
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
}
brw_reg = retype(brw_reg, reg->type);
i965/fs: Handle arbitrary offsets in brw_reg_from_fs_reg for MRF/VGRF registers. This restriction seemed rather artificial... Removing it actually simplifies things slightly. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-02 05:25:18 +01:00
brw_reg = byte_offset(brw_reg, reg->offset);
i965: Remove fixed_hw_reg field from backend_reg. Since backend_reg now inherits brw_reg, we can use it in place of the fixed_hw_reg field. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-24 23:29:03 +01:00
brw_reg.abs = reg->abs;
brw_reg.negate = reg->negate;
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
break;
i965: Replace HW_REG with ARF/FIXED_GRF. HW_REGs are (were!) kind of awful. If the file was HW_REG, you had to look at different fields for type, abs, negate, writemask, swizzle, and a second file. They also caused annoying problems like immediate sources being considered scheduling barriers (commit 6148e94e2) and other such nonsense. Instead use ARF/FIXED_GRF/MRF for fixed registers in those files. After a sufficient amount of time has passed since "GRF" was used, we can rename FIXED_GRF -> GRF, but doing so now would make rebasing awful. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-27 00:52:57 +00:00
case ARF:
case FIXED_GRF:
i965: Drop IMM fs_reg/src_reg -> brw_reg conversions. The previous two commits make this unnecessary. Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 20:25:24 +00:00
case IMM:
i965/fs: Handle arbitrary offsets in brw_reg_from_fs_reg for MRF/VGRF registers. This restriction seemed rather artificial... Removing it actually simplifies things slightly. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-02 05:25:18 +01:00
assert(reg->offset == 0);
i965: Prevent implicit upcasts to brw_reg. Now that backend_reg inherits from brw_reg, we have to be careful to avoid the object slicing problem. Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-11-20 05:51:37 +00:00
brw_reg = reg->as_brw_reg();
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
break;
case BAD_FILE:
/* Probably unused. */
brw_reg = brw_null_reg();
break;
i965: Replace default case with list of enum values. If we add a new file type, we'd like to get warnings if it's not handled. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2015-10-26 13:58:56 +00:00
case ATTR:
case UNIFORM:
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
unreachable("not reached");
}
i965: Use <0,2,1> region for scalar DF sources on IVB/BYT. On HSW+, scalar DF sources can be accessed using the normal <0,1,0> region, but on IVB and BYT DF regions must be programmed in terms of floats. A <0,2,1> region accomplishes this. v2: - Apply region <0,2,1> in brw_reg_from_fs_reg() (Curro). v3: - Added comment explaining the reason (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2017-01-11 03:33:22 +00:00
/* On HSW+, scalar DF sources can be accessed using the normal <0,1,0>
* region, but on IVB and BYT DF regions must be programmed in terms of
* floats. A <0,2,1> region accomplishes this.
*/
if (devinfo->gen == 7 && !devinfo->is_haswell &&
type_sz(reg->type) == 8 &&
brw_reg.vstride == BRW_VERTICAL_STRIDE_0 &&
brw_reg.width == BRW_WIDTH_1 &&
brw_reg.hstride == BRW_HORIZONTAL_STRIDE_0) {
brw_reg.width = BRW_WIDTH_2;
brw_reg.hstride = BRW_HORIZONTAL_STRIDE_1;
}
i965/fs: Make brw_reg_from_fs_reg static and remove prototype. And move it above its first use in brw_fs_generator.cpp. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-28 20:21:03 +00:00
return brw_reg;
}
i965: Remove the dependance on brw_context from the generators Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-16 22:34:04 +01:00
fs_generator::fs_generator(const struct brw_compiler *compiler, void *log_data,
i965/fs: Plumb a mem_ctx all the way through the FS compile. 'c' is going away, but we still need a memory context that lives for the duration of the compile. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-05-14 09:21:02 +01:00
void *mem_ctx,
i965: Generalize fs_generator further This removes all stage specific data from the generator, and lets us create a generator for any stage. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 06:53:31 +01:00
struct brw_stage_prog_data *prog_data,
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
unsigned promoted_constants,
i965: Pass a shader stage abbreviation to fs_generator(). A lot of messages hardcoded the string "FS", which is confusing on Broadwell, where we use this code for VS support as well. shader-db particularly got confused, as it reported two "FS SIMD8" shaders, and no vertex shaders at all. Craziness ensued. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-01-13 22:28:13 +00:00
bool runtime_check_aads_emit,
i965/fs/generator: Take an actual shader stage rather than a string Cc: "11.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-15 04:27:51 +00:00
gl_shader_stage stage)
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
i965: Remove the dependance on brw_context from the generators Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-16 22:34:04 +01:00
: compiler(compiler), log_data(log_data),
intel/fs: Remove program key argument from generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 01:20:35 +01:00
devinfo(compiler->devinfo),
i965: Remove shader program argument and member from fs_generator Now that the caller passes in the shader debug name, we don't need this anymore. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-28 02:43:31 +00:00
prog_data(prog_data),
i965/fs: Remove the gl_program from the generator Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-10-06 00:01:33 +01:00
promoted_constants(promoted_constants),
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
runtime_check_aads_emit(runtime_check_aads_emit), debug_flag(false),
i965/fs/generator: Take an actual shader stage rather than a string Cc: "11.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-15 04:27:51 +00:00
stage(stage), mem_ctx(mem_ctx)
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
{
i965: Rename brw_compile to brw_codegen This name better matches what it's actually used for. The patch was generated with the following command: for file in *; do sed -i -e s/brw_compile/brw_codegen/g $file done Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-16 19:06:57 +01:00
p = rzalloc(mem_ctx, struct brw_codegen);
brw_init_codegen(devinfo, p, mem_ctx);
intel/fs: Don't use automatic exec size inference The automatic exec size inference can accidentally mess things up if we're not careful. For instance, if we have add(4) g38.2<4>D g38.1<8,2,4>D g38.2<8,2,4>D then the destination register will end up having a width of 2 with a horizontal stride of 4 and a vertical stride of 8. The EU emit code sees the width of 2 and decides that we really wanted an exec size of 2 which doesn't do what we wanted. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-31 17:53:02 +01:00
/* In the FS code generator, we are very careful to ensure that we always
* set the right execution size so we don't need the EU code to "help" us
* by trying to infer it. Sometimes, it infers the wrong thing.
*/
p->automatic_exec_sizes = false;
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
}
fs_generator::~fs_generator()
{
}
i965/fs: Move ip_record class to its one use. Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2014-11-12 19:01:16 +00:00
class ip_record : public exec_node {
public:
DECLARE_RALLOC_CXX_OPERATORS(ip_record)
ip_record(int ip)
{
this->ip = ip;
}
int ip;
};
i965/fs: Make patch_discard_jumps_to_fb_writes return bool. ... to tell us whether it emitted any code. Will be used to determine whether we need to skip an annotation for it. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Juha-Pekka Heikkila <juhapekka.heikkila@gmail.com>
2014-05-16 21:06:45 +01:00
bool
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
fs_generator::patch_discard_jumps_to_fb_writes()
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen < 6 || this->discard_halt_patches.is_empty())
i965/fs: Make patch_discard_jumps_to_fb_writes return bool. ... to tell us whether it emitted any code. Will be used to determine whether we need to skip an annotation for it. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Juha-Pekka Heikkila <juhapekka.heikkila@gmail.com>
2014-05-16 21:06:45 +01:00
return false;
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
int scale = brw_jump_scale(p->devinfo);
i965/eu: Refactor jump distance scaling to use a helper function. Different generations of hardware measure jump distances in different units. Previously, every function that needed to set a jump target open coded this scaling, or made a hardcoded assumption (i.e. just used 2). Most functions start with the number of instructions to jump, and scale up to the hardware-specific value. So, I made the function match that. Others start with a byte offset, and divide by a constant (8) to obtain the jump distance. This is actually 16 / 2 (the jump scale for Gen5-7). v2: Make the helper a static inline defined in brw_eu.h, instead of an actual function in brw_eu_emit.c (as suggested by Matt). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-30 16:00:25 +01:00
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
/* There is a somewhat strange undocumented requirement of using
* HALT, according to the simulator. If some channel has HALTed to
* a particular UIP, then by the end of the program, every channel
* must have HALTed to that UIP. Furthermore, the tracking is a
* stack, so you can't do the final halt of a UIP after starting
* halting to a new UIP.
*
* Symptoms of not emitting this instruction on actual hardware
* included GPU hangs and sparkly rendering on the piglit discard
* tests.
*/
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 22:29:25 +01:00
brw_inst *last_halt = gen6_HALT(p);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_uip(p->devinfo, last_halt, 1 * scale);
brw_inst_set_jip(p->devinfo, last_halt, 1 * scale);
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
int ip = p->nr_insn;
i965: Use typed foreach_in_list instead of foreach_list. Acked-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-24 23:53:19 +01:00
foreach_in_list(ip_record, patch_ip, &discard_halt_patches) {
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 22:29:25 +01:00
brw_inst *patch = &p->store[patch_ip->ip];
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
assert(brw_inst_opcode(p->devinfo, patch) == BRW_OPCODE_HALT);
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
/* HALT takes a half-instruction distance from the pre-incremented IP. */
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_uip(p->devinfo, patch, (ip - patch_ip->ip) * scale);
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
}
this->discard_halt_patches.make_empty();
i965/fs: Make patch_discard_jumps_to_fb_writes return bool. ... to tell us whether it emitted any code. Will be used to determine whether we need to skip an annotation for it. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Juha-Pekka Heikkila <juhapekka.heikkila@gmail.com>
2014-05-16 21:06:45 +01:00
return true;
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
}
intel/fs: Add a generic SEND opcode Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 20:06:14 +00:00
void
fs_generator::generate_send(fs_inst *inst,
struct brw_reg dst,
struct brw_reg desc,
struct brw_reg ex_desc,
struct brw_reg payload,
struct brw_reg payload2)
{
/* SENDS not yet supported */
assert(ex_desc.file == BRW_IMMEDIATE_VALUE && ex_desc.d == 0);
assert(payload2.file == BRW_ARCHITECTURE_REGISTER_FILE &&
payload2.nr == BRW_ARF_NULL);
const bool dst_is_null = dst.file == BRW_ARCHITECTURE_REGISTER_FILE &&
dst.nr == BRW_ARF_NULL;
const unsigned rlen = dst_is_null ? 0 : inst->size_written / REG_SIZE;
uint32_t desc_imm = inst->desc |
brw_message_desc(devinfo, inst->mlen, rlen, inst->header_size);
brw_send_indirect_message(p, inst->sfid, dst, payload, desc, desc_imm);
brw_inst_set_eot(p->devinfo, brw_last_inst, inst->eot);
if (inst->check_tdr)
brw_inst_set_opcode(p->devinfo, brw_last_inst, BRW_OPCODE_SENDC);
}
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
void
fs_generator::fire_fb_write(fs_inst *inst,
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
struct brw_reg payload,
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
struct brw_reg implied_header,
GLuint nr)
{
uint32_t msg_control;
i965: Introduce downcast helpers for prog_data structures. Similar to brw_context(...), intel_texture_object(...), and so on. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Timothy Arceri <timothy.arcero@collabora.com>
2016-09-09 07:48:51 +01:00
struct brw_wm_prog_data *prog_data = brw_wm_prog_data(this->prog_data);
i965/fs: Remove direct fs_generator brw_wm_prog_data dependence Instead we store a brw_stage_prog_data pointer, and cast it to brw_wm_prog_data for fragment shader specific code paths. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-29 20:50:46 +01:00
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen < 6) {
i965: Fix Haswell discard regressions since Gen4-5 line AA fix. In commit dc2d3a7f5c217a7cee92380fbf503924a9591bea, Iago accidentally moved fire_fb_write() above the brw_pop_insn_state(), which caused the SEND to lose its predication and change from WE_normal to WE_all. Haswell uses predicated SENDs for discards, so this broke Piglit's tests for discards. We want the Gen4-5 MOV to be uncompressed, unpredicated, and unmasked, but the actual FB write itself should respect those. So, pop state first, and force it again around the single MOV. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=79903
2014-06-11 02:50:03 +01:00
brw_push_insn_state(p);
i965: Replace guess_execution_size with something simpler. guess_execution_size() does two things: 1. Cope with small destination registers. 2. Cope with SIMD8 vs SIMD16 mode. This patch replaces the first with a simple if block in brw_set_dest: if the destination register width is less than 8, you probably want the execution size to match. (I didn't put this in the 3src block because it doesn't seem to matter.) Since only the FS compiler cares about SIMD16 mode, it's easy to just set the default execution size there. This pattern was already been proven in the Gen8+ generator, but we didn't port it back to the existing generator when we combined the two. This is based on a patch from Ken from about a year ago. I've rebased it and and fixed a few bugs. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 20:40:34 +01:00
brw_set_default_exec_size(p, BRW_EXECUTE_8);
i965: Fix Haswell discard regressions since Gen4-5 line AA fix. In commit dc2d3a7f5c217a7cee92380fbf503924a9591bea, Iago accidentally moved fire_fb_write() above the brw_pop_insn_state(), which caused the SEND to lose its predication and change from WE_normal to WE_all. Haswell uses predicated SENDs for discards, so this broke Piglit's tests for discards. We want the Gen4-5 MOV to be uncompressed, unpredicated, and unmasked, but the actual FB write itself should respect those. So, pop state first, and force it again around the single MOV. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=79903
2014-06-11 02:50:03 +01:00
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
intel/fs: Pull FB write implied headers from src[0] Now that we have the implied header in src[0] for tracking purposes, we may as well use it in the generator. This makes things a tiny bit more general. Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-18 02:47:19 +01:00
brw_MOV(p, offset(retype(payload, BRW_REGISTER_TYPE_UD), 1),
offset(retype(implied_header, BRW_REGISTER_TYPE_UD), 1));
i965: Fix Haswell discard regressions since Gen4-5 line AA fix. In commit dc2d3a7f5c217a7cee92380fbf503924a9591bea, Iago accidentally moved fire_fb_write() above the brw_pop_insn_state(), which caused the SEND to lose its predication and change from WE_normal to WE_all. Haswell uses predicated SENDs for discards, so this broke Piglit's tests for discards. We want the Gen4-5 MOV to be uncompressed, unpredicated, and unmasked, but the actual FB write itself should respect those. So, pop state first, and force it again around the single MOV. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=79903
2014-06-11 02:50:03 +01:00
brw_pop_insn_state(p);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
}
intel/fs: Fix FB write message control codegen for SIMD32. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 22:22:19 +00:00
if (inst->opcode == FS_OPCODE_REP_FB_WRITE) {
assert(inst->group == 0 && inst->exec_size == 16);
i965: Add optimization pass to let us use the replicate data message The data port has a SIMD16 'replicate data' message, which lets us write the same color for all 16 pixels by sending the four floats in the lower half of a register instead of sending 4 times 16 identical component values in 8 registers. The message comes with a lot of restrictions and could be made generally useful by recognizing when those restriction are satisfied. For now, this lets us enable the optimization when we know it's safe, but we don't enable it by default. The optimization works for simple color clear shaders only, but does recognized and support multiple render targets. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2014-07-07 23:27:17 +01:00
msg_control = BRW_DATAPORT_RENDER_TARGET_WRITE_SIMD16_SINGLE_SOURCE_REPLICATED;
intel/fs: Fix FB write message control codegen for SIMD32. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 22:22:19 +00:00
} else if (prog_data->dual_src_blend) {
assert(inst->exec_size == 8);
if (inst->group % 16 == 0)
i965/fs: Implement SIMD16 dual source blending. From the SNB PRM, volume 4, part 1, page 193: "The dual source render target messages only have SIMD8 forms due to maximum message length limitations. SIMD16 pixel shaders must send two of these messages to cover all of the pixels. Each message contains two colors (4 channels each) for each pixel in the message payload." Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82831 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-03-05 08:43:38 +00:00
msg_control = BRW_DATAPORT_RENDER_TARGET_WRITE_SIMD8_DUAL_SOURCE_SUBSPAN01;
intel/fs: Fix FB write message control codegen for SIMD32. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 22:22:19 +00:00
else if (inst->group % 16 == 8)
i965/fs: Implement SIMD16 dual source blending. From the SNB PRM, volume 4, part 1, page 193: "The dual source render target messages only have SIMD8 forms due to maximum message length limitations. SIMD16 pixel shaders must send two of these messages to cover all of the pixels. Each message contains two colors (4 channels each) for each pixel in the message payload." Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=82831 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-03-05 08:43:38 +00:00
msg_control = BRW_DATAPORT_RENDER_TARGET_WRITE_SIMD8_DUAL_SOURCE_SUBSPAN23;
intel/fs: Fix FB write message control codegen for SIMD32. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 22:22:19 +00:00
else
unreachable("Invalid dual-source FB write instruction group");
} else {
assert(inst->group == 0 || (inst->group == 16 && inst->exec_size == 16));
if (inst->exec_size == 16)
msg_control = BRW_DATAPORT_RENDER_TARGET_WRITE_SIMD16_SINGLE_SOURCE;
else if (inst->exec_size == 8)
msg_control = BRW_DATAPORT_RENDER_TARGET_WRITE_SIMD8_SINGLE_SOURCE_SUBSPAN01;
else
unreachable("Invalid FB write execution size");
}
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
i965: Stop using wm_prog_data->binding_table.render_target_start. Render target surfaces always start at binding table index 0. This is required for us to use headerless FB writes, which we really want to do. So, we'll never change that. Given that, it's not necessary to look up a wm_prog_data field which we already know contains 0. We can drop the dependency in brw_renderbuffer_surfaces (Gen4-5)...which was already confusingly missing from gen6_renderbuffer_surfaces. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2017-08-17 20:09:41 +01:00
/* We assume render targets start at 0, because headerless FB write
* messages set "Render Target Index" to 0. Using a different binding
* table index would make it impossible to use headerless messages.
*/
const uint32_t surf_index = inst->target;
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
intel/fs: Fix FB write message control codegen for SIMD32. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-01-13 22:22:19 +00:00
brw_inst *insn = brw_fb_WRITE(p,
payload,
retype(implied_header, BRW_REGISTER_TYPE_UW),
msg_control,
surf_index,
nr,
0,
inst->eot,
inst->last_rt,
inst->header_size != 0);
if (devinfo->gen >= 6)
brw_inst_set_rt_slot_group(devinfo, insn, inst->group / 16);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
void
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
fs_generator::generate_fb_write(fs_inst *inst, struct brw_reg payload)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen < 8 && !devinfo->is_haswell) {
i965/fs: Treat the FB_WRITE as predicated if we're discarding. Pre-Haswell hardware couldn't actually predicate it, but it's easier to pretend as if it's predicated in the visitor since it will generate a MOV from f0.1. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-12 02:02:23 +00:00
brw_set_default_predicate_control(p, BRW_PREDICATE_NONE);
}
intel/fs: Pull FB write implied headers from src[0] Now that we have the implied header in src[0] for tracking purposes, we may as well use it in the generator. This makes things a tiny bit more general. Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-18 02:47:19 +01:00
const struct brw_reg implied_header =
devinfo->gen < 6 ? payload : brw_null_reg();
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
if (inst->base_mrf >= 0)
payload = brw_message_reg(inst->base_mrf);
i965/fs: Remove direct fs_generator brw_wm_prog_data dependence Instead we store a brw_stage_prog_data pointer, and cast it to brw_wm_prog_data for fragment shader specific code paths. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-29 20:50:46 +01:00
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
if (!runtime_check_aads_emit) {
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
fire_fb_write(inst, payload, implied_header, inst->mlen);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
} else {
/* This can only happen in gen < 6 */
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen < 6);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
struct brw_reg v1_null_ud = vec1(retype(brw_null_reg(), BRW_REGISTER_TYPE_UD));
/* Check runtime bit to detect if we have to send AA data or not */
intel/fs: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 20:07:00 +01:00
brw_push_insn_state(p);
intel/fs: Don't use automatic exec size inference The automatic exec size inference can accidentally mess things up if we're not careful. For instance, if we have add(4) g38.2<4>D g38.1<8,2,4>D g38.2<8,2,4>D then the destination register will end up having a width of 2 with a horizontal stride of 4 and a vertical stride of 8. The EU emit code sees the width of 2 and decides that we really wanted an exec size of 2 which doesn't do what we wanted. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-31 17:53:02 +01:00
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
brw_AND(p,
v1_null_ud,
retype(brw_vec1_grf(1, 6), BRW_REGISTER_TYPE_UD),
brw_imm_ud(1<<26));
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_cond_modifier(p->devinfo, brw_last_inst, BRW_CONDITIONAL_NZ);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
int jmp = brw_JMPI(p, brw_imm_ud(0), BRW_PREDICATE_NORMAL) - p->store;
intel/fs: Don't use automatic exec size inference The automatic exec size inference can accidentally mess things up if we're not careful. For instance, if we have add(4) g38.2<4>D g38.1<8,2,4>D g38.2<8,2,4>D then the destination register will end up having a width of 2 with a horizontal stride of 4 and a vertical stride of 8. The EU emit code sees the width of 2 and decides that we really wanted an exec size of 2 which doesn't do what we wanted. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-31 17:53:02 +01:00
brw_pop_insn_state(p);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
{
/* Don't send AA data */
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
fire_fb_write(inst, offset(payload, 1), implied_header, inst->mlen-1);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
}
brw_land_fwd_jump(p, jmp);
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
fire_fb_write(inst, payload, implied_header, inst->mlen);
i965/fs: Add Gen < 6 runtime checks for line antialiasing. In Gen < 6 the hardware generates a runtime bit that indicates whether AA data has to be sent as part of the framebuffer write SEND message. This affects the specific case where we have setup antialiased line rendering and we render polygons which have one face setup in GL_LINE mode (line antialiasing will be used) and the other one in GL_FILL mode (no line antialiasing needed). Currently we are not doing this runtime test and instead we always send AA data, which produces incorrect rendering of the GL_FILL face of the polygon in in the aforementioned scenario (verified in ironlake and gm45). In Gen4 this is, likely, a regression introduced with commit 098acf6c843. In Gen5 this has never worked properly. Gen > 5 are not affected by this. The patch fixes the problem by adding the appropriate runtime check and adjusting the framebuffer write message accordingly in the conflictive scenario. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=78679 Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-05 14:03:08 +01:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965/fs: Define framebuffer read virtual opcode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-22 00:52:33 +01:00
void
fs_generator::generate_fb_read(fs_inst *inst, struct brw_reg dst,
struct brw_reg payload)
{
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
assert(inst->size_written % REG_SIZE == 0);
i965: Introduce downcast helpers for prog_data structures. Similar to brw_context(...), intel_texture_object(...), and so on. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Timothy Arceri <timothy.arcero@collabora.com>
2016-09-09 07:48:51 +01:00
struct brw_wm_prog_data *prog_data = brw_wm_prog_data(this->prog_data);
i965: Drop render_target_start from binding table struct. We have to start render targets at binding table index 0 in order to use headerless FB write messages, and in fact already assume this in a bunch of places in the code. Let's finish that off, and not bother storing 0 in a struct to pretend to add it in a few places. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-01-03 07:59:06 +00:00
/* We assume that render targets start at binding table index 0. */
const unsigned surf_index = inst->target;
i965/fs: Define framebuffer read virtual opcode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-22 00:52:33 +01:00
gen9_fb_READ(p, dst, payload, surf_index,
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
inst->header_size, inst->size_written / REG_SIZE,
i965/fs: Define framebuffer read virtual opcode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-22 00:52:33 +01:00
prog_data->persample_dispatch);
}
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
void
fs_generator::generate_mov_indirect(fs_inst *inst,
struct brw_reg dst,
struct brw_reg reg,
struct brw_reg indirect_byte_offset)
{
assert(indirect_byte_offset.type == BRW_REGISTER_TYPE_UD);
assert(indirect_byte_offset.file == BRW_GENERAL_REGISTER_FILE);
intel/compiler: Add some restrictions to MOV_INDIRECT and BROADCAST These restrictions effectively already existed due to the way we use indirect sources but weren't being directly enforced. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-10-17 19:57:48 +01:00
assert(!reg.abs && !reg.negate);
assert(reg.type == dst.type);
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
unsigned imm_byte_offset = reg.nr * REG_SIZE + reg.subnr;
i965/fs: Add support for MOV_INDIRECT on pre-Broadwell hardware While we're at it, we also add support for the possibility that the indirect is, in fact, a constant. This shouldn't happen in the common case (if it does, that means NIR failed to constant-fold something), but it's possible so we should handle it. Reviewed-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-24 17:01:11 +00:00
if (indirect_byte_offset.file == BRW_IMMEDIATE_VALUE) {
imm_byte_offset += indirect_byte_offset.ud;
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
i965/fs: Add support for MOV_INDIRECT on pre-Broadwell hardware While we're at it, we also add support for the possibility that the indirect is, in fact, a constant. This shouldn't happen in the common case (if it does, that means NIR failed to constant-fold something), but it's possible so we should handle it. Reviewed-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-24 17:01:11 +00:00
reg.nr = imm_byte_offset / REG_SIZE;
reg.subnr = imm_byte_offset % REG_SIZE;
brw_MOV(p, dst, reg);
} else {
/* Prior to Broadwell, there are only 8 address registers. */
i965/fs: Get 64-bit indirect moves working on IVB. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2017-02-09 18:16:58 +00:00
assert(inst->exec_size <= 8 || devinfo->gen >= 8);
i965/fs: Add support for MOV_INDIRECT on pre-Broadwell hardware While we're at it, we also add support for the possibility that the indirect is, in fact, a constant. This shouldn't happen in the common case (if it does, that means NIR failed to constant-fold something), but it's possible so we should handle it. Reviewed-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-24 17:01:11 +00:00
/* We use VxH indirect addressing, clobbering a0.0 through a0.7. */
struct brw_reg addr = vec8(brw_address_reg(0));
/* The destination stride of an instruction (in bytes) must be greater
* than or equal to the size of the rest of the instruction. Since the
* address register is of type UW, we can't use a D-type instruction.
* In order to get around this, re retype to UW and use a stride.
*/
indirect_byte_offset =
retype(spread(indirect_byte_offset, 2), BRW_REGISTER_TYPE_UW);
i965/fs/generator: Don't use the address immediate for MOV_INDIRECT The address immediate field is only 9 bits and, since the value is in bytes, the highest GRF we can point to with it is g15. This makes it pretty close to useless for MOV_INDIRECT. There were already piles of restrictions preventing us from using it prior to Broadwell, so let's get rid of the gen8+ code path entirely. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97779 Cc: "12.0 13.0" <mesa-stable@lists.freedesktop.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-10-28 22:48:53 +01:00
/* There are a number of reasons why we don't use the base offset here.
* One reason is that the field is only 9 bits which means we can only
* use it to access the first 16 GRFs. Also, from the Haswell PRM
* section "Register Region Restrictions":
*
* "The lower bits of the AddressImmediate must not overflow to
* change the register address. The lower 5 bits of Address
* Immediate when added to lower 5 bits of address register gives
* the sub-register offset. The upper bits of Address Immediate
* when added to upper bits of address register gives the register
* address. Any overflow from sub-register offset is dropped."
*
* Since the indirect may cause us to cross a register boundary, this
* makes the base offset almost useless. We could try and do something
* clever where we use a actual base offset if base_offset % 32 == 0 but
* that would mean we were generating different code depending on the
* base offset. Instead, for the sake of consistency, we'll just do the
* add ourselves. This restriction is only listed in the Haswell PRM
* but empirical testing indicates that it applies on all older
* generations and is lifted on Broadwell.
*
* In the end, while base_offset is nice to look at in the generated
* code, using it saves us 0 instructions and would require quite a bit
* of case-by-case work. It's just not worth it.
*/
intel/fs: Fix MOV_INDIRECT for 64-bit values on little-core The same workaround we need for 64-bit values on little core also takes care of the Ivy Bridge problem and does so a bit more efficiently so we can drop that code while we're here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Cc: mesa-stable@lists.freedesktop.org
2017-10-17 22:45:43 +01:00
brw_ADD(p, addr, indirect_byte_offset, brw_imm_uw(imm_byte_offset));
if (type_sz(reg.type) > 4 &&
((devinfo->gen == 7 && !devinfo->is_haswell) ||
intel/compiler: Split 64-bit MOV-indirects if needed Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-11-27 19:20:20 +00:00
devinfo->is_cherryview || gen_device_info_is_9lp(devinfo) ||
!devinfo->has_64bit_types)) {
intel/fs: Fix MOV_INDIRECT for 64-bit values on little-core The same workaround we need for 64-bit values on little core also takes care of the Ivy Bridge problem and does so a bit more efficiently so we can drop that code while we're here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Cc: mesa-stable@lists.freedesktop.org
2017-10-17 22:45:43 +01:00
/* IVB has an issue (which we found empirically) where it reads two
* address register components per channel for indirectly addressed
* 64-bit sources.
*
* From the Cherryview PRM Vol 7. "Register Region Restrictions":
*
* "When source or destination datatype is 64b or operation is
* integer DWord multiply, indirect addressing must not be used."
*
* To work around both of these, we do two integer MOVs insead of one
* 64-bit MOV. Because no double value should ever cross a register
* boundary, it's safe to use the immediate offset in the indirect
* here to handle adding 4 bytes to the offset and avoid the extra
* ADD to the register file.
i965/fs: Get 64-bit indirect moves working on IVB. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2017-02-09 18:16:58 +00:00
*/
intel/fs: Fix MOV_INDIRECT for 64-bit values on little-core The same workaround we need for 64-bit values on little core also takes care of the Ivy Bridge problem and does so a bit more efficiently so we can drop that code while we're here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Cc: mesa-stable@lists.freedesktop.org
2017-10-17 22:45:43 +01:00
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 0),
retype(brw_VxH_indirect(0, 0), BRW_REGISTER_TYPE_D));
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 1),
retype(brw_VxH_indirect(0, 4), BRW_REGISTER_TYPE_D));
} else {
struct brw_reg ind_src = brw_VxH_indirect(0, 0);
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
intel/fs: Fix MOV_INDIRECT for 64-bit values on little-core The same workaround we need for 64-bit values on little core also takes care of the Ivy Bridge problem and does so a bit more efficiently so we can drop that code while we're here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Cc: mesa-stable@lists.freedesktop.org
2017-10-17 22:45:43 +01:00
brw_inst *mov = brw_MOV(p, dst, retype(ind_src, reg.type));
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
intel/fs: Fix MOV_INDIRECT for 64-bit values on little-core The same workaround we need for 64-bit values on little core also takes care of the Ivy Bridge problem and does so a bit more efficiently so we can drop that code while we're here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Cc: mesa-stable@lists.freedesktop.org
2017-10-17 22:45:43 +01:00
if (devinfo->gen == 6 && dst.file == BRW_MESSAGE_REGISTER_FILE &&
!inst->get_next()->is_tail_sentinel() &&
((fs_inst *)inst->get_next())->mlen > 0) {
/* From the Sandybridge PRM:
*
* "[Errata: DevSNB(SNB)] If MRF register is updated by any
* instruction that “indexed/indirect” source AND is followed
* by a send, the instruction requires a “Switch”. This is to
* avoid race condition where send may dispatch before MRF is
* updated."
*/
brw_inst_set_thread_control(devinfo, mov, BRW_THREAD_SWITCH);
}
i965/fs: Add support for MOV_INDIRECT on pre-Broadwell hardware While we're at it, we also add support for the possibility that the indirect is, in fact, a constant. This shouldn't happen in the common case (if it does, that means NIR failed to constant-fold something), but it's possible so we should handle it. Reviewed-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-24 17:01:11 +00:00
}
}
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
}
i965/fs: Add support for nir_intrinsic_shuffle Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-29 17:21:32 +01:00
void
fs_generator::generate_shuffle(fs_inst *inst,
struct brw_reg dst,
struct brw_reg src,
struct brw_reg idx)
{
/* Ivy bridge has some strange behavior that makes this a real pain to
* implement for 64-bit values so we just don't bother.
*/
assert(devinfo->gen >= 8 || devinfo->is_haswell || type_sz(src.type) <= 4);
/* Because we're using the address register, we're limited to 8-wide
* execution on gen7. On gen8, we're limited to 16-wide by the address
* register file and 8-wide for 64-bit types. We could try and make this
* instruction splittable higher up in the compiler but that gets weird
* because it reads all of the channels regardless of execution size. It's
* easier just to split it here.
*/
const unsigned lower_width =
(devinfo->gen <= 7 || type_sz(src.type) > 4) ?
8 : MIN2(16, inst->exec_size);
brw_set_default_exec_size(p, cvt(lower_width) - 1);
for (unsigned group = 0; group < inst->exec_size; group += lower_width) {
brw_set_default_group(p, group);
if ((src.vstride == 0 && src.hstride == 0) ||
idx.file == BRW_IMMEDIATE_VALUE) {
/* Trivial, the source is already uniform or the index is a constant.
* We will typically not get here if the optimizer is doing its job,
* but asserting would be mean.
*/
const unsigned i = idx.file == BRW_IMMEDIATE_VALUE ? idx.ud : 0;
brw_MOV(p, suboffset(dst, group), stride(suboffset(src, i), 0, 1, 0));
} else {
/* We use VxH indirect addressing, clobbering a0.0 through a0.7. */
struct brw_reg addr = vec8(brw_address_reg(0));
struct brw_reg group_idx = suboffset(idx, group);
if (lower_width == 8 && group_idx.width == BRW_WIDTH_16) {
/* Things get grumpy if the register is too wide. */
group_idx.width--;
group_idx.vstride--;
}
assert(type_sz(group_idx.type) <= 4);
if (type_sz(group_idx.type) == 4) {
/* The destination stride of an instruction (in bytes) must be
* greater than or equal to the size of the rest of the
* instruction. Since the address register is of type UW, we
* can't use a D-type instruction. In order to get around this,
* re retype to UW and use a stride.
*/
group_idx = retype(spread(group_idx, 2), BRW_REGISTER_TYPE_W);
}
/* Take into account the component size and horizontal stride. */
assert(src.vstride == src.hstride + src.width);
brw_SHL(p, addr, group_idx,
brw_imm_uw(_mesa_logbase2(type_sz(src.type)) +
src.hstride - 1));
/* Add on the register start offset */
brw_ADD(p, addr, addr, brw_imm_uw(src.nr * REG_SIZE + src.subnr));
if (type_sz(src.type) > 4 &&
((devinfo->gen == 7 && !devinfo->is_haswell) ||
devinfo->is_cherryview || gen_device_info_is_9lp(devinfo))) {
/* IVB has an issue (which we found empirically) where it reads
* two address register components per channel for indirectly
* addressed 64-bit sources.
*
* From the Cherryview PRM Vol 7. "Register Region Restrictions":
*
* "When source or destination datatype is 64b or operation is
* integer DWord multiply, indirect addressing must not be
* used."
*
* To work around both of these, we do two integer MOVs insead of
* one 64-bit MOV. Because no double value should ever cross a
* register boundary, it's safe to use the immediate offset in the
* indirect here to handle adding 4 bytes to the offset and avoid
* the extra ADD to the register file.
*/
struct brw_reg gdst = suboffset(dst, group);
struct brw_reg dst_d = retype(spread(gdst, 2),
BRW_REGISTER_TYPE_D);
brw_MOV(p, dst_d,
retype(brw_VxH_indirect(0, 0), BRW_REGISTER_TYPE_D));
brw_MOV(p, byte_offset(dst_d, 4),
retype(brw_VxH_indirect(0, 4), BRW_REGISTER_TYPE_D));
} else {
brw_MOV(p, suboffset(dst, group),
retype(brw_VxH_indirect(0, 0), src.type));
}
}
}
}
intel/fs: Implement quad swizzles on ICL+. Align16 is no longer a thing, so a new implementation is provided using Align1 instead. Not all possible swizzles can be represented as a single Align1 region, but some fast paths are provided for frequently used swizzles that can be represented efficiently in Align1 mode. Fixes ~90 subgroup quad swap Vulkan CTS tests. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-12-06 22:11:34 +00:00
void
fs_generator::generate_quad_swizzle(const fs_inst *inst,
struct brw_reg dst, struct brw_reg src,
unsigned swiz)
{
/* Requires a quad. */
assert(inst->exec_size >= 4);
if (src.file == BRW_IMMEDIATE_VALUE ||
has_scalar_region(src)) {
/* The value is uniform across all channels */
brw_MOV(p, dst, src);
} else if (devinfo->gen < 11 && type_sz(src.type) == 4) {
/* This only works on 8-wide 32-bit values */
assert(inst->exec_size == 8);
assert(src.hstride == BRW_HORIZONTAL_STRIDE_1);
assert(src.vstride == src.width + 1);
brw_set_default_access_mode(p, BRW_ALIGN_16);
struct brw_reg swiz_src = stride(src, 4, 4, 1);
swiz_src.swizzle = swiz;
brw_MOV(p, dst, swiz_src);
} else {
assert(src.hstride == BRW_HORIZONTAL_STRIDE_1);
assert(src.vstride == src.width + 1);
const struct brw_reg src_0 = suboffset(src, BRW_GET_SWZ(swiz, 0));
switch (swiz) {
case BRW_SWIZZLE_XXXX:
case BRW_SWIZZLE_YYYY:
case BRW_SWIZZLE_ZZZZ:
case BRW_SWIZZLE_WWWW:
brw_MOV(p, dst, stride(src_0, 4, 4, 0));
break;
case BRW_SWIZZLE_XXZZ:
case BRW_SWIZZLE_YYWW:
brw_MOV(p, dst, stride(src_0, 2, 2, 0));
break;
case BRW_SWIZZLE_XYXY:
case BRW_SWIZZLE_ZWZW:
assert(inst->exec_size == 4);
brw_MOV(p, dst, stride(src_0, 0, 2, 1));
break;
default:
assert(inst->force_writemask_all);
brw_set_default_exec_size(p, cvt(inst->exec_size / 4) - 1);
for (unsigned c = 0; c < 4; c++) {
brw_inst *insn = brw_MOV(
p, stride(suboffset(dst, c),
4 * inst->dst.stride, 1, 4 * inst->dst.stride),
stride(suboffset(src, BRW_GET_SWZ(swiz, c)), 4, 1, 0));
brw_inst_set_no_dd_clear(devinfo, insn, c < 3);
brw_inst_set_no_dd_check(devinfo, insn, c > 0);
}
break;
}
}
}
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
void
fs_generator::generate_urb_read(fs_inst *inst,
struct brw_reg dst,
struct brw_reg header)
{
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
assert(inst->size_written % REG_SIZE == 0);
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
assert(header.file == BRW_GENERAL_REGISTER_FILE);
assert(header.type == BRW_REGISTER_TYPE_UD);
brw_inst *send = brw_next_insn(p, BRW_OPCODE_SEND);
i965: Defeat the register stride checker in URB reads. Pulling DF inputs from the URB generates messages like: send(8) g23<1>DF g1<8,8,1>UD urb 3 SIMD8 read mlen 1 rlen 2 { align1 1Q }; which makes the simulator angry: "For 64-bit Align1 operation or multiplication of dwords in CHV, source horizontal stride must be aligned to qword." This seems to be documented in the Cherryview PRM, Volume 7, Page 823: "When source or destination datatype is 64b or operation is integer DWord multiply, regioning in Align1 must follow these rules: 1. Source and Destination horizontal stride must be aligned to the same qword." Setting the source horizontal stride to QWord is insane, as it's the message header containing 8 URB handles in a single 32-bit DWord. Instead, we should whack the destination type to UD, D, or F so that the register stride checker doesn't notice. The destination type of send messages is basically irrelevant anyway. Cc: "12.0" <mesa-stable@lists.freedesktop.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95462 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2016-06-09 00:24:50 +01:00
brw_set_dest(p, send, retype(dst, BRW_REGISTER_TYPE_UD));
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
brw_set_src0(p, send, header);
brw_set_src1(p, send, brw_imm_ud(0u));
brw_inst_set_sfid(p->devinfo, send, BRW_SFID_URB);
brw_inst_set_urb_opcode(p->devinfo, send, GEN8_URB_OPCODE_SIMD8_READ);
i965: Add a SHADER_OPCODE_URB_READ_SIMD8_PER_SLOT opcode. We need to use per-slot offsets when there's non-uniform indexing, as each SIMD channel could have a different index. We want to use them for any non-constant index (even if uniform), as it lives in the message header instead of the descriptor, allowing us to set offsets in GRFs rather than immediates. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
2015-11-07 09:37:33 +00:00
if (inst->opcode == SHADER_OPCODE_URB_READ_SIMD8_PER_SLOT)
brw_inst_set_urb_per_slot_offset(p->devinfo, send, true);
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
brw_inst_set_mlen(p->devinfo, send, inst->mlen);
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
brw_inst_set_rlen(p->devinfo, send, inst->size_written / REG_SIZE);
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
brw_inst_set_header_present(p->devinfo, send, true);
brw_inst_set_urb_global_offset(p->devinfo, send, inst->offset);
}
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
void
fs_generator::generate_urb_write(fs_inst *inst, struct brw_reg payload)
{
brw_inst *insn;
intel/compiler: Implement WaClearTDRRegBeforeEOTForNonPS. The bspec describes: "WA: Clear tdr register before send EOT in all non-PS shader kernels mov(8) tdr0:ud 0x0:ud {NoMask}" Signed-off-by: Rafael Antognolli <rafael.antognolli@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-10-06 19:41:54 +01:00
/* WaClearTDRRegBeforeEOTForNonPS.
*
* WA: Clear tdr register before send EOT in all non-PS shader kernels
*
* mov(8) tdr0:ud 0x0:ud {NoMask}"
*/
if (inst->eot && p->devinfo->gen == 10) {
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_MOV(p, brw_tdr_reg(), brw_imm_uw(0));
brw_pop_insn_state(p);
}
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
insn = brw_next_insn(p, BRW_OPCODE_SEND);
brw_set_dest(p, insn, brw_null_reg());
brw_set_src0(p, insn, payload);
intel/compiler: Change src1 reg type to unsigned doubleword To have uniform behavior while disassembling send(c) instruction use register type of unsigned doubleword for src1 when message descriptor is immediate value. Bspec does not specifiy anything for src1 immediate default type. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
2018-10-20 02:25:23 +01:00
brw_set_src1(p, insn, brw_imm_ud(0u));
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_sfid(p->devinfo, insn, BRW_SFID_URB);
brw_inst_set_urb_opcode(p->devinfo, insn, GEN8_URB_OPCODE_SIMD8_WRITE);
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
i965: Introduce new SHADER_OPCODE_URB_WRITE_SIMD8_MASKED/PER_SLOT opcodes. In the vec4 backend, we have a vec4_instruction::urb_write_flags field. There are many kinds of flags for SIMD4x2 messages. However, there are really only two (per-slot offset, use channel masks) for SIMD8 messages. Rather than adding a boolean flag for per-slot offsets (polluting all instructions), I decided to just make three new opcodes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-05-06 08:04:10 +01:00
if (inst->opcode == SHADER_OPCODE_URB_WRITE_SIMD8_PER_SLOT ||
inst->opcode == SHADER_OPCODE_URB_WRITE_SIMD8_MASKED_PER_SLOT)
brw_inst_set_urb_per_slot_offset(p->devinfo, insn, true);
if (inst->opcode == SHADER_OPCODE_URB_WRITE_SIMD8_MASKED ||
inst->opcode == SHADER_OPCODE_URB_WRITE_SIMD8_MASKED_PER_SLOT)
brw_inst_set_urb_channel_mask_present(p->devinfo, insn, true);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_mlen(p->devinfo, insn, inst->mlen);
brw_inst_set_rlen(p->devinfo, insn, 0);
brw_inst_set_eot(p->devinfo, insn, inst->eot);
brw_inst_set_header_present(p->devinfo, insn, true);
brw_inst_set_urb_global_offset(p->devinfo, insn, inst->offset);
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
}
i965/cs: Add generator support for CS_OPCODE_CS_TERMINATE v2: * Don't rely on brw_eu* to generate the send instruction. We now generate the send here, and drop the "i965/cs: Add support for the SEND message that terminates a CS thread" brw_eu* patch. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-27 19:33:25 +01:00
void
fs_generator::generate_cs_terminate(fs_inst *inst, struct brw_reg payload)
{
struct brw_inst *insn;
insn = brw_next_insn(p, BRW_OPCODE_SEND);
i965: Set dest type to UW for several send messages Without this, on SIMD 16 the send instruction destination will appear to write more than one destination register, causing the simulator to report an error. Of course, the send instruction can actually write more than one destination register regardless of the type set for the destination, so this is a bit strange. Suggested-by: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-02-01 02:28:42 +00:00
brw_set_dest(p, insn, retype(brw_null_reg(), BRW_REGISTER_TYPE_UW));
i965: Use a UW source type for CS_OPCODE_CS_TERMINATE. SIMD16 compute shaders use a send(16) with mlen 1 for the EOT message, using a source of g127 for the single register. With a UD type, this supposedly could read g128, which doesn't exist, causing the simulator to get cranky. Use a UW type to avoid this. Cc: "17.0" <mesa-stable@lists.freedesktop.org> Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2017-01-24 08:45:53 +00:00
brw_set_src0(p, insn, retype(payload, BRW_REGISTER_TYPE_UW));
intel/compiler: Change src1 reg type to unsigned doubleword To have uniform behavior while disassembling send(c) instruction use register type of unsigned doubleword for src1 when message descriptor is immediate value. Bspec does not specifiy anything for src1 immediate default type. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
2018-10-20 02:25:23 +01:00
brw_set_src1(p, insn, brw_imm_ud(0u));
i965/cs: Add generator support for CS_OPCODE_CS_TERMINATE v2: * Don't rely on brw_eu* to generate the send instruction. We now generate the send here, and drop the "i965/cs: Add support for the SEND message that terminates a CS thread" brw_eu* patch. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-27 19:33:25 +01:00
/* Terminate a compute shader by sending a message to the thread spawner.
*/
brw_inst_set_sfid(devinfo, insn, BRW_SFID_THREAD_SPAWNER);
brw_inst_set_mlen(devinfo, insn, 1);
brw_inst_set_rlen(devinfo, insn, 0);
brw_inst_set_eot(devinfo, insn, inst->eot);
brw_inst_set_header_present(devinfo, insn, false);
brw_inst_set_ts_opcode(devinfo, insn, 0); /* Dereference resource */
brw_inst_set_ts_request_type(devinfo, insn, 0); /* Root thread */
/* Note that even though the thread has a URB resource associated with it,
* we set the "do not dereference URB" bit, because the URB resource is
* managed by the fixed-function unit, so it will free it automatically.
*/
brw_inst_set_ts_resource_select(devinfo, insn, 1); /* Do not dereference URB */
brw_inst_set_mask_control(devinfo, insn, BRW_MASK_DISABLE);
}
i965/fs: Implement support for ir_barrier Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-27 19:32:08 +01:00
void
intel/compiler: Silence unused parameter warnings in generate_foo methods Since all of the fs_generator::generate_foo methods take a fs_inst * as the first parameter, just remove the name to quiet the compiler. src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_barrier(fs_inst*, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:743:41: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_barrier(fs_inst *inst, struct brw_reg src) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_discard_jump(fs_inst*)’: src/intel/compiler/brw_fs_generator.cpp:1326:46: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_discard_jump(fs_inst *inst) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_pack_half_2x16_split(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1675:54: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_pack_half_2x16_split(fs_inst *inst, ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_shader_time_add(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1743:49: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_shader_time_add(fs_inst *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_set_simd4x2_header_gen9(brw_codegen*, brw::vec4_instruction*, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1412:52: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_mov_indirect(brw_codegen*, brw::vec4_instruction*, brw_reg, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1430:41: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp:1432:63: warning: unused parameter ‘length’ [-Wunused-parameter] struct brw_reg indirect, struct brw_reg length) ^~~~~~ Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2018-03-29 00:29:45 +01:00
fs_generator::generate_barrier(fs_inst *, struct brw_reg src)
i965/fs: Implement support for ir_barrier Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-27 19:32:08 +01:00
{
brw_barrier(p, src);
brw_WAIT(p);
}
intel/compiler/fs: Return multiple_instructions_emitted from generate_linterp If multiple instructions are emitted, special handling of things like conditional mod and NoDDClr/NoDDChk need to be performed. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 19:06:45 +01:00
bool
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
fs_generator::generate_linterp(fs_inst *inst,
intel/compiler/fs: Return multiple_instructions_emitted from generate_linterp If multiple instructions are emitted, special handling of things like conditional mod and NoDDClr/NoDDChk need to be performed. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 19:06:45 +01:00
struct brw_reg dst, struct brw_reg *src)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965/fs: Calculate delta_x and delta_y together. This lets SIMD16 programs on G45 and Gen5 use the PLN instruction. On Ironlake: total instructions in shared programs: 5634757 -> 5518055 (-2.07%) instructions in affected programs: 1745837 -> 1629135 (-6.68%) helped: 11439 HURT: 4 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-07 01:44:40 +01:00
/* PLN reads:
* / in SIMD16 \
* -----------------------------------
* | src1+0 | src1+1 | src1+2 | src1+3 |
* |-----------------------------------|
* |(x0, x1)|(y0, y1)|(x2, x3)|(y2, y3)|
* -----------------------------------
*
* but for the LINE/MAC pair, the LINE reads Xs and the MAC reads Ys:
*
* -----------------------------------
* | src1+0 | src1+1 | src1+2 | src1+3 |
* |-----------------------------------|
* |(x0, x1)|(y0, y1)| | | in SIMD8
* |-----------------------------------|
* |(x0, x1)|(x2, x3)|(y0, y1)|(y2, y3)| in SIMD16
* -----------------------------------
*
* See also: emit_interpolation_setup_gen4().
*/
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
struct brw_reg delta_x = src[0];
i965/fs: Clean up remaining uses of dispatch_width in the generator. Most of these are bugs because the intended execution size of an instruction and the dispatch width of the shader aren't necessarily the same (especially in SIMD32 programs). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 08:13:33 +01:00
struct brw_reg delta_y = offset(src[0], inst->exec_size / 8);
i965/fs: Calculate delta_x and delta_y together. This lets SIMD16 programs on G45 and Gen5 use the PLN instruction. On Ironlake: total instructions in shared programs: 5634757 -> 5518055 (-2.07%) instructions in affected programs: 1745837 -> 1629135 (-6.68%) helped: 11439 HURT: 4 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-07 01:44:40 +01:00
struct brw_reg interp = src[1];
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
brw_inst *i[4];
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
if (devinfo->gen >= 11) {
struct brw_reg acc = retype(brw_acc_reg(8), BRW_REGISTER_TYPE_NF);
struct brw_reg dwP = suboffset(interp, 0);
struct brw_reg dwQ = suboffset(interp, 1);
struct brw_reg dwR = suboffset(interp, 3);
intel/fs: Use groups for SIMD16 LINTERP on gen11+ This is better than compression control because it naturally extends to SIMD32. v2: - Push/pop instruction state around adjusted codegen (Ken) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-17 01:33:17 +01:00
brw_push_insn_state(p);
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
brw_set_default_exec_size(p, BRW_EXECUTE_8);
if (inst->exec_size == 8) {
i[0] = brw_MAD(p, acc, dwR, offset(delta_x, 0), dwP);
i[1] = brw_MAD(p, offset(dst, 0), acc, offset(delta_y, 0), dwQ);
brw_inst_set_cond_modifier(p->devinfo, i[1], inst->conditional_mod);
/* brw_set_default_saturate() is called before emitting instructions,
* so the saturate bit is set in each instruction, so we need to unset
* it on the first instruction of each pair.
*/
brw_inst_set_saturate(p->devinfo, i[0], false);
} else {
intel/fs: Use groups for SIMD16 LINTERP on gen11+ This is better than compression control because it naturally extends to SIMD32. v2: - Push/pop instruction state around adjusted codegen (Ken) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-17 01:33:17 +01:00
brw_set_default_group(p, inst->group);
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
i[0] = brw_MAD(p, acc, dwR, offset(delta_x, 0), dwP);
i[1] = brw_MAD(p, offset(dst, 0), acc, offset(delta_x, 1), dwQ);
intel/fs: Use groups for SIMD16 LINTERP on gen11+ This is better than compression control because it naturally extends to SIMD32. v2: - Push/pop instruction state around adjusted codegen (Ken) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-17 01:33:17 +01:00
brw_set_default_group(p, inst->group + 8);
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
i[2] = brw_MAD(p, acc, dwR, offset(delta_y, 0), dwP);
i[3] = brw_MAD(p, offset(dst, 1), acc, offset(delta_y, 1), dwQ);
brw_inst_set_cond_modifier(p->devinfo, i[1], inst->conditional_mod);
brw_inst_set_cond_modifier(p->devinfo, i[3], inst->conditional_mod);
/* brw_set_default_saturate() is called before emitting instructions,
* so the saturate bit is set in each instruction, so we need to unset
* it on the first instruction of each pair.
*/
brw_inst_set_saturate(p->devinfo, i[0], false);
brw_inst_set_saturate(p->devinfo, i[2], false);
}
intel/fs: Use groups for SIMD16 LINTERP on gen11+ This is better than compression control because it naturally extends to SIMD32. v2: - Push/pop instruction state around adjusted codegen (Ken) Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-17 01:33:17 +01:00
brw_pop_insn_state(p);
intel/compiler/fs: Implement FS_OPCODE_LINTERP with MADs on Gen11+ The PLN instruction is no more. Its functionality is now implemented using two MAD instructions with the new native-float type. Instead of pln(16) r20.0<1>:F r10.4<0;1,0>:F r4.0<8;8,1>:F we now have mad(8) acc0<1>:NF r10.7<0;1,0>:F r4.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r20.0<1>:F acc0<8;8,1>:NF r5.0<8;8,1>:F r10.5<0;1,0>:F mad(8) acc0<1>:NF r10.7<0;1,0>:F r6.0<8;8,1>:F r10.4<0;1,0>:F mad(8) r21.0<1>:F acc0<8;8,1>:NF r7.0<8;8,1>:F r10.5<0;1,0>:F ... and in the case of SIMD8 only the first pair of MAD instructions is used. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
return true;
intel/fs: Assert that the gen4-6 plane restrictions are followed The fall-back does not work correctly in SIMD16 mode and the register allocator should ensure that we never hit this case anyway. Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-17 01:30:04 +01:00
} else if (devinfo->has_pln) {
intel/fs: Emit LINE+MAC for LINTERP with unaligned coordinates On g4x through Sandy Bridge, src1 (the coordinates) of the PLN instruction is required to be an even register number. When it's odd (which can happen with SIMD32), we have to emit a LINE+MAC combination instead. Unfortunately, we can't just fall through to the gen4 case because the input registers are still set up for PLN which lays out the four src1 registers differently in SIMD16 than LINE. v2 (Jason Ekstrand): - Take advantage of both accumulators and emit LINE LINE MAC MAC (Based on a patch from Francisco Jerez) - Unify the gen4 and gen4x-6 cases using a loop v3 (Jason Ekstrand): - Don't unify gen4 with gen4x-6 as this turns out to be more fragile than first thought without reworking the gen4 barycentric coordinate layout. Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-26 06:23:30 +01:00
if (devinfo->gen <= 6 && (delta_x.nr & 1) != 0) {
/* From the Sandy Bridge PRM Vol. 4, Pt. 2, Section 8.3.53, "Plane":
*
* "[DevSNB]:<src1> must be even register aligned.
*
* This restriction is lifted on Ivy Bridge.
*
* This means that we need to split PLN into LINE+MAC on-the-fly.
* Unfortunately, the inputs are laid out for PLN and not LINE+MAC so
* we have to split into SIMD8 pieces. For gen4 (!has_pln), the
* coordinate registers are laid out differently so we leave it as a
* SIMD16 instruction.
*/
assert(inst->exec_size == 8 || inst->exec_size == 16);
assert(inst->group % 16 == 0);
intel/compiler/fs: Return multiple_instructions_emitted from generate_linterp If multiple instructions are emitted, special handling of things like conditional mod and NoDDClr/NoDDChk need to be performed. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 19:06:45 +01:00
intel/fs: Emit LINE+MAC for LINTERP with unaligned coordinates On g4x through Sandy Bridge, src1 (the coordinates) of the PLN instruction is required to be an even register number. When it's odd (which can happen with SIMD32), we have to emit a LINE+MAC combination instead. Unfortunately, we can't just fall through to the gen4 case because the input registers are still set up for PLN which lays out the four src1 registers differently in SIMD16 than LINE. v2 (Jason Ekstrand): - Take advantage of both accumulators and emit LINE LINE MAC MAC (Based on a patch from Francisco Jerez) - Unify the gen4 and gen4x-6 cases using a loop v3 (Jason Ekstrand): - Don't unify gen4 with gen4x-6 as this turns out to be more fragile than first thought without reworking the gen4 barycentric coordinate layout. Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-26 06:23:30 +01:00
brw_push_insn_state(p);
brw_set_default_exec_size(p, BRW_EXECUTE_8);
/* Thanks to two accumulators, we can emit all the LINEs and then all
* the MACs. This improves parallelism a bit.
*/
for (unsigned g = 0; g < inst->exec_size / 8; g++) {
brw_inst *line = brw_LINE(p, brw_null_reg(), interp,
offset(delta_x, g * 2));
brw_inst_set_group(devinfo, line, inst->group + g * 8);
/* LINE writes the accumulator automatically on gen4-5. On Sandy
* Bridge and later, we have to explicitly enable it.
*/
if (devinfo->gen >= 6)
brw_inst_set_acc_wr_control(p->devinfo, line, true);
/* brw_set_default_saturate() is called before emitting
* instructions, so the saturate bit is set in each instruction,
* so we need to unset it on the LINE instructions.
*/
brw_inst_set_saturate(p->devinfo, line, false);
}
for (unsigned g = 0; g < inst->exec_size / 8; g++) {
brw_inst *mac = brw_MAC(p, offset(dst, g), suboffset(interp, 1),
offset(delta_x, g * 2 + 1));
brw_inst_set_group(devinfo, mac, inst->group + g * 8);
brw_inst_set_cond_modifier(p->devinfo, mac, inst->conditional_mod);
}
brw_pop_insn_state(p);
return true;
} else {
brw_PLN(p, dst, interp, delta_x);
return false;
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
} else {
intel/compiler/fs: Fix application of cmod and saturate to LINE/MAC pair This isn't technically broken, but the next patch will make this function report whether it generated multiple instructions, and that information will be used to disable the application of conditional mod by the generic code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 22:47:19 +01:00
i[0] = brw_LINE(p, brw_null_reg(), interp, delta_x);
i[1] = brw_MAC(p, dst, suboffset(interp, 1), delta_y);
brw_inst_set_cond_modifier(p->devinfo, i[1], inst->conditional_mod);
/* brw_set_default_saturate() is called before emitting instructions, so
* the saturate bit is set in each instruction, so we need to unset it on
* the first instruction.
*/
brw_inst_set_saturate(p->devinfo, i[0], false);
intel/compiler/fs: Return multiple_instructions_emitted from generate_linterp If multiple instructions are emitted, special handling of things like conditional mod and NoDDClr/NoDDChk need to be performed. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 19:06:45 +01:00
return true;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
}
i965/fs: Implement FS_OPCODE_GET_BUFFER_SIZE Signed-off-by: Samuel Iglesias Gonsalvez <siglesias@igalia.com> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-04-13 15:55:49 +01:00
void
fs_generator::generate_get_buffer_size(fs_inst *inst,
struct brw_reg dst,
struct brw_reg src,
struct brw_reg surf_index)
{
assert(devinfo->gen >= 7);
assert(surf_index.file == BRW_IMMEDIATE_VALUE);
uint32_t simd_mode;
int rlen = 4;
switch (inst->exec_size) {
case 8:
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD8;
break;
case 16:
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
break;
default:
unreachable("Invalid width for texture instruction");
}
if (simd_mode == BRW_SAMPLER_SIMD_MODE_SIMD16) {
rlen = 8;
dst = vec16(dst);
}
brw_SAMPLE(p,
retype(dst, BRW_REGISTER_TYPE_UW),
inst->base_mrf,
src,
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 03:41:30 +01:00
surf_index.ud,
i965/fs: Implement FS_OPCODE_GET_BUFFER_SIZE Signed-off-by: Samuel Iglesias Gonsalvez <siglesias@igalia.com> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-04-13 15:55:49 +01:00
0,
GEN5_SAMPLER_MESSAGE_SAMPLE_RESINFO,
rlen, /* response length */
inst->mlen,
inst->header_size > 0,
simd_mode,
BRW_SAMPLER_RETURN_FORMAT_SINT32);
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
void
i965/fs/Gen4-7: Pass sampler_index to generate_tex Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-03 10:23:31 +01:00
fs_generator::generate_tex(fs_inst *inst, struct brw_reg dst, struct brw_reg src,
i965/fs: Separate the sampler from the surface in generate_tex Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 23:24:05 +00:00
struct brw_reg surface_index,
i965/fs/Gen4-7: Pass sampler_index to generate_tex Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-03 10:23:31 +01:00
struct brw_reg sampler_index)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
assert(inst->size_written % REG_SIZE == 0);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
int msg_type = -1;
i965/fs: Use the instruction execution size directly for texture generation Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-30 01:22:57 +01:00
uint32_t simd_mode;
i965/gen4: Fix sampling from integer textures. On original gen4, the surface format didn't determine the return data type from sampling like it does on g45 and later. Fixes GL_EXT_texture_integer/texture_integer_glsl130 Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-11-10 00:07:57 +00:00
uint32_t return_format;
i965/fs: Combine tex/fb_write operations (opt) Certain platforms support the ability to sample from a texture, and write it out to the file RT - thus saving a costly send instructions (note that this is a potnential win if one wanted to backport to a tag that didn't have the patch from Topi which removed excess MOVs from LOAD_PAYLOAD - 97caf5fa04dbd2), v2: Modify the algorithm. Instead of iterating in reverse through blocks and insts, since the last block/inst is the only thing which can benefit. Rebased on top of Ken's patching modifying is_last_send v3: Rebased over almost 2 months, and Incorporated feedback from Matt: Some comment typo fixes and rewordings. Whitespace Move the optimization pass outside of the optimize loop v4: Some cosmetic changes requested from Ken. These changes ensured that the optimization function always returned true when an optimization occurred, and false when one did not. This behavior did not exist with the original patch. As a result, having the separate helper function which Matt did not like no longer made sense, and so now I believe everyone should be happy. Benchmark (n=20) %diff *OglBatch5 -1.4 *OglBatch7 -1.79 OglFillTexMulti 5.57 OglFillTexSingle 1.16 OglShMapPcf 0.05 OglTexFilterAniso 3.01 OglTexFilterTri 1.94 No piglit regressions: (http://otc-gfxtest-01.jf.intel.com:8080/view/dev/job/bwidawsk/112/) [*] I believe my measurements are incorrect for Batch5-7. If I add this new optimization, but never emit the new instruction I see similar results. v5: Remove declaration of combine_tex_header since v4 dropped that function (Ben) Remove check for impossible case of an empty block (Matt) Set dest earlier to avoid extra special-casing in generate_tex (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-08 21:59:57 +00:00
bool is_combined_send = inst->eot;
i965/gen4: Fix sampling from integer textures. On original gen4, the surface format didn't determine the return data type from sampling like it does on g45 and later. Fixes GL_EXT_texture_integer/texture_integer_glsl130 Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-11-10 00:07:57 +00:00
intel/fs: Disable opt_sampler_eot() in 32-wide dispatch. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 01:08:42 +01:00
/* Sampler EOT message of less than the dispatch width would kill the
* thread prematurely.
*/
assert(!is_combined_send || inst->exec_size == dispatch_width);
i965/gen4: Fix sampling from integer textures. On original gen4, the surface format didn't determine the return data type from sampling like it does on g45 and later. Fixes GL_EXT_texture_integer/texture_integer_glsl130 Reviewed-by: Paul Berry <stereotype441@gmail.com>
2011-11-10 00:07:57 +00:00
switch (dst.type) {
case BRW_REGISTER_TYPE_D:
return_format = BRW_SAMPLER_RETURN_FORMAT_SINT32;
break;
case BRW_REGISTER_TYPE_UD:
return_format = BRW_SAMPLER_RETURN_FORMAT_UINT32;
break;
default:
return_format = BRW_SAMPLER_RETURN_FORMAT_FLOAT32;
break;
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965/fs: Stomp the texture return type to UINT32 for resinfo messages Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-11 23:46:55 +00:00
/* Stomp the resinfo output type to UINT32. On gens 4-5, the output type
* is set as part of the message descriptor. On gen4, the PRM seems to
* allow UINT32 and FLOAT32 (i965 PRM, Vol. 4 Section 4.8.1.1), but on
* later gens UINT32 is required. Once you hit Sandy Bridge, the bit is
* gone from the message descriptor entirely and you just get UINT32 all
* the time regasrdless. Since we can really only do non-UINT32 on gen4,
* just stomp it to UINT32 all the time.
*/
if (inst->opcode == SHADER_OPCODE_TXS)
return_format = BRW_SAMPLER_RETURN_FORMAT_UINT32;
i965/fs: Use the instruction execution size directly for texture generation Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-30 01:22:57 +01:00
switch (inst->exec_size) {
case 8:
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD8;
break;
case 16:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
i965/fs: Use the instruction execution size directly for texture generation Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-30 01:22:57 +01:00
break;
default:
unreachable("Invalid width for texture instruction");
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen >= 5) {
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
switch (inst->opcode) {
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TEX:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
if (inst->shadow_compare) {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_COMPARE;
} else {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE;
}
break;
case FS_OPCODE_TXB:
if (inst->shadow_compare) {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_BIAS_COMPARE;
} else {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_BIAS;
}
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXL:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
if (inst->shadow_compare) {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_LOD_COMPARE;
} else {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_LOD;
}
break;
i965: Add infrastucture for sample lod-zero operations. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-05-04 23:46:45 +01:00
case SHADER_OPCODE_TXL_LZ:
assert(devinfo->gen >= 9);
if (inst->shadow_compare) {
msg_type = GEN9_SAMPLER_MESSAGE_SAMPLE_C_LZ;
} else {
msg_type = GEN9_SAMPLER_MESSAGE_SAMPLE_LZ;
}
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXS:
intel: Use TXS for image_size when we have a typed surface Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-08-16 17:01:24 +01:00
case SHADER_OPCODE_IMAGE_SIZE:
i965/fs: Implement textureSize (TXS) on Gen5+. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2011-06-19 09:47:50 +01:00
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_RESINFO;
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXD:
i965: Use Haswell's sample_d_c for textureGrad with shadow samplers. The new hardware actually just supports this now. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-01-04 15:53:09 +00:00
if (inst->shadow_compare) {
i965: remove brw_lower_texture_gradients This has been ported to NIR now so we don'tneed to keep the GLSL IR lowering any more. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-11-30 10:45:59 +00:00
/* Gen7.5+. Otherwise, lowered in NIR */
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 8 || devinfo->is_haswell);
i965: Use Haswell's sample_d_c for textureGrad with shadow samplers. The new hardware actually just supports this now. Reviewed-by: Eric Anholt <eric@anholt.net>
2013-01-04 15:53:09 +00:00
msg_type = HSW_SAMPLER_MESSAGE_SAMPLE_DERIV_COMPARE;
} else {
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_DERIVS;
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXF:
i965/fs: Implement texelFetch() on Ironlake and Sandybridge. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-26 01:13:37 +01:00
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_LD;
break;
i965: Add infrastucture for sample lod-zero operations. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-05-04 23:46:45 +01:00
case SHADER_OPCODE_TXF_LZ:
assert(devinfo->gen >= 9);
msg_type = GEN9_SAMPLER_MESSAGE_SAMPLE_LD_LZ;
break;
i965/fs/skl+: Use ld2dms_w instead of ld2dms In order to support 16x MSAA, skl+ has a wider version of ld2dms that takes two parameters for the MCS data. The MCS data retrieved from the ld_mcs instruction already returns 4 or 8 registers and is documented to return zeroes for the mcsh value when the sample count is less than 16. v2: Use get_lowered_simd_width to fall back to SIMD8 instructions when the message length would be too long in SIMD16. Reviewed-by: Ben Widawsky <ben@bwidawsk.net>
2015-09-08 15:52:09 +01:00
case SHADER_OPCODE_TXF_CMS_W:
assert(devinfo->gen >= 9);
msg_type = GEN9_SAMPLER_MESSAGE_SAMPLE_LD2DMS_W;
break;
i965: rename tex_ms to tex_cms Prepares for the introduction of non-compressed multi-sampled lookup used in the blorp programs. v2: now also taking into account gen8 Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-10 14:36:31 +00:00
case SHADER_OPCODE_TXF_CMS:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen >= 7)
i965: add a new virtual opcode: SHADER_OPCODE_TXF_MS This is very similar to the TXF opcode, but lowers to `ld2dms` rather than `ld` on Gen7. V4: - add SHADER_OPCODE_TXF_MS to is_tex() functions, so regalloc thinks it actually writes the correct number of registers. Otherwise in nontrivial shaders some of the registers tend to get clobbered, producing bad results. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-01-24 08:35:15 +00:00
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_LD2DMS;
else
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_LD;
break;
i965/fs: introduce non-compressed equivalent of tex_cms v2: introduces 'SHADER_OPCODE_TXF_UMS' also for gen8 Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-10 14:38:15 +00:00
case SHADER_OPCODE_TXF_UMS:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: introduce non-compressed equivalent of tex_cms v2: introduces 'SHADER_OPCODE_TXF_UMS' also for gen8 Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-10 14:38:15 +00:00
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_LD2DSS;
break;
i965: Add shader opcode for sampling MCS surface Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Paul Berry <stereotype441@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-29 21:32:16 +00:00
case SHADER_OPCODE_TXF_MCS:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965: Add shader opcode for sampling MCS surface Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Paul Berry <stereotype441@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-29 21:32:16 +00:00
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_LD_MCS;
break;
i965/fs: Generate LOD sampler message from ir_lod. v2: Support Ironlake as well. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-06 22:47:01 +00:00
case SHADER_OPCODE_LOD:
msg_type = GEN5_SAMPLER_MESSAGE_LOD;
break;
i965: add SHADER_OPCODE_TG4 Adds the Gen7 message IDs, a new SHADER_OPCODE_TG4 pseudo-op, and low-level support for emitting it via generate_tex(). V3: Updated for changes in master. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-31 09:31:12 +01:00
case SHADER_OPCODE_TG4:
i965/fs: Add support for shadow comparitors with gather4 Note that gather4_po_c's parameters are too long for SIMD16. It might be worth emitting 2xSIMD8 messages in this case at some point. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-10-10 07:57:29 +01:00
if (inst->shadow_compare) {
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: Add support for shadow comparitors with gather4 Note that gather4_po_c's parameters are too long for SIMD16. It might be worth emitting 2xSIMD8 messages in this case at some point. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-10-10 07:57:29 +01:00
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_GATHER4_C;
} else {
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 6);
i965/fs: Add support for shadow comparitors with gather4 Note that gather4_po_c's parameters are too long for SIMD16. It might be worth emitting 2xSIMD8 messages in this case at some point. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-10-10 07:57:29 +01:00
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_GATHER4;
}
i965: add SHADER_OPCODE_TG4 Adds the Gen7 message IDs, a new SHADER_OPCODE_TG4 pseudo-op, and low-level support for emitting it via generate_tex(). V3: Updated for changes in master. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-31 09:31:12 +01:00
break;
i965: Add SHADER_OPCODE_TG4_OFFSET for gather with nonconstant offsets. The generator code ends up clearer this way than if we had to sniff via the message length. Implemented via the gather4_po message in hardware, which is present in Gen7 and later. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2013-10-08 09:42:10 +01:00
case SHADER_OPCODE_TG4_OFFSET:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: Add support for shadow comparitors with gather4 Note that gather4_po_c's parameters are too long for SIMD16. It might be worth emitting 2xSIMD8 messages in this case at some point. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-10-10 07:57:29 +01:00
if (inst->shadow_compare) {
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_GATHER4_PO_C;
} else {
msg_type = GEN7_SAMPLER_MESSAGE_SAMPLE_GATHER4_PO;
}
i965: Add SHADER_OPCODE_TG4_OFFSET for gather with nonconstant offsets. The generator code ends up clearer this way than if we had to sniff via the message length. Implemented via the gather4_po message in hardware, which is present in Gen7 and later. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2013-10-08 09:42:10 +01:00
break;
i965: add support for textureSamples function Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu> [v2: kayden-supplied code in fs_nir replacing need for logical opcode] Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-08-12 01:37:32 +01:00
case SHADER_OPCODE_SAMPLEINFO:
msg_type = GEN6_SAMPLER_MESSAGE_SAMPLE_SAMPLEINFO;
break;
i965: Create a shared enum for hardware and compiler-internal opcodes. This should make gdbing more pleasant, and it might be used in sharing part of the codegen between the VS and FS backends.
2011-05-03 18:55:50 +01:00
default:
i965: Use unreachable() instead of unconditional assert(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-29 22:54:01 +01:00
unreachable("not reached");
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
} else {
switch (inst->opcode) {
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TEX:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
/* Note that G45 and older determines shadow compare and dispatch width
* from message length for most messages.
*/
i965/fs: Use exec_size instead of dispatch_width to determine the message variant. dispatch_width is global for a single compilation and doesn't necessarily match the desired execution width if we had to lower the original full-width instruction due to hardware limitations. These were all inside a Gen4-specific branch so this patch shouldn't have any effect on more recent hardware. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-13 13:42:20 +01:00
if (inst->exec_size == 8) {
i965: Implement SIMD16 texturing on Gen4. This allows SIMD16 mode to work for a lot more programs. Texturing is also more efficient in SIMD16 mode than SIMD8. Several messages don't actually exist in SIMD8 mode, so we did SIMD16 messages and threw away half of the data. Now we compute real data in both halves. Also, the SIMD16 "sample" message doesn't require all three coordinate components to exist (like the SIMD8 one), so we can shorten the message lengths, cutting register usage a bit. I chose to implement the visitor functionality in a separate function, since mixing true SIMD16 with SIMD8 code that uses SIMD16 fallbacks seemed like a mess. The new code bails on a few cases where we'd have to do two SIMD8 messages - we just fall back to SIMD8 for now. Improves performance in "Shadowrun: Dragonfall - Director's Cut" by about 20% on GM45 (measured with LIBGL_SHOW_FPS=1 while standing around in the first mission). v2: Add ir_txf to the has_lod case (caught by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-02-20 23:11:49 +00:00
msg_type = BRW_SAMPLER_MESSAGE_SIMD8_SAMPLE;
if (inst->shadow_compare) {
assert(inst->mlen == 6);
} else {
assert(inst->mlen <= 4);
}
} else {
if (inst->shadow_compare) {
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_SAMPLE_COMPARE;
assert(inst->mlen == 9);
} else {
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_SAMPLE;
assert(inst->mlen <= 7 && inst->mlen % 2 == 1);
}
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
case FS_OPCODE_TXB:
if (inst->shadow_compare) {
i965/fs: Use exec_size instead of dispatch_width to determine the message variant. dispatch_width is global for a single compilation and doesn't necessarily match the desired execution width if we had to lower the original full-width instruction due to hardware limitations. These were all inside a Gen4-specific branch so this patch shouldn't have any effect on more recent hardware. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-13 13:42:20 +01:00
assert(inst->exec_size == 8);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
assert(inst->mlen == 6);
msg_type = BRW_SAMPLER_MESSAGE_SIMD8_SAMPLE_BIAS_COMPARE;
} else {
assert(inst->mlen == 9);
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_SAMPLE_BIAS;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
}
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXL:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
if (inst->shadow_compare) {
i965/fs: Use exec_size instead of dispatch_width to determine the message variant. dispatch_width is global for a single compilation and doesn't necessarily match the desired execution width if we had to lower the original full-width instruction due to hardware limitations. These were all inside a Gen4-specific branch so this patch shouldn't have any effect on more recent hardware. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-13 13:42:20 +01:00
assert(inst->exec_size == 8);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
assert(inst->mlen == 6);
msg_type = BRW_SAMPLER_MESSAGE_SIMD8_SAMPLE_LOD_COMPARE;
} else {
assert(inst->mlen == 9);
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_SAMPLE_LOD;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
}
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXD:
i965/fs: Add support for TXD with shadow comparisons. Our hardware doesn't have a sample_d_c message, so we have to do a regular sample_d and emit instructions to manually perform the comparison. This requires a state dependent recompile whenever the sampler's compare mode or function change. This adds the per-sampler comparison functions to brw_wm_prog_key, but only sets them when the sampler's compare mode is GL_COMPARE_R_TO_TEXTURE (i.e. only for shadow sampling). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-06-10 22:48:46 +01:00
/* There is no sample_d_c message; comparisons are done manually */
i965/fs: Use exec_size instead of dispatch_width to determine the message variant. dispatch_width is global for a single compilation and doesn't necessarily match the desired execution width if we had to lower the original full-width instruction due to hardware limitations. These were all inside a Gen4-specific branch so this patch shouldn't have any effect on more recent hardware. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-13 13:42:20 +01:00
assert(inst->exec_size == 8);
i965/fs: Add support for non-shadow textureGrad (TXD) on gen4. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-06-09 00:05:34 +01:00
assert(inst->mlen == 7 || inst->mlen == 10);
msg_type = BRW_SAMPLER_MESSAGE_SIMD8_SAMPLE_GRADIENTS;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXF:
i965: Implement SIMD16 texturing on Gen4. This allows SIMD16 mode to work for a lot more programs. Texturing is also more efficient in SIMD16 mode than SIMD8. Several messages don't actually exist in SIMD8 mode, so we did SIMD16 messages and threw away half of the data. Now we compute real data in both halves. Also, the SIMD16 "sample" message doesn't require all three coordinate components to exist (like the SIMD8 one), so we can shorten the message lengths, cutting register usage a bit. I chose to implement the visitor functionality in a separate function, since mixing true SIMD16 with SIMD8 code that uses SIMD16 fallbacks seemed like a mess. The new code bails on a few cases where we'd have to do two SIMD8 messages - we just fall back to SIMD8 for now. Improves performance in "Shadowrun: Dragonfall - Director's Cut" by about 20% on GM45 (measured with LIBGL_SHOW_FPS=1 while standing around in the first mission). v2: Add ir_txf to the has_lod case (caught by Jordan Justen). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2015-02-20 23:11:49 +00:00
assert(inst->mlen <= 9 && inst->mlen % 2 == 1);
i965/fs: Implement texelFetch() on Gen4. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-07 00:39:01 +01:00
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_LD;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXS:
i965: Implement textureSize (TXS) on Gen4. Also, remove the BRW_SAMPLER_MESSAGE_SIMD8_RESINFO #define because there totally isn't a SIMD8 variant. Unfortunately, resinfo returns FLOAT32 on Broadwater/Crestline, unlike G45 which returns a proper UINT32. This turns out to be simple, however: when we emit MOVs to select the desired half of the SIMD16 result, we can simply override the register type to be float so it's converted to an integer. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2011-08-17 18:45:47 +01:00
assert(inst->mlen == 3);
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_RESINFO;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
break;
i965: Create a shared enum for hardware and compiler-internal opcodes. This should make gdbing more pleasant, and it might be used in sharing part of the codegen between the VS and FS backends.
2011-05-03 18:55:50 +01:00
default:
i965: Use unreachable() instead of unconditional assert(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-29 22:54:01 +01:00
unreachable("not reached");
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
}
assert(msg_type != -1);
if (simd_mode == BRW_SAMPLER_SIMD_MODE_SIMD16) {
dst = vec16(dst);
}
i965: Change header_present to header_size in backend_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-24 17:17:32 +00:00
assert(devinfo->gen < 7 || inst->header_size == 0 ||
i965/fs_reg: Allocate double the number of vgrfs in SIMD16 mode This is actually the squash of a bunch of different changes. Individual commit titles follow: i965/fs: Always 2-align registers SIMD16 for gen <= 5 i965/fs: Use the register width when applying offsets This reworks both byte_offset() and offset() to be more intelligent. The byte_offset() function now supports offsets bigger than 32. The offset() function uses the byte_offset() function together with the register width and the type size to offset the register by the correct amount. i965/fs: Change regs_read to be in hardware registers i965/fs: Change regs_written to be actual hardware registers i965/fs: Properly handle register widths in LOAD_PAYLOAD The LOAD_PAYLOAD instruction is a bit special because it collects a bunch of registers (with possibly different widths) into a single payload block. Once the payload is constructed, it's treated as a single block of data and most of the information such as register widths doesn't matter anymore. In particular, the offset of any particular source register is the accumulation of the sizes of the previous source registers. i965/fs: Properly set writemasks in LOAD_PAYLOAD i965/fs: Handle register widths in demote_pull_constants i965/fs: Get rid of implicit register doubling in the allocator i965/fs: Reserve enough registers for PLN instructions i965/fs: Make sources and destinations interfere in 16-wide i965/fs: Properly handle register widths in CSE i965/fs: Properly handle register widths in register_coalesce i965/fs: Properly handle widths in copy propagation i965/fs: Properly handle register widths in VARYING_PULL_CONSTANT_LOAD i965/fs: Properly handle register widths and odd register sizes in spilling i965/fs: Don't waste a register on texture lookups for gen >= 7 Previously, we were waisting a register in SIMD16 mode because we could only allocate registers in pairs. Now that we can allocate and address odd-sized registers, let's get rid of this special-case. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-18 22:27:55 +01:00
src.file == BRW_GENERAL_REGISTER_FILE);
i965/fs: Convert gen7 to using GRFs for texture messages. Looking at Lightsmark's shaders, the way we used MRFs (or in gen7's case, GRFs) was bad in a couple of ways. One was that it prevented compute-to-MRF for the common case of a texcoord that gets used exactly once, but where the texcoord setup all gets emitted before the texture calls (such as when it's a bare fragment shader input, which gets interpolated before processing main()). Another was that it introduced a bunch of dependencies that constrained scheduling, and forced waits for texture operations to be done before they are required. For example, we can now move the compute-to-MRF interpolation for the second texture send down after the first send. The downside is that this generally prevents remove_duplicate_mrf_writes() from doing anything, whereas previously it avoided work for the case of sampling from the same texcoord twice. However, I suspect that most of the win that originally justified that code was in avoiding the WAR stall on the first send, which this patch also avoids, rather than the small cost of the extra instruction. We see instruction count regressions in shaders in unigine, yofrankie, savage2, hon, and gstreamer. Improves GLB2.7 performance by 0.633628% +/- 0.491809% (n=121/125, avg of ~66fps, outliers below 61 dropped). Improves openarena performance by 1.01092% +/- 0.66897% (n=425). No significant difference on Lightsmark (n=44). v2: Squash in the fix for register unspilling for send-from-GRF, fixing a segfault in lightsmark. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Matt Turner <mattst88@gmail.com>
2013-10-10 01:17:59 +01:00
i965/fs/Gen4-7: Pass sampler_index to generate_tex Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-08-03 10:23:31 +01:00
assert(sampler_index.type == BRW_REGISTER_TYPE_UD);
i965/fs: Move message header and texture offset setup to generate_tex(). Setting the texture offset bits in the message header involves very specific hardware register descriptions. As such, I feel it's better suited for the lower level "generate" layer that has direct access to the weird register layouts, rather than at the fs_inst abstraction layer. This also parallels the approach I took in the VS backend. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2012-08-05 04:33:13 +01:00
/* Load the message header if present. If there's a texture offset,
* we need to set it up explicitly and load the offset bitfield.
* Otherwise, we can use an implied move from g0 to the first message reg.
*/
intel/fs: Set up sampler message headers in the visitor on gen7+ This gives the scheduler visibility into the headers which should improve scheduling. More importantly, however, it lets the scheduler know that the header gets written. As-is, the scheduler thinks that a texture instruction only reads it's payload and is unaware that it may write to the first register so it may reorder it with respect to a read from that register. This is causing issues in a couple of Dota 2 vertex shaders. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=104923 Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2018-03-01 03:57:44 +00:00
if (inst->header_size != 0 && devinfo->gen < 7) {
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen < 6 && !inst->offset) {
i965/fs: Refactor sampler message header to duplicate less code. Previously, the code to copy g0 to the message header existed in two places - one for the texture offset case, and one for any other case. By treating texture_offset as a special case of header_present, we can remove this duplication and shorten the code. Future patches which add new header fields also won't have to add additional duplication. This also clarifies a confusing construct. The old code contained: } else if (inst->header_present) { if (brw->gen >= 7) { ...explicit copy from g0 to the message header... } else { /* Set up an implied move from g0 to the MRF. */ } } This looks like it might set up an implied move on Sandybridge, which doesn't support those. However, Sandybridge only uses a message header for texture offsets, so it would never hit this code path. The new code avoids this implicit knowledge by only setting up an implied move on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-01-18 20:48:18 +00:00
/* Set up an implied move from g0 to the MRF. */
src = retype(brw_vec8_grf(0, 0), BRW_REGISTER_TYPE_UW);
i965/fs: Convert gen7 to using GRFs for texture messages. Looking at Lightsmark's shaders, the way we used MRFs (or in gen7's case, GRFs) was bad in a couple of ways. One was that it prevented compute-to-MRF for the common case of a texcoord that gets used exactly once, but where the texcoord setup all gets emitted before the texture calls (such as when it's a bare fragment shader input, which gets interpolated before processing main()). Another was that it introduced a bunch of dependencies that constrained scheduling, and forced waits for texture operations to be done before they are required. For example, we can now move the compute-to-MRF interpolation for the second texture send down after the first send. The downside is that this generally prevents remove_duplicate_mrf_writes() from doing anything, whereas previously it avoided work for the case of sampling from the same texcoord twice. However, I suspect that most of the win that originally justified that code was in avoiding the WAR stall on the first send, which this patch also avoids, rather than the small cost of the extra instruction. We see instruction count regressions in shaders in unigine, yofrankie, savage2, hon, and gstreamer. Improves GLB2.7 performance by 0.633628% +/- 0.491809% (n=121/125, avg of ~66fps, outliers below 61 dropped). Improves openarena performance by 1.01092% +/- 0.66897% (n=425). No significant difference on Lightsmark (n=44). v2: Squash in the fix for register unspilling for send-from-GRF, fixing a segfault in lightsmark. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Acked-by: Matt Turner <mattst88@gmail.com>
2013-10-10 01:17:59 +01:00
} else {
intel/fs: Set up sampler message headers in the visitor on gen7+ This gives the scheduler visibility into the headers which should improve scheduling. More importantly, however, it lets the scheduler know that the header gets written. As-is, the scheduler thinks that a texture instruction only reads it's payload and is unaware that it may write to the first register so it may reorder it with respect to a read from that register. This is causing issues in a couple of Dota 2 vertex shaders. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=104923 Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2018-03-01 03:57:44 +00:00
assert(inst->base_mrf != -1);
struct brw_reg header_reg = brw_message_reg(inst->base_mrf);
i965/fs: Refactor sampler message header to duplicate less code. Previously, the code to copy g0 to the message header existed in two places - one for the texture offset case, and one for any other case. By treating texture_offset as a special case of header_present, we can remove this duplication and shorten the code. Future patches which add new header fields also won't have to add additional duplication. This also clarifies a confusing construct. The old code contained: } else if (inst->header_present) { if (brw->gen >= 7) { ...explicit copy from g0 to the message header... } else { /* Set up an implied move from g0 to the MRF. */ } } This looks like it might set up an implied move on Sandybridge, which doesn't support those. However, Sandybridge only uses a message header for texture offsets, so it would never hit this code path. The new code avoids this implicit knowledge by only setting up an implied move on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-01-18 20:48:18 +00:00
i965/fs: Fix handling of sampler messages with header but zero offset Gather unconditionally uses a header, but in some cases the texture_offset value will be zero. V2: Don't introduce a bogus conversion. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-10-13 00:20:03 +01:00
brw_push_insn_state(p);
i965: Replace guess_execution_size with something simpler. guess_execution_size() does two things: 1. Cope with small destination registers. 2. Cope with SIMD8 vs SIMD16 mode. This patch replaces the first with a simple if block in brw_set_dest: if the destination register width is less than 8, you probably want the execution size to match. (I didn't put this in the 3src block because it doesn't seem to matter.) Since only the FS compiler cares about SIMD16 mode, it's easy to just set the default execution size there. This pattern was already been proven in the Gen8+ generator, but we didn't port it back to the existing generator when we combined the two. This is based on a patch from Ken from about a year ago. I've rebased it and and fixed a few bugs. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 20:40:34 +01:00
brw_set_default_exec_size(p, BRW_EXECUTE_8);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-01 00:57:02 +01:00
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_compression_control(p, BRW_COMPRESSION_NONE);
i965/fs: Refactor sampler message header to duplicate less code. Previously, the code to copy g0 to the message header existed in two places - one for the texture offset case, and one for any other case. By treating texture_offset as a special case of header_present, we can remove this duplication and shorten the code. Future patches which add new header fields also won't have to add additional duplication. This also clarifies a confusing construct. The old code contained: } else if (inst->header_present) { if (brw->gen >= 7) { ...explicit copy from g0 to the message header... } else { /* Set up an implied move from g0 to the MRF. */ } } This looks like it might set up an implied move on Sandybridge, which doesn't support those. However, Sandybridge only uses a message header for texture offsets, so it would never hit this code path. The new code avoids this implicit knowledge by only setting up an implied move on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-01-18 20:48:18 +00:00
/* Explicitly set up the message header by copying g0 to the MRF. */
brw_MOV(p, header_reg, brw_vec8_grf(0, 0));
intel/fs: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 20:07:00 +01:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965: Combine offset/texture_offset fields. texture_offset was only used by some texturing operations, and offset was only used by spill/unspill and some URB operations. These fields are never used at the same time. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-12 19:28:02 +00:00
if (inst->offset) {
i965/fs: Refactor sampler message header to duplicate less code. Previously, the code to copy g0 to the message header existed in two places - one for the texture offset case, and one for any other case. By treating texture_offset as a special case of header_present, we can remove this duplication and shorten the code. Future patches which add new header fields also won't have to add additional duplication. This also clarifies a confusing construct. The old code contained: } else if (inst->header_present) { if (brw->gen >= 7) { ...explicit copy from g0 to the message header... } else { /* Set up an implied move from g0 to the MRF. */ } } This looks like it might set up an implied move on Sandybridge, which doesn't support those. However, Sandybridge only uses a message header for texture offsets, so it would never hit this code path. The new code avoids this implicit knowledge by only setting up an implied move on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-01-18 20:48:18 +00:00
/* Set the offset bits in DWord 2. */
brw_MOV(p, get_element_ud(header_reg, 2),
i965: Combine offset/texture_offset fields. texture_offset was only used by some texturing operations, and offset was only used by spill/unspill and some URB operations. These fields are never used at the same time. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2014-11-12 19:28:02 +00:00
brw_imm_ud(inst->offset));
i965/fs: Refactor sampler message header to duplicate less code. Previously, the code to copy g0 to the message header existed in two places - one for the texture offset case, and one for any other case. By treating texture_offset as a special case of header_present, we can remove this duplication and shorten the code. Future patches which add new header fields also won't have to add additional duplication. This also clarifies a confusing construct. The old code contained: } else if (inst->header_present) { if (brw->gen >= 7) { ...explicit copy from g0 to the message header... } else { /* Set up an implied move from g0 to the MRF. */ } } This looks like it might set up an implied move on Sandybridge, which doesn't support those. However, Sandybridge only uses a message header for texture offsets, so it would never hit this code path. The new code avoids this implicit knowledge by only setting up an implied move on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-01-18 20:48:18 +00:00
}
i965/fs: Support arbitrarily large sampler state indices on Haswell+. The message descriptor's "Sampler Index" field is only 4 bits (on all generations of hardware), so it can only represent indices 0 through 15. Haswell introduced a new field in the message header - "Sampler State Pointer". Normally, this is copied straight from g0, but we can also add a byte offset (as long as it's a multiple of 32). This patch uses a "Sampler State Pointer" offset to select a group of 16 sampler states, and then uses the "Sampler Index" field to select the state within that group. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-01-18 21:29:39 +00:00
i965/fs: Fix handling of sampler messages with header but zero offset Gather unconditionally uses a header, but in some cases the texture_offset value will be zero. V2: Don't introduce a bogus conversion. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-10-13 00:20:03 +01:00
brw_pop_insn_state(p);
}
i965/fs: Move message header and texture offset setup to generate_tex(). Setting the texture offset bits in the message header involves very specific hardware register descriptions. As such, I feel it's better suited for the lower level "generate" layer that has direct access to the weird register layouts, rather than at the fs_inst abstraction layer. This also parallels the approach I took in the VS backend. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2012-08-05 04:33:13 +01:00
}
intel: Use TXS for image_size when we have a typed surface Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-08-16 17:01:24 +01:00
uint32_t base_binding_table_index;
switch (inst->opcode) {
case SHADER_OPCODE_TG4:
case SHADER_OPCODE_TG4_OFFSET:
base_binding_table_index = prog_data->binding_table.gather_texture_start;
break;
case SHADER_OPCODE_IMAGE_SIZE:
base_binding_table_index = prog_data->binding_table.image_start;
break;
default:
base_binding_table_index = prog_data->binding_table.texture_start;
break;
}
i965/fs: Refactor generate_tex in prep for nonconst sampler indexing Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 00:58:06 +01:00
i965/fs: Separate the sampler from the surface in generate_tex Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 23:24:05 +00:00
if (surface_index.file == BRW_IMMEDIATE_VALUE &&
sampler_index.file == BRW_IMMEDIATE_VALUE) {
uint32_t surface = surface_index.ud;
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 03:41:30 +01:00
uint32_t sampler = sampler_index.ud;
i965/fs: Refactor generate_tex in prep for nonconst sampler indexing Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 00:58:06 +01:00
brw_SAMPLE(p,
retype(dst, BRW_REGISTER_TYPE_UW),
inst->base_mrf,
src,
i965/fs: Separate the sampler from the surface in generate_tex Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 23:24:05 +00:00
surface + base_binding_table_index,
i965/fs: Refactor generate_tex in prep for nonconst sampler indexing Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 00:58:06 +01:00
sampler % 16,
msg_type,
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
inst->size_written / REG_SIZE,
i965/fs: Refactor generate_tex in prep for nonconst sampler indexing Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 00:58:06 +01:00
inst->mlen,
i965: Change header_present to header_size in backend_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-24 17:17:32 +00:00
inst->header_size != 0,
i965/fs: Refactor generate_tex in prep for nonconst sampler indexing Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 00:58:06 +01:00
simd_mode,
return_format);
} else {
i965/fs: Add support for non-const sampler indices in generator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 01:02:22 +01:00
/* Non-const sampler index */
struct brw_reg addr = vec1(retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD));
i965/fs: Separate the sampler from the surface in generate_tex Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 23:24:05 +00:00
struct brw_reg surface_reg = vec1(retype(surface_index, BRW_REGISTER_TYPE_UD));
i965/fs: Add support for non-const sampler indices in generator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 01:02:22 +01:00
struct brw_reg sampler_reg = vec1(retype(sampler_index, BRW_REGISTER_TYPE_UD));
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_set_default_access_mode(p, BRW_ALIGN_1);
intel/fs: Explicitly set EXECUTE_1 where needed Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-30 20:07:00 +01:00
brw_set_default_exec_size(p, BRW_EXECUTE_1);
i965/fs: Add support for non-const sampler indices in generator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 01:02:22 +01:00
i965: Fix undefined df bits in brw_reg comparisons. Commit 5310bca024f77da40ea6f4c275455f9cb0528f9e added a new "double df" field to the brw_reg struct, adding an extra 4 bytes of data that isn't usually initialized (or may contain irrelevant garbage if the struct is mutated). This means that it's no longer safe to memcmp(). Instead, add a brw_regs_equal() function which ignores the extra df bits unless they matter. To keep the implementation cheap, we wrap the first set of fields in a union/struct so that we can use a single DWord comparison. v2: Drop unnecessary casts (caught by Francisco Jerez). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
2016-05-14 00:41:13 +01:00
if (brw_regs_equal(&surface_reg, &sampler_reg)) {
i965/fs: Separate the sampler from the surface in generate_tex Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 23:24:05 +00:00
brw_MUL(p, addr, sampler_reg, brw_imm_uw(0x101));
} else {
i965/generator/tex: Handle an immediate sampler with an indirect texture In this case we were dying when we tried to do SHL addr sampler imm(8) because that puts an immediate in src0 of a two source instruction. This fixes 2704 of the new separate sampler Vulkan CTS tests on Sky Lake. Reviewed-by: Eduardo Lima Mitev <elima@igalia.com> Cc: "13.0" <mesa-stable@lists.freedesktop.org>
2016-12-22 23:26:12 +00:00
if (sampler_reg.file == BRW_IMMEDIATE_VALUE) {
brw_OR(p, addr, surface_reg, brw_imm_ud(sampler_reg.ud << 8));
} else {
brw_SHL(p, addr, sampler_reg, brw_imm_ud(8));
brw_OR(p, addr, addr, surface_reg);
}
i965/fs: Separate the sampler from the surface in generate_tex Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-11-02 23:24:05 +00:00
}
i965: Don't add base_binding_table_index if it's zero When calculating the binding table index for non-constant sampler array indexing it needs to add the base binding table index which is a constant within the generated code. Often this base is zero so we can avoid a redundant instruction in that case. It looks like nothing in shader-db is doing non-constant sampler array indexing so this patch doesn't make any difference but it might be worth having anyway. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Acked-by: Ben Widawsky <ben@bwidawsk.net>
2015-05-29 13:41:48 +01:00
if (base_binding_table_index)
brw_ADD(p, addr, addr, brw_imm_ud(base_binding_table_index));
i965: Don't use a temporary when generating an indirect sample Previously when generating the send instruction for a sample instruction with an indirect sampler it would use the destination register as a temporary store. This breaks when used in combination with the opt_sampler_eot optimisation because that forces the destination to be null. This patch fixes that by avoiding the temp register altogether. The reason the temporary register was needed was because it was trying to ensure the binding table index doesn't overflow a byte by and'ing it with 0xff. The result is then or'd with samper_index<<8. This patch instead just and's the whole thing by 0xfff. This will ensure that a bogus sampler index won't overflow into the rest of the message descriptor but unlike the previous code it won't ensure that the binding table index doesn't overflow into the sampler index. It doesn't seem like that should matter very much though because if the shader is generating a bogus sampler index then it's going to just get garbage out either way. Instead of doing sampler_index<<8|(sampler_index+base_table_index) the new code avoids one operation by doing sampler_index*0x101+base_table_index which should be equivalent. However if we wanted to avoid the multiply for some reason we could do this by adding an extra or instruction still without needing the temporary register. This fixes a number of Piglit tests on Skylake that were using indirect samplers such as: spec@arb_gpu_shader5@execution@sampler_array_indexing@fs-simple Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Acked-by: Ben Widawsky <ben@bwidawsk.net> Tested-by: Anuj Phogat <anuj.phogat@gmail.com>
2015-05-28 15:27:31 +01:00
brw_AND(p, addr, addr, brw_imm_ud(0xfff));
i965/fs: Add support for non-const sampler indices in generator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 01:02:22 +01:00
i965: Don't disable exec masking for sampler message sends. This was telling the sampler to do texture fetches for *all* channels in the non-constant surface index case, what could have reduced throughput unnecessarily when some of the channels were disabled by control flow. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 15:24:03 +00:00
brw_pop_insn_state(p);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 13:44:24 +00:00
/* dst = send(offset, a0.0 | <descriptor>) */
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_send_indirect_message(
p, BRW_SFID_SAMPLER, dst, src, addr,
brw_message_desc(devinfo, inst->mlen, inst->size_written / REG_SIZE,
inst->header_size) |
brw_sampler_desc(devinfo,
0 /* surface */,
0 /* sampler */,
msg_type,
simd_mode,
return_format));
i965/fs: Add support for non-const sampler indices in generator Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 01:02:22 +01:00
/* visitor knows more than we do about the surface limit required,
* so has already done marking.
*/
i965/fs: Refactor generate_tex in prep for nonconst sampler indexing Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-10 00:58:06 +01:00
}
i965/fs: Combine tex/fb_write operations (opt) Certain platforms support the ability to sample from a texture, and write it out to the file RT - thus saving a costly send instructions (note that this is a potnential win if one wanted to backport to a tag that didn't have the patch from Topi which removed excess MOVs from LOAD_PAYLOAD - 97caf5fa04dbd2), v2: Modify the algorithm. Instead of iterating in reverse through blocks and insts, since the last block/inst is the only thing which can benefit. Rebased on top of Ken's patching modifying is_last_send v3: Rebased over almost 2 months, and Incorporated feedback from Matt: Some comment typo fixes and rewordings. Whitespace Move the optimization pass outside of the optimize loop v4: Some cosmetic changes requested from Ken. These changes ensured that the optimization function always returned true when an optimization occurred, and false when one did not. This behavior did not exist with the original patch. As a result, having the separate helper function which Matt did not like no longer made sense, and so now I believe everyone should be happy. Benchmark (n=20) %diff *OglBatch5 -1.4 *OglBatch7 -1.79 OglFillTexMulti 5.57 OglFillTexSingle 1.16 OglShMapPcf 0.05 OglTexFilterAniso 3.01 OglTexFilterTri 1.94 No piglit regressions: (http://otc-gfxtest-01.jf.intel.com:8080/view/dev/job/bwidawsk/112/) [*] I believe my measurements are incorrect for Batch5-7. If I add this new optimization, but never emit the new instruction I see similar results. v5: Remove declaration of combine_tex_header since v4 dropped that function (Ben) Remove check for impossible case of an empty block (Matt) Set dest earlier to avoid extra special-casing in generate_tex (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-08 21:59:57 +00:00
if (is_combined_send) {
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_eot(p->devinfo, brw_last_inst, true);
brw_inst_set_opcode(p->devinfo, brw_last_inst, BRW_OPCODE_SENDC);
i965/fs: Combine tex/fb_write operations (opt) Certain platforms support the ability to sample from a texture, and write it out to the file RT - thus saving a costly send instructions (note that this is a potnential win if one wanted to backport to a tag that didn't have the patch from Topi which removed excess MOVs from LOAD_PAYLOAD - 97caf5fa04dbd2), v2: Modify the algorithm. Instead of iterating in reverse through blocks and insts, since the last block/inst is the only thing which can benefit. Rebased on top of Ken's patching modifying is_last_send v3: Rebased over almost 2 months, and Incorporated feedback from Matt: Some comment typo fixes and rewordings. Whitespace Move the optimization pass outside of the optimize loop v4: Some cosmetic changes requested from Ken. These changes ensured that the optimization function always returned true when an optimization occurred, and false when one did not. This behavior did not exist with the original patch. As a result, having the separate helper function which Matt did not like no longer made sense, and so now I believe everyone should be happy. Benchmark (n=20) %diff *OglBatch5 -1.4 *OglBatch7 -1.79 OglFillTexMulti 5.57 OglFillTexSingle 1.16 OglShMapPcf 0.05 OglTexFilterAniso 3.01 OglTexFilterTri 1.94 No piglit regressions: (http://otc-gfxtest-01.jf.intel.com:8080/view/dev/job/bwidawsk/112/) [*] I believe my measurements are incorrect for Batch5-7. If I add this new optimization, but never emit the new instruction I see similar results. v5: Remove declaration of combine_tex_header since v4 dropped that function (Ben) Remove check for impossible case of an empty block (Matt) Set dest earlier to avoid extra special-casing in generate_tex (Matt) Signed-off-by: Ben Widawsky <ben@bwidawsk.net> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-08 21:59:57 +00:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
/* For OPCODE_DDX and OPCODE_DDY, per channel of output we've got input
* looking like:
*
* arg0: ss0.tl ss0.tr ss0.bl ss0.br ss1.tl ss1.tr ss1.bl ss1.br
*
i965: compute DDX in a subspan based only on top row Consider only the top-left and top-right pixels to approximate DDX in a 2x2 subspan, unless the application requests a more accurate approximation via GL_FRAGMENT_SHADER_DERIVATIVE_HINT or this optimization is disabled from the new driconf option disable_derivative_optimization. This results in a less accurate approximation. However, it improves the performance of Xonotic with Ultra settings by 24.3879% +/- 0.832202% (at 95.0% confidence) on Haswell. No noticeable image quality difference observed. The improvement comes from faster sample_d. It seems, on Haswell, some optimizations are introduced to allow faster sample_d when all pixels in a subspan have the same derivative. I considered SAMPLE_STATE too, which allows one to control the quality of sample_d on Haswell. But it gave much worse image quality without giving better performance comparing to this change. No piglit quick.tests regression on Haswell (tested with v1). v2: better guess for precompile program key Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2013-09-12 06:00:52 +01:00
* Ideally, we want to produce:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
*
* DDX DDY
* dst: (ss0.tr - ss0.tl) (ss0.tl - ss0.bl)
* (ss0.tr - ss0.tl) (ss0.tr - ss0.br)
* (ss0.br - ss0.bl) (ss0.tl - ss0.bl)
* (ss0.br - ss0.bl) (ss0.tr - ss0.br)
* (ss1.tr - ss1.tl) (ss1.tl - ss1.bl)
* (ss1.tr - ss1.tl) (ss1.tr - ss1.br)
* (ss1.br - ss1.bl) (ss1.tl - ss1.bl)
* (ss1.br - ss1.bl) (ss1.tr - ss1.br)
*
* and add another set of two more subspans if in 16-pixel dispatch mode.
*
* For DDX, it ends up being easy: width = 2, horiz=0 gets us the same result
* for each pair, and vertstride = 2 jumps us 2 elements after processing a
i965: compute DDX in a subspan based only on top row Consider only the top-left and top-right pixels to approximate DDX in a 2x2 subspan, unless the application requests a more accurate approximation via GL_FRAGMENT_SHADER_DERIVATIVE_HINT or this optimization is disabled from the new driconf option disable_derivative_optimization. This results in a less accurate approximation. However, it improves the performance of Xonotic with Ultra settings by 24.3879% +/- 0.832202% (at 95.0% confidence) on Haswell. No noticeable image quality difference observed. The improvement comes from faster sample_d. It seems, on Haswell, some optimizations are introduced to allow faster sample_d when all pixels in a subspan have the same derivative. I considered SAMPLE_STATE too, which allows one to control the quality of sample_d on Haswell. But it gave much worse image quality without giving better performance comparing to this change. No piglit quick.tests regression on Haswell (tested with v1). v2: better guess for precompile program key Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2013-09-12 06:00:52 +01:00
* pair. But the ideal approximation may impose a huge performance cost on
* sample_d. On at least Haswell, sample_d instruction does some
* optimizations if the same LOD is used for all pixels in the subspan.
*
i965/fs: Improve accuracy of dFdy() to match dFdx(). Previously, we computed dFdy() using the following instruction: add(8) dst<1>F src<4,4,0)F -src.2<4,4,0>F { align1 1Q } That had the disadvantage that it computed the same value for all 4 pixels of a 2x2 subspan, which meant that it was less accurate than dFdx(). This patch changes it to the following instruction when c->key.high_quality_derivatives is set: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } This gives it comparable accuracy to dFdx(). Unfortunately, align16 instructions can't be compressed, so in SIMD16 shaders, instead of emitting this instruction: add(16) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1H } We need to unroll to two instructions: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } add(8) (dst+1)<1>F (src+1)<4,4,1>.xyxyF -(src+1)<4,4,1>.zwzwF { align16 2Q } Fixes piglit test spec/glsl-1.10/execution/fs-dfdy-accuracy. Acked-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-20 17:04:31 +01:00
* For DDY, we need to use ALIGN16 mode since it's capable of doing the
* appropriate swizzling.
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
*/
void
intel/compiler/fs: Pass fs_inst to generate_ddx/ddy instead of opcode In a future patch, generate_ddy will want to inspect inst->exec_size. Change generate_ddx as well for consistency. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 23:41:40 +01:00
fs_generator::generate_ddx(const fs_inst *inst,
i965/fs: Handle derivative quality decisions in the front-end. Kristian noted that there's very little use of brw_wm_prog_key in the generator, and that it basically just generates what it's told, without caring about what stage it's handling. One exception to this is derivative handling. When handling dFdxCoarse and dFdxFine, we packed an enum value in a second source register, explicitly telling the generator what to do. For dFdx, we specified an enum value of "please use the hint", then checked the program key in the generator level code. A natural method is to define separate FS_OPCODE_DD[XY]_{COARSE,FINE} opcodes, and have the front-end (which already decides what IR to generate based on the program key) decide which dPdx/dPdy should correspond to. This consolidates the decision making in one place. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-08 09:39:14 +00:00
struct brw_reg dst, struct brw_reg src)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965: compute DDX in a subspan based only on top row Consider only the top-left and top-right pixels to approximate DDX in a 2x2 subspan, unless the application requests a more accurate approximation via GL_FRAGMENT_SHADER_DERIVATIVE_HINT or this optimization is disabled from the new driconf option disable_derivative_optimization. This results in a less accurate approximation. However, it improves the performance of Xonotic with Ultra settings by 24.3879% +/- 0.832202% (at 95.0% confidence) on Haswell. No noticeable image quality difference observed. The improvement comes from faster sample_d. It seems, on Haswell, some optimizations are introduced to allow faster sample_d when all pixels in a subspan have the same derivative. I considered SAMPLE_STATE too, which allows one to control the quality of sample_d on Haswell. But it gave much worse image quality without giving better performance comparing to this change. No piglit quick.tests regression on Haswell (tested with v1). v2: better guess for precompile program key Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2013-09-12 06:00:52 +01:00
unsigned vstride, width;
intel/compiler/fs: Pass fs_inst to generate_ddx/ddy instead of opcode In a future patch, generate_ddy will want to inspect inst->exec_size. Change generate_ddx as well for consistency. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 23:41:40 +01:00
if (inst->opcode == FS_OPCODE_DDX_FINE) {
i965: compute DDX in a subspan based only on top row Consider only the top-left and top-right pixels to approximate DDX in a 2x2 subspan, unless the application requests a more accurate approximation via GL_FRAGMENT_SHADER_DERIVATIVE_HINT or this optimization is disabled from the new driconf option disable_derivative_optimization. This results in a less accurate approximation. However, it improves the performance of Xonotic with Ultra settings by 24.3879% +/- 0.832202% (at 95.0% confidence) on Haswell. No noticeable image quality difference observed. The improvement comes from faster sample_d. It seems, on Haswell, some optimizations are introduced to allow faster sample_d when all pixels in a subspan have the same derivative. I considered SAMPLE_STATE too, which allows one to control the quality of sample_d on Haswell. But it gave much worse image quality without giving better performance comparing to this change. No piglit quick.tests regression on Haswell (tested with v1). v2: better guess for precompile program key Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2013-09-12 06:00:52 +01:00
/* produce accurate derivatives */
vstride = BRW_VERTICAL_STRIDE_2;
width = BRW_WIDTH_2;
i965/fs: Handle derivative quality decisions in the front-end. Kristian noted that there's very little use of brw_wm_prog_key in the generator, and that it basically just generates what it's told, without caring about what stage it's handling. One exception to this is derivative handling. When handling dFdxCoarse and dFdxFine, we packed an enum value in a second source register, explicitly telling the generator what to do. For dFdx, we specified an enum value of "please use the hint", then checked the program key in the generator level code. A natural method is to define separate FS_OPCODE_DD[XY]_{COARSE,FINE} opcodes, and have the front-end (which already decides what IR to generate based on the program key) decide which dPdx/dPdy should correspond to. This consolidates the decision making in one place. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-08 09:39:14 +00:00
} else {
i965: compute DDX in a subspan based only on top row Consider only the top-left and top-right pixels to approximate DDX in a 2x2 subspan, unless the application requests a more accurate approximation via GL_FRAGMENT_SHADER_DERIVATIVE_HINT or this optimization is disabled from the new driconf option disable_derivative_optimization. This results in a less accurate approximation. However, it improves the performance of Xonotic with Ultra settings by 24.3879% +/- 0.832202% (at 95.0% confidence) on Haswell. No noticeable image quality difference observed. The improvement comes from faster sample_d. It seems, on Haswell, some optimizations are introduced to allow faster sample_d when all pixels in a subspan have the same derivative. I considered SAMPLE_STATE too, which allows one to control the quality of sample_d on Haswell. But it gave much worse image quality without giving better performance comparing to this change. No piglit quick.tests regression on Haswell (tested with v1). v2: better guess for precompile program key Signed-off-by: Chia-I Wu <olv@lunarg.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2013-09-12 06:00:52 +01:00
/* replicate the derivative at the top-left pixel to other pixels */
vstride = BRW_VERTICAL_STRIDE_4;
width = BRW_WIDTH_4;
}
intel/compiler/fs: Simplify ddx/ddy code generation The brw_reg() constructor just obfuscates things here, in my opinion. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-16 01:20:29 +01:00
struct brw_reg src0 = src;
struct brw_reg src1 = src;
src0.subnr = sizeof(float);
src0.vstride = vstride;
src0.width = width;
src0.hstride = BRW_HORIZONTAL_STRIDE_0;
src1.vstride = vstride;
src1.width = width;
src1.hstride = BRW_HORIZONTAL_STRIDE_0;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
brw_ADD(p, dst, src0, negate(src1));
}
i965: Compute dFdy() correctly for FBOs. On i965, dFdx() and dFdy() are computed by taking advantage of the fact that each consecutive set of 4 pixels dispatched to the fragment shader always constitutes a contiguous 2x2 block of pixels in a fixed arrangement known as a "sub-span". So we calculate dFdx() by taking the difference between the values computed for the left and right halves of the sub-span, and we calculate dFdy() by taking the difference between the values computed for the top and bottom halves of the sub-span. However, there's a subtlety when FBOs are in use: since FBOs use a coordinate system where the origin is at the upper left, and window system framebuffers use a coordinate system where the origin is at the lower left, the computation of dFdy() needs to be negated for FBOs. This patch modifies the fragment shader back-ends to negate the value of dFdy() when an FBO is in use. It also modifies the code that populates the program key (brw_wm_populate_key() and brw_fs_precompile()) so that they always record in the program key whether we are rendering to an FBO or to a window system framebuffer; this ensures that the fragment shader will get recompiled when switching between FBO and non-FBO use. This will result in unnecessary recompiles of fragment shaders that don't use dFdy(). To fix that, we will need to adapt the GLSL and NV_fragment_program front-ends to record whether or not a given shader uses dFdy(). I plan to implement this in a future patch series; I've left FIXME comments in the code as a reminder. Fixes Piglit test "fbo-deriv". NOTE: This is a candidate for stable release branches. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-06-20 21:40:45 +01:00
/* The negate_value boolean is used to negate the derivative computation for
* FBOs, since they place the origin at the upper left instead of the lower
* left.
*/
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
void
intel/compiler/fs: Pass fs_inst to generate_ddx/ddy instead of opcode In a future patch, generate_ddy will want to inspect inst->exec_size. Change generate_ddx as well for consistency. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 23:41:40 +01:00
fs_generator::generate_ddy(const fs_inst *inst,
i965, anv: Use NIR FragCoord re-center and y-transform passes. This handles gl_FragCoord transformations and other window system vs. user FBO coordinate system flipping by multiplying/adding uniform values, rather than recompiles. This is much better because we have no decent way to guess whether the application is going to use a shader with the window system FBO or a user FBO, much less the drawable height. This led to a lot of recompiles in many applications. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-17 09:52:16 +01:00
struct brw_reg dst, struct brw_reg src)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
intel/compiler/fs: Pass fs_inst to generate_ddx/ddy instead of opcode In a future patch, generate_ddy will want to inspect inst->exec_size. Change generate_ddx as well for consistency. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 23:41:40 +01:00
if (inst->opcode == FS_OPCODE_DDY_FINE) {
i965/fs: Improve accuracy of dFdy() to match dFdx(). Previously, we computed dFdy() using the following instruction: add(8) dst<1>F src<4,4,0)F -src.2<4,4,0>F { align1 1Q } That had the disadvantage that it computed the same value for all 4 pixels of a 2x2 subspan, which meant that it was less accurate than dFdx(). This patch changes it to the following instruction when c->key.high_quality_derivatives is set: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } This gives it comparable accuracy to dFdx(). Unfortunately, align16 instructions can't be compressed, so in SIMD16 shaders, instead of emitting this instruction: add(16) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1H } We need to unroll to two instructions: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } add(8) (dst+1)<1>F (src+1)<4,4,1>.xyxyF -(src+1)<4,4,1>.zwzwF { align16 2Q } Fixes piglit test spec/glsl-1.10/execution/fs-dfdy-accuracy. Acked-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-20 17:04:31 +01:00
/* produce accurate derivatives */
intel/compiler/fs: Implement ddy without using align16 for Gen11+ Align16 is no more. We previously generated an align16 ADD instruction to calculate DDY: add(16) g25<1>F -g23<4>.xyxyF g23<4>.zwzwF { align16 1H }; Without align16, we now implement it as: add(4) g25<1>F -g23<0,2,1>F g23.2<0,2,1>F { align1 1N }; add(4) g25.4<1>F -g23.4<0,2,1>F g23.6<0,2,1>F { align1 1N }; add(4) g26<1>F -g24<0,2,1>F g24.2<0,2,1>F { align1 1N }; add(4) g26.4<1>F -g24.4<0,2,1>F g24.6<0,2,1>F { align1 1N }; where only the first two instructions are needed in SIMD8 mode. Note: an earlier version of the patch implemented this in two instructions in SIMD16: add(8) g25<2>F -g23<4,2,0>F g23.2<4,2,0>F { align1 1N }; add(8) g25.1<2>F -g23.1<4,2,0>F g23.3<4,2,0>F { align1 1N }; but I realized that the channel enable bits will not be correct. If we knew we were under uniform control flow, we could emit only those two instructions however. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-16 01:29:16 +01:00
if (devinfo->gen >= 11) {
src = stride(src, 0, 2, 1);
struct brw_reg src_0 = byte_offset(src, 0 * sizeof(float));
struct brw_reg src_2 = byte_offset(src, 2 * sizeof(float));
struct brw_reg src_4 = byte_offset(src, 4 * sizeof(float));
struct brw_reg src_6 = byte_offset(src, 6 * sizeof(float));
struct brw_reg src_8 = byte_offset(src, 8 * sizeof(float));
struct brw_reg src_10 = byte_offset(src, 10 * sizeof(float));
struct brw_reg src_12 = byte_offset(src, 12 * sizeof(float));
struct brw_reg src_14 = byte_offset(src, 14 * sizeof(float));
struct brw_reg dst_0 = byte_offset(dst, 0 * sizeof(float));
struct brw_reg dst_4 = byte_offset(dst, 4 * sizeof(float));
struct brw_reg dst_8 = byte_offset(dst, 8 * sizeof(float));
struct brw_reg dst_12 = byte_offset(dst, 12 * sizeof(float));
intel/compiler/fs: Simplify ddx/ddy code generation The brw_reg() constructor just obfuscates things here, in my opinion. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-16 01:20:29 +01:00
intel/compiler/fs: Implement ddy without using align16 for Gen11+ Align16 is no more. We previously generated an align16 ADD instruction to calculate DDY: add(16) g25<1>F -g23<4>.xyxyF g23<4>.zwzwF { align16 1H }; Without align16, we now implement it as: add(4) g25<1>F -g23<0,2,1>F g23.2<0,2,1>F { align1 1N }; add(4) g25.4<1>F -g23.4<0,2,1>F g23.6<0,2,1>F { align1 1N }; add(4) g26<1>F -g24<0,2,1>F g24.2<0,2,1>F { align1 1N }; add(4) g26.4<1>F -g24.4<0,2,1>F g24.6<0,2,1>F { align1 1N }; where only the first two instructions are needed in SIMD8 mode. Note: an earlier version of the patch implemented this in two instructions in SIMD16: add(8) g25<2>F -g23<4,2,0>F g23.2<4,2,0>F { align1 1N }; add(8) g25.1<2>F -g23.1<4,2,0>F g23.3<4,2,0>F { align1 1N }; but I realized that the channel enable bits will not be correct. If we knew we were under uniform control flow, we could emit only those two instructions however. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-16 01:29:16 +01:00
brw_push_insn_state(p);
brw_set_default_exec_size(p, BRW_EXECUTE_4);
brw_ADD(p, dst_0, negate(src_0), src_2);
brw_ADD(p, dst_4, negate(src_4), src_6);
if (inst->exec_size == 16) {
brw_ADD(p, dst_8, negate(src_8), src_10);
brw_ADD(p, dst_12, negate(src_12), src_14);
}
brw_pop_insn_state(p);
} else {
struct brw_reg src0 = stride(src, 4, 4, 1);
struct brw_reg src1 = stride(src, 4, 4, 1);
src0.swizzle = BRW_SWIZZLE_XYXY;
src1.swizzle = BRW_SWIZZLE_ZWZW;
brw_push_insn_state(p);
brw_set_default_access_mode(p, BRW_ALIGN_16);
brw_ADD(p, dst, negate(src0), src1);
brw_pop_insn_state(p);
}
i965/fs: Improve accuracy of dFdy() to match dFdx(). Previously, we computed dFdy() using the following instruction: add(8) dst<1>F src<4,4,0)F -src.2<4,4,0>F { align1 1Q } That had the disadvantage that it computed the same value for all 4 pixels of a 2x2 subspan, which meant that it was less accurate than dFdx(). This patch changes it to the following instruction when c->key.high_quality_derivatives is set: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } This gives it comparable accuracy to dFdx(). Unfortunately, align16 instructions can't be compressed, so in SIMD16 shaders, instead of emitting this instruction: add(16) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1H } We need to unroll to two instructions: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } add(8) (dst+1)<1>F (src+1)<4,4,1>.xyxyF -(src+1)<4,4,1>.zwzwF { align16 2Q } Fixes piglit test spec/glsl-1.10/execution/fs-dfdy-accuracy. Acked-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-20 17:04:31 +01:00
} else {
/* replicate the derivative at the top-left pixel to other pixels */
intel/compiler/fs: Simplify ddx/ddy code generation The brw_reg() constructor just obfuscates things here, in my opinion. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-16 01:20:29 +01:00
struct brw_reg src0 = stride(src, 4, 4, 0);
struct brw_reg src1 = stride(src, 4, 4, 0);
src0.subnr = 0 * sizeof(float);
src1.subnr = 2 * sizeof(float);
i965: Delete dead dFdy flipping code. Rob's nir_lower_wpos_ytransform() pass flips dFdy in the opposite case of what I expected, so we always take the negate_value case. It doesn't really matter. v2: Write src0 before src1 in ADD instructions (requested by Matt). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-18 18:35:54 +01:00
brw_ADD(p, dst, negate(src0), src1);
i965/fs: Improve accuracy of dFdy() to match dFdx(). Previously, we computed dFdy() using the following instruction: add(8) dst<1>F src<4,4,0)F -src.2<4,4,0>F { align1 1Q } That had the disadvantage that it computed the same value for all 4 pixels of a 2x2 subspan, which meant that it was less accurate than dFdx(). This patch changes it to the following instruction when c->key.high_quality_derivatives is set: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } This gives it comparable accuracy to dFdx(). Unfortunately, align16 instructions can't be compressed, so in SIMD16 shaders, instead of emitting this instruction: add(16) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1H } We need to unroll to two instructions: add(8) dst<1>F src<4,4,1>.xyxyF -src<4,4,1>.zwzwF { align16 1Q } add(8) (dst+1)<1>F (src+1)<4,4,1>.xyxyF -(src+1)<4,4,1>.zwzwF { align16 2Q } Fixes piglit test spec/glsl-1.10/execution/fs-dfdy-accuracy. Acked-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-09-20 17:04:31 +01:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
void
intel/compiler: Silence unused parameter warnings in generate_foo methods Since all of the fs_generator::generate_foo methods take a fs_inst * as the first parameter, just remove the name to quiet the compiler. src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_barrier(fs_inst*, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:743:41: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_barrier(fs_inst *inst, struct brw_reg src) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_discard_jump(fs_inst*)’: src/intel/compiler/brw_fs_generator.cpp:1326:46: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_discard_jump(fs_inst *inst) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_pack_half_2x16_split(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1675:54: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_pack_half_2x16_split(fs_inst *inst, ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_shader_time_add(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1743:49: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_shader_time_add(fs_inst *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_set_simd4x2_header_gen9(brw_codegen*, brw::vec4_instruction*, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1412:52: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_mov_indirect(brw_codegen*, brw::vec4_instruction*, brw_reg, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1430:41: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp:1432:63: warning: unused parameter ‘length’ [-Wunused-parameter] struct brw_reg indirect, struct brw_reg length) ^~~~~~ Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2018-03-29 00:29:45 +01:00
fs_generator::generate_discard_jump(fs_inst *)
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 6);
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
/* This HALT will be patched up at FB write time to point UIP at the end of
* the program, and at brw_uip_jip() JIP will be set to the end of the
* current block (or the program).
*/
this->discard_halt_patches.push_tail(new(mem_ctx) ip_record(p->nr_insn));
gen6_HALT(p);
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
void
i965: Merge together opcodes for SHADER_OPCODE_GEN4_SCRATCH_READ/WRITE I'm going to be introducing gen7 variants, and the previous naming was going to get confusing. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:45:06 +01:00
fs_generator::generate_scratch_write(fs_inst *inst, struct brw_reg src)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965/fs: Lower 32-wide scratch writes in the generator. The hardware has messages that can write 32 32bit components at once but the channel enable mask gets messed up. We need to split them into several 16-wide scratch writes for the channel enables to be applied correctly. The SIMD lowering pass cannot be used for this because scratch writes are emitted rather late during register allocation long after SIMD lowering has been done. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-28 07:29:02 +01:00
/* The 32-wide messages only respect the first 16-wide half of the channel
* enable signals which are replicated identically for the second group of
* 16 channels, so we cannot use them unless the write is marked
* force_writemask_all.
*/
const unsigned lower_size = inst->force_writemask_all ? inst->exec_size :
MIN2(16, inst->exec_size);
const unsigned block_size = 4 * lower_size / REG_SIZE;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
assert(inst->mlen != 0);
i965/fs: Lower 32-wide scratch writes in the generator. The hardware has messages that can write 32 32bit components at once but the channel enable mask gets messed up. We need to split them into several 16-wide scratch writes for the channel enables to be applied correctly. The SIMD lowering pass cannot be used for this because scratch writes are emitted rather late during register allocation long after SIMD lowering has been done. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-28 07:29:02 +01:00
brw_push_insn_state(p);
brw_set_default_exec_size(p, cvt(lower_size) - 1);
brw_set_default_compression(p, lower_size > 8);
for (unsigned i = 0; i < inst->exec_size / lower_size; i++) {
i965/fs: Expose arbitrary channel execution groups to the IR. This generalizes the current fs_inst::force_sechalf flag to allow specifying channel enable groups other than 0 or 8. At some point it will likely make sense to fix the vec4 generator to support arbitrary execution groups and then move the definition of fs_inst::group into backend_instruction (e.g. so we can do FP64 in the VEC4 back-end). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-21 00:14:13 +01:00
brw_set_default_group(p, inst->group + lower_size * i);
i965/fs: Lower 32-wide scratch writes in the generator. The hardware has messages that can write 32 32bit components at once but the channel enable mask gets messed up. We need to split them into several 16-wide scratch writes for the channel enables to be applied correctly. The SIMD lowering pass cannot be used for this because scratch writes are emitted rather late during register allocation long after SIMD lowering has been done. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-28 07:29:02 +01:00
brw_MOV(p, brw_uvec_mrf(lower_size, inst->base_mrf + 1, 0),
retype(offset(src, block_size * i), BRW_REGISTER_TYPE_UD));
brw_oword_block_write_scratch(p, brw_message_reg(inst->base_mrf),
block_size,
inst->offset + block_size * REG_SIZE * i);
}
brw_pop_insn_state(p);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
void
i965: Merge together opcodes for SHADER_OPCODE_GEN4_SCRATCH_READ/WRITE I'm going to be introducing gen7 variants, and the previous naming was going to get confusing. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:45:06 +01:00
fs_generator::generate_scratch_read(fs_inst *inst, struct brw_reg dst)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965/fs: Implement scratch reads and writes of 4 GRFs at a time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 23:47:39 +01:00
assert(inst->exec_size <= 16 || inst->force_writemask_all);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
assert(inst->mlen != 0);
i965/fs: Fix register spilling for 16-wide. Things blew up when I enabled the debug register spill code without disabling 16-wide, so I decided to just fix 16-wide spilling. We still don't generate 16-wide when register spilling happens as part of allocation (since we expect it to be slower), but now we can experiment with allowing it in some cases in the future. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 20:16:51 +01:00
brw_oword_block_read_scratch(p, dst, brw_message_reg(inst->base_mrf),
i965/fs: Use instruction execution sizes when generating scratch reads/writes Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2014-10-24 19:35:51 +01:00
inst->exec_size / 8, inst->offset);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:51:22 +01:00
void
fs_generator::generate_scratch_read_gen7(fs_inst *inst, struct brw_reg dst)
{
i965/fs: Implement scratch reads and writes of 4 GRFs at a time. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-16 23:47:39 +01:00
assert(inst->exec_size <= 16 || inst->force_writemask_all);
i965/fs: Use instruction execution sizes when generating scratch reads/writes Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2014-10-24 19:35:51 +01:00
gen7_block_read_scratch(p, dst, inst->exec_size / 8, inst->offset);
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:51:22 +01:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
void
i965/fs: Rename the existing pull constant load opcode. We're going to use another send message for handling loads with a varying per-fragment array index.
2012-11-07 18:42:34 +00:00
fs_generator::generate_uniform_pull_constant_load(fs_inst *inst,
struct brw_reg dst,
struct brw_reg index,
struct brw_reg offset)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-09 04:05:18 +00:00
assert(type_sz(dst.type) == 4);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
assert(inst->mlen != 0);
i965/fs: Communicate the pull constant block read parameters through fs_regs. I wanted to add the surface index as a variable value for UBO support, and a reg seemed like the obvious way to go. This exposes more of the information to CSE, which we'll probably want to apply to pull constant loads for UBOs eventually (you might access 4 floats in a row, each of which would produce an oword block read of the same block). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-06-20 23:41:14 +01:00
assert(index.file == BRW_IMMEDIATE_VALUE &&
index.type == BRW_REGISTER_TYPE_UD);
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 03:41:30 +01:00
uint32_t surf_index = index.ud;
i965/fs: Communicate the pull constant block read parameters through fs_regs. I wanted to add the surface index as a variable value for UBO support, and a reg seemed like the obvious way to go. This exposes more of the information to CSE, which we'll probably want to apply to pull constant loads for UBOs eventually (you might access 4 floats in a row, each of which would produce an oword block read of the same block). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-06-20 23:41:14 +01:00
assert(offset.file == BRW_IMMEDIATE_VALUE &&
offset.type == BRW_REGISTER_TYPE_UD);
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 03:41:30 +01:00
uint32_t read_offset = offset.ud;
i965/fs: Communicate the pull constant block read parameters through fs_regs. I wanted to add the surface index as a variable value for UBO support, and a reg seemed like the obvious way to go. This exposes more of the information to CSE, which we'll probably want to apply to pull constant loads for UBOs eventually (you might access 4 floats in a row, each of which would produce an oword block read of the same block). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-06-20 23:41:14 +01:00
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
brw_oword_block_read(p, dst, brw_message_reg(inst->base_mrf),
i965/fs: Communicate the pull constant block read parameters through fs_regs. I wanted to add the surface index as a variable value for UBO support, and a reg seemed like the obvious way to go. This exposes more of the information to CSE, which we'll probably want to apply to pull constant loads for UBOs eventually (you might access 4 floats in a row, each of which would produce an oword block read of the same block). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-06-20 23:41:14 +01:00
read_offset, surf_index);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965/fs: Set up gen7 UBO loads as sends from GRFs. This gives the instruction scheduler a chance to schedule between the loads, whereas before it was restricted due to the dependencies between the MRFs for setting them up. For one shader in gles3conform, it goes from getting stuck in register allocation for as long as anybody's bothered to leave it running down to 23 seconds, thanks to the LIFO scheduling. Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-05 08:06:30 +00:00
void
fs_generator::generate_uniform_pull_constant_load_gen7(fs_inst *inst,
struct brw_reg dst,
struct brw_reg index,
i965/fs: Switch to the constant cache for uniform pull constants. This reverts to using the oword block read messages for uniform pull constant loads, as used to be the case until 4c1fdae0a01b3f92ec03b61aac1d3df5. There are two important differences though: Now the L3 cacheability bits are set up correctly for UBOs (since 11f5d8a5d4fbb861ec161f68593e429cbd65d1cd), and we target the constant cache instead of the data cache. The latter used to get no L3 way allocation on boot on all platforms that existed at the time, so oword read messages wouldn't get cached on L3 regardless of the MOCS bits, what probably explains the apparent slowness of oword fetches. Constant cache loads seem to perform better than SIMD4x2 sampler loads in a number of cases, they alleviate some of the cache thrashing caused by the competition with textures for the L1/L2 sampler caches, and they allow fetching up to 128B worth of constants with a single oword fetch message. Note that IVB devices suffer from a hardware bug that leads to serialization of L3 read requests overlapping the same cacheline as result of a (on IVB buggy) mechanism of the L3 to preserve coherency. Since read requests for matching cachelines from any L3 client are not pipelined, throughput may decrease in cases where there are no non-overlapping requests left in the queue that can be processed between them. This situation should be relatively uncommon as long as we make sure that we don't use the 1/2 oword messages in cases where the shader intends to read from any other location of the same cacheline at some other point. This is generally a good idea anyway on all generations because using the 1 and 2 oword messages is expected to waste bandwidth since the minimum L3 request size for the DC is exactly 4 owords (i.e. one cacheline). A future commit will have this effect. I haven't been able to find any real-world example where this would still result in a regression on IVB, but if someone happens to find one it shouldn't be too difficult to add an IVB-specific check to have it fall back to the sampler cache for pull constant loads. Note that on SKL+ this change has the additional benefit of reducing the register footprint of pull constant loads. The following table summarizes the effect of the whole series on several shader-db stats: Total instructions Total cycles BWR: 4571248 -> 4568342 (-0.06%) 123375740 -> 123373296 (-0.00%) ELK: 3989020 -> 3985402 (-0.09%) 98757068 -> 98754058 (-0.00%) ILK: 6383591 -> 6376787 (-0.11%) 143649910 -> 143648914 (-0.00%) SNB: 7528395 -> 7501446 (-0.36%) 103503796 -> 102460370 (-1.01%) IVB: 6949221 -> 6943317 (-0.08%) 60592262 -> 60584422 (-0.01%) HSW: 6409753 -> 6403702 (-0.09%) 60609070 -> 60604414 (-0.01%) BDW: 8043467 -> 7976364 (-0.83%) 68427730 -> 68483042 (0.08%) CHV: 8045019 -> 7977916 (-0.83%) 68297426 -> 68352756 (0.08%) SKL: 8204037 -> 7939086 (-3.23%) 66583900 -> 65624378 (-1.44%) Lost->Gained Total spills Total fills BWR: 5 -> 5 1488 -> 1488 (0.00%) 1957 -> 1957 (0.00%) ELK: 5 -> 5 1489 -> 1489 (0.00%) 1958 -> 1958 (0.00%) ILK: 1 -> 4 1449 -> 1449 (0.00%) 1921 -> 1921 (0.00%) SNB: 0 -> 0 549 -> 549 (0.00%) 52 -> 52 (0.00%) IVB: 13 -> 3 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) HSW: 11 -> 0 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) BDW: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) CHV: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) SKL: 0 -> 120 1269 -> 375 (-70.45%) 1563 -> 690 (-55.85%) v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-10-26 22:25:06 +01:00
struct brw_reg payload)
i965/fs: Set up gen7 UBO loads as sends from GRFs. This gives the instruction scheduler a chance to schedule between the loads, whereas before it was restricted due to the dependencies between the MRFs for setting them up. For one shader in gles3conform, it goes from getting stuck in register allocation for as long as anybody's bothered to leave it running down to 23 seconds, thanks to the LIFO scheduling. Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-05 08:06:30 +00:00
{
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
assert(index.type == BRW_REGISTER_TYPE_UD);
i965/fs: Switch to the constant cache for uniform pull constants. This reverts to using the oword block read messages for uniform pull constant loads, as used to be the case until 4c1fdae0a01b3f92ec03b61aac1d3df5. There are two important differences though: Now the L3 cacheability bits are set up correctly for UBOs (since 11f5d8a5d4fbb861ec161f68593e429cbd65d1cd), and we target the constant cache instead of the data cache. The latter used to get no L3 way allocation on boot on all platforms that existed at the time, so oword read messages wouldn't get cached on L3 regardless of the MOCS bits, what probably explains the apparent slowness of oword fetches. Constant cache loads seem to perform better than SIMD4x2 sampler loads in a number of cases, they alleviate some of the cache thrashing caused by the competition with textures for the L1/L2 sampler caches, and they allow fetching up to 128B worth of constants with a single oword fetch message. Note that IVB devices suffer from a hardware bug that leads to serialization of L3 read requests overlapping the same cacheline as result of a (on IVB buggy) mechanism of the L3 to preserve coherency. Since read requests for matching cachelines from any L3 client are not pipelined, throughput may decrease in cases where there are no non-overlapping requests left in the queue that can be processed between them. This situation should be relatively uncommon as long as we make sure that we don't use the 1/2 oword messages in cases where the shader intends to read from any other location of the same cacheline at some other point. This is generally a good idea anyway on all generations because using the 1 and 2 oword messages is expected to waste bandwidth since the minimum L3 request size for the DC is exactly 4 owords (i.e. one cacheline). A future commit will have this effect. I haven't been able to find any real-world example where this would still result in a regression on IVB, but if someone happens to find one it shouldn't be too difficult to add an IVB-specific check to have it fall back to the sampler cache for pull constant loads. Note that on SKL+ this change has the additional benefit of reducing the register footprint of pull constant loads. The following table summarizes the effect of the whole series on several shader-db stats: Total instructions Total cycles BWR: 4571248 -> 4568342 (-0.06%) 123375740 -> 123373296 (-0.00%) ELK: 3989020 -> 3985402 (-0.09%) 98757068 -> 98754058 (-0.00%) ILK: 6383591 -> 6376787 (-0.11%) 143649910 -> 143648914 (-0.00%) SNB: 7528395 -> 7501446 (-0.36%) 103503796 -> 102460370 (-1.01%) IVB: 6949221 -> 6943317 (-0.08%) 60592262 -> 60584422 (-0.01%) HSW: 6409753 -> 6403702 (-0.09%) 60609070 -> 60604414 (-0.01%) BDW: 8043467 -> 7976364 (-0.83%) 68427730 -> 68483042 (0.08%) CHV: 8045019 -> 7977916 (-0.83%) 68297426 -> 68352756 (0.08%) SKL: 8204037 -> 7939086 (-3.23%) 66583900 -> 65624378 (-1.44%) Lost->Gained Total spills Total fills BWR: 5 -> 5 1488 -> 1488 (0.00%) 1957 -> 1957 (0.00%) ELK: 5 -> 5 1489 -> 1489 (0.00%) 1958 -> 1958 (0.00%) ILK: 1 -> 4 1449 -> 1449 (0.00%) 1921 -> 1921 (0.00%) SNB: 0 -> 0 549 -> 549 (0.00%) 52 -> 52 (0.00%) IVB: 13 -> 3 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) HSW: 11 -> 0 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) BDW: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) CHV: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) SKL: 0 -> 120 1269 -> 375 (-70.45%) 1563 -> 690 (-55.85%) v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-10-26 22:25:06 +01:00
assert(payload.file == BRW_GENERAL_REGISTER_FILE);
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-09 04:05:18 +00:00
assert(type_sz(dst.type) == 4);
i965/skl: Always use a header for SIMD4x2 sampler messages SKL+ overloads the SIMD4x2 SIMD mode to mean either SIMD8D or SIMD4x2 depending on bit 22 in the message header. If the bit is 0 or there is no header we get SIMD8D. We always wand SIMD4x2 in vec4 and for fs pull constants, so use a message header in those cases and set bit 22 there. Based on an initial patch from Ken. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Kristian Høgsberg <krh@bitplanet.net>
2014-12-10 22:59:26 +00:00
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
if (index.file == BRW_IMMEDIATE_VALUE) {
i965/fs: Switch to the constant cache for uniform pull constants. This reverts to using the oword block read messages for uniform pull constant loads, as used to be the case until 4c1fdae0a01b3f92ec03b61aac1d3df5. There are two important differences though: Now the L3 cacheability bits are set up correctly for UBOs (since 11f5d8a5d4fbb861ec161f68593e429cbd65d1cd), and we target the constant cache instead of the data cache. The latter used to get no L3 way allocation on boot on all platforms that existed at the time, so oword read messages wouldn't get cached on L3 regardless of the MOCS bits, what probably explains the apparent slowness of oword fetches. Constant cache loads seem to perform better than SIMD4x2 sampler loads in a number of cases, they alleviate some of the cache thrashing caused by the competition with textures for the L1/L2 sampler caches, and they allow fetching up to 128B worth of constants with a single oword fetch message. Note that IVB devices suffer from a hardware bug that leads to serialization of L3 read requests overlapping the same cacheline as result of a (on IVB buggy) mechanism of the L3 to preserve coherency. Since read requests for matching cachelines from any L3 client are not pipelined, throughput may decrease in cases where there are no non-overlapping requests left in the queue that can be processed between them. This situation should be relatively uncommon as long as we make sure that we don't use the 1/2 oword messages in cases where the shader intends to read from any other location of the same cacheline at some other point. This is generally a good idea anyway on all generations because using the 1 and 2 oword messages is expected to waste bandwidth since the minimum L3 request size for the DC is exactly 4 owords (i.e. one cacheline). A future commit will have this effect. I haven't been able to find any real-world example where this would still result in a regression on IVB, but if someone happens to find one it shouldn't be too difficult to add an IVB-specific check to have it fall back to the sampler cache for pull constant loads. Note that on SKL+ this change has the additional benefit of reducing the register footprint of pull constant loads. The following table summarizes the effect of the whole series on several shader-db stats: Total instructions Total cycles BWR: 4571248 -> 4568342 (-0.06%) 123375740 -> 123373296 (-0.00%) ELK: 3989020 -> 3985402 (-0.09%) 98757068 -> 98754058 (-0.00%) ILK: 6383591 -> 6376787 (-0.11%) 143649910 -> 143648914 (-0.00%) SNB: 7528395 -> 7501446 (-0.36%) 103503796 -> 102460370 (-1.01%) IVB: 6949221 -> 6943317 (-0.08%) 60592262 -> 60584422 (-0.01%) HSW: 6409753 -> 6403702 (-0.09%) 60609070 -> 60604414 (-0.01%) BDW: 8043467 -> 7976364 (-0.83%) 68427730 -> 68483042 (0.08%) CHV: 8045019 -> 7977916 (-0.83%) 68297426 -> 68352756 (0.08%) SKL: 8204037 -> 7939086 (-3.23%) 66583900 -> 65624378 (-1.44%) Lost->Gained Total spills Total fills BWR: 5 -> 5 1488 -> 1488 (0.00%) 1957 -> 1957 (0.00%) ELK: 5 -> 5 1489 -> 1489 (0.00%) 1958 -> 1958 (0.00%) ILK: 1 -> 4 1449 -> 1449 (0.00%) 1921 -> 1921 (0.00%) SNB: 0 -> 0 549 -> 549 (0.00%) 52 -> 52 (0.00%) IVB: 13 -> 3 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) HSW: 11 -> 0 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) BDW: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) CHV: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) SKL: 0 -> 120 1269 -> 375 (-70.45%) 1563 -> 690 (-55.85%) v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-10-26 22:25:06 +01:00
const uint32_t surf_index = index.ud;
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_inst *send = brw_next_insn(p, BRW_OPCODE_SEND);
brw_pop_insn_state(p);
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 18:50:20 +01:00
brw_inst_set_sfid(devinfo, send, GEN6_SFID_DATAPORT_CONSTANT_CACHE);
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-09 04:05:18 +00:00
brw_set_dest(p, send, retype(dst, BRW_REGISTER_TYPE_UD));
brw_set_src0(p, send, retype(payload, BRW_REGISTER_TYPE_UD));
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 18:50:20 +01:00
brw_set_desc(p, send,
brw_message_desc(devinfo, 1, DIV_ROUND_UP(inst->size_written,
REG_SIZE), true) |
brw_dp_read_desc(devinfo, surf_index,
BRW_DATAPORT_OWORD_BLOCK_DWORDS(inst->exec_size),
GEN7_DATAPORT_DC_OWORD_BLOCK_READ,
BRW_DATAPORT_READ_TARGET_DATA_CACHE));
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
i965/fs: Switch to the constant cache for uniform pull constants. This reverts to using the oword block read messages for uniform pull constant loads, as used to be the case until 4c1fdae0a01b3f92ec03b61aac1d3df5. There are two important differences though: Now the L3 cacheability bits are set up correctly for UBOs (since 11f5d8a5d4fbb861ec161f68593e429cbd65d1cd), and we target the constant cache instead of the data cache. The latter used to get no L3 way allocation on boot on all platforms that existed at the time, so oword read messages wouldn't get cached on L3 regardless of the MOCS bits, what probably explains the apparent slowness of oword fetches. Constant cache loads seem to perform better than SIMD4x2 sampler loads in a number of cases, they alleviate some of the cache thrashing caused by the competition with textures for the L1/L2 sampler caches, and they allow fetching up to 128B worth of constants with a single oword fetch message. Note that IVB devices suffer from a hardware bug that leads to serialization of L3 read requests overlapping the same cacheline as result of a (on IVB buggy) mechanism of the L3 to preserve coherency. Since read requests for matching cachelines from any L3 client are not pipelined, throughput may decrease in cases where there are no non-overlapping requests left in the queue that can be processed between them. This situation should be relatively uncommon as long as we make sure that we don't use the 1/2 oword messages in cases where the shader intends to read from any other location of the same cacheline at some other point. This is generally a good idea anyway on all generations because using the 1 and 2 oword messages is expected to waste bandwidth since the minimum L3 request size for the DC is exactly 4 owords (i.e. one cacheline). A future commit will have this effect. I haven't been able to find any real-world example where this would still result in a regression on IVB, but if someone happens to find one it shouldn't be too difficult to add an IVB-specific check to have it fall back to the sampler cache for pull constant loads. Note that on SKL+ this change has the additional benefit of reducing the register footprint of pull constant loads. The following table summarizes the effect of the whole series on several shader-db stats: Total instructions Total cycles BWR: 4571248 -> 4568342 (-0.06%) 123375740 -> 123373296 (-0.00%) ELK: 3989020 -> 3985402 (-0.09%) 98757068 -> 98754058 (-0.00%) ILK: 6383591 -> 6376787 (-0.11%) 143649910 -> 143648914 (-0.00%) SNB: 7528395 -> 7501446 (-0.36%) 103503796 -> 102460370 (-1.01%) IVB: 6949221 -> 6943317 (-0.08%) 60592262 -> 60584422 (-0.01%) HSW: 6409753 -> 6403702 (-0.09%) 60609070 -> 60604414 (-0.01%) BDW: 8043467 -> 7976364 (-0.83%) 68427730 -> 68483042 (0.08%) CHV: 8045019 -> 7977916 (-0.83%) 68297426 -> 68352756 (0.08%) SKL: 8204037 -> 7939086 (-3.23%) 66583900 -> 65624378 (-1.44%) Lost->Gained Total spills Total fills BWR: 5 -> 5 1488 -> 1488 (0.00%) 1957 -> 1957 (0.00%) ELK: 5 -> 5 1489 -> 1489 (0.00%) 1958 -> 1958 (0.00%) ILK: 1 -> 4 1449 -> 1449 (0.00%) 1921 -> 1921 (0.00%) SNB: 0 -> 0 549 -> 549 (0.00%) 52 -> 52 (0.00%) IVB: 13 -> 3 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) HSW: 11 -> 0 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) BDW: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) CHV: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) SKL: 0 -> 120 1269 -> 375 (-70.45%) 1563 -> 690 (-55.85%) v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-10-26 22:25:06 +01:00
} else {
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
struct brw_reg addr = vec1(retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD));
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
/* a0.0 = surf_index & 0xff */
brw_inst *insn_and = brw_next_insn(p, BRW_OPCODE_AND);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_exec_size(p->devinfo, insn_and, BRW_EXECUTE_1);
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
brw_set_dest(p, insn_and, addr);
brw_set_src0(p, insn_and, vec1(retype(index, BRW_REGISTER_TYPE_UD)));
brw_set_src1(p, insn_and, brw_imm_ud(0x0ff));
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 13:44:24 +00:00
/* dst = send(payload, a0.0 | <descriptor>) */
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 18:50:20 +01:00
brw_send_indirect_message(
i965/fs: Switch to the constant cache for uniform pull constants. This reverts to using the oword block read messages for uniform pull constant loads, as used to be the case until 4c1fdae0a01b3f92ec03b61aac1d3df5. There are two important differences though: Now the L3 cacheability bits are set up correctly for UBOs (since 11f5d8a5d4fbb861ec161f68593e429cbd65d1cd), and we target the constant cache instead of the data cache. The latter used to get no L3 way allocation on boot on all platforms that existed at the time, so oword read messages wouldn't get cached on L3 regardless of the MOCS bits, what probably explains the apparent slowness of oword fetches. Constant cache loads seem to perform better than SIMD4x2 sampler loads in a number of cases, they alleviate some of the cache thrashing caused by the competition with textures for the L1/L2 sampler caches, and they allow fetching up to 128B worth of constants with a single oword fetch message. Note that IVB devices suffer from a hardware bug that leads to serialization of L3 read requests overlapping the same cacheline as result of a (on IVB buggy) mechanism of the L3 to preserve coherency. Since read requests for matching cachelines from any L3 client are not pipelined, throughput may decrease in cases where there are no non-overlapping requests left in the queue that can be processed between them. This situation should be relatively uncommon as long as we make sure that we don't use the 1/2 oword messages in cases where the shader intends to read from any other location of the same cacheline at some other point. This is generally a good idea anyway on all generations because using the 1 and 2 oword messages is expected to waste bandwidth since the minimum L3 request size for the DC is exactly 4 owords (i.e. one cacheline). A future commit will have this effect. I haven't been able to find any real-world example where this would still result in a regression on IVB, but if someone happens to find one it shouldn't be too difficult to add an IVB-specific check to have it fall back to the sampler cache for pull constant loads. Note that on SKL+ this change has the additional benefit of reducing the register footprint of pull constant loads. The following table summarizes the effect of the whole series on several shader-db stats: Total instructions Total cycles BWR: 4571248 -> 4568342 (-0.06%) 123375740 -> 123373296 (-0.00%) ELK: 3989020 -> 3985402 (-0.09%) 98757068 -> 98754058 (-0.00%) ILK: 6383591 -> 6376787 (-0.11%) 143649910 -> 143648914 (-0.00%) SNB: 7528395 -> 7501446 (-0.36%) 103503796 -> 102460370 (-1.01%) IVB: 6949221 -> 6943317 (-0.08%) 60592262 -> 60584422 (-0.01%) HSW: 6409753 -> 6403702 (-0.09%) 60609070 -> 60604414 (-0.01%) BDW: 8043467 -> 7976364 (-0.83%) 68427730 -> 68483042 (0.08%) CHV: 8045019 -> 7977916 (-0.83%) 68297426 -> 68352756 (0.08%) SKL: 8204037 -> 7939086 (-3.23%) 66583900 -> 65624378 (-1.44%) Lost->Gained Total spills Total fills BWR: 5 -> 5 1488 -> 1488 (0.00%) 1957 -> 1957 (0.00%) ELK: 5 -> 5 1489 -> 1489 (0.00%) 1958 -> 1958 (0.00%) ILK: 1 -> 4 1449 -> 1449 (0.00%) 1921 -> 1921 (0.00%) SNB: 0 -> 0 549 -> 549 (0.00%) 52 -> 52 (0.00%) IVB: 13 -> 3 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) HSW: 11 -> 0 1271 -> 1271 (0.00%) 1162 -> 1162 (0.00%) BDW: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) CHV: 12 -> 0 1340 -> 1340 (0.00%) 1452 -> 1452 (0.00%) SKL: 0 -> 120 1269 -> 375 (-70.45%) 1563 -> 690 (-55.85%) v3: Non-trivial rebase. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-10-26 22:25:06 +01:00
p, GEN6_SFID_DATAPORT_CONSTANT_CACHE,
i965/fs: Expose arbitrary pull constant load sizes to the IR. Change the FS generator to ask the dataport for enough owords worth of constants to fill the execution size of the instruction -- Which means that the visitor now needs to set the execution size correctly for uniform pull constant load instructions, which we were kind of neglecting until now. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-12-09 04:05:18 +00:00
retype(dst, BRW_REGISTER_TYPE_UD),
intel/eu: Use descriptor constructors for dataport read messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-07 18:50:20 +01:00
retype(payload, BRW_REGISTER_TYPE_UD), addr,
brw_message_desc(devinfo, 1,
DIV_ROUND_UP(inst->size_written, REG_SIZE), true) |
brw_dp_read_desc(devinfo, 0 /* surface */,
BRW_DATAPORT_OWORD_BLOCK_DWORDS(inst->exec_size),
GEN7_DATAPORT_DC_OWORD_BLOCK_READ,
BRW_DATAPORT_READ_TARGET_DATA_CACHE));
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
brw_pop_insn_state(p);
}
i965/fs: Set up gen7 UBO loads as sends from GRFs. This gives the instruction scheduler a chance to schedule between the loads, whereas before it was restricted due to the dependencies between the MRFs for setting them up. For one shader in gles3conform, it goes from getting stuck in register allocation for as long as anybody's bothered to leave it running down to 23 seconds, thanks to the LIFO scheduling. Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-05 08:06:30 +00:00
}
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
void
i965/fs: Rename Gen4 physical varying pull constant load opcode. For consistency with the Gen7 variant. I'm not doing the same to the uniform pull constant message at this point because the non-GEN7 one is still overloaded to be either an expression-like logical instruction or a Gen4-specific physical send message. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 21:03:31 +01:00
fs_generator::generate_varying_pull_constant_load_gen4(fs_inst *inst,
struct brw_reg dst,
struct brw_reg index)
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen < 7); /* Should use the gen7 variant. */
i965: Change header_present to header_size in backend_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-24 17:17:32 +00:00
assert(inst->header_size != 0);
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
assert(inst->mlen);
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
assert(index.file == BRW_IMMEDIATE_VALUE &&
index.type == BRW_REGISTER_TYPE_UD);
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 03:41:30 +01:00
uint32_t surf_index = index.ud;
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
uint32_t simd_mode, rlen, msg_type;
i965/fs: Clean up remaining uses of dispatch_width in the generator. Most of these are bugs because the intended execution size of an instruction and the dispatch width of the shader aren't necessarily the same (especially in SIMD32 programs). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 08:13:33 +01:00
if (inst->exec_size == 16) {
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
rlen = 8;
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
} else {
i965/fs: Clean up remaining uses of dispatch_width in the generator. Most of these are bugs because the intended execution size of an instruction and the dispatch width of the shader aren't necessarily the same (especially in SIMD32 programs). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 08:13:33 +01:00
assert(inst->exec_size == 8);
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD8;
rlen = 4;
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
}
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen >= 5)
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
msg_type = GEN5_SAMPLER_MESSAGE_SAMPLE_LD;
else {
/* We always use the SIMD16 message so that we only have to load U, and
* not V or R.
*/
msg_type = BRW_SAMPLER_MESSAGE_SIMD16_LD;
assert(inst->mlen == 3);
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
assert(inst->size_written == 8 * REG_SIZE);
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
rlen = 8;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
}
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
struct brw_reg header = brw_vec8_grf(0, 0);
gen6_resolve_implied_move(p, &header, inst->base_mrf);
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 22:29:25 +01:00
brw_inst *send = brw_next_insn(p, BRW_OPCODE_SEND);
i965/eu: Fix a bunch of compression control bugs in the generator. Most of these were resetting quarter control to zero incorrectly even though everything they needed to do was disable instruction compression -- The brw_SAMPLE() case was doing the right thing but it can be simplified slightly by using the new compression control interface. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 23:29:27 +01:00
brw_inst_set_compression(devinfo, send, false);
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_inst_set_sfid(devinfo, send, BRW_SFID_SAMPLER);
i965: Retype pre-Gen6 varying pull load destination to UW. This sets up the proper execution mask for sends in SIMD16 mode. Fixes Piglit's glsl-fs-normalmatrix, glsl-fs-uniform-array-2, glsl-fs-uniform-array-6, and glsl-fs-uniform-array-7 on Ironlake, which regressed when I enabled SIMD16 pull parameter support in commit b207e88b25e526d0f1ada7b19605b880a27866dc. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2014-04-17 04:15:23 +01:00
brw_set_dest(p, send, retype(dst, BRW_REGISTER_TYPE_UW));
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
brw_set_src0(p, send, header);
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
if (devinfo->gen < 6)
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_base_mrf(p->devinfo, send, inst->base_mrf);
i965/fs: Use LD messages for pre-gen7 varying-index uniform loads This comes at a minor performance cost at the moment (-3.2% +/- 0.2%, n=14 on my GM45 forced to load all uniforms through the varying-index path), but we get a whole vec4 at a time to reuse in the next commit. v2: Fix comment about channels in the other message. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> NOTE: This is a candidate for the 9.1 branch.
2013-03-18 17:16:42 +00:00
/* Our surface is set up as floats, regardless of what actual data is
* stored in it.
*/
uint32_t return_format = BRW_SAMPLER_RETURN_FORMAT_FLOAT32;
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_set_desc(p, send,
brw_message_desc(devinfo, inst->mlen, rlen, inst->header_size) |
brw_sampler_desc(devinfo, surf_index,
0, /* sampler (unused) */
msg_type, simd_mode, return_format));
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
}
void
fs_generator::generate_varying_pull_constant_load_gen7(fs_inst *inst,
struct brw_reg dst,
struct brw_reg index,
struct brw_reg offset)
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
/* Varying-offset pull constant loads are treated as a normal expression on
* gen7, so the fact that it's a send message is hidden at the IR level.
*/
i965: Change header_present to header_size in backend_instruction Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-24 17:17:32 +00:00
assert(inst->header_size == 0);
intel/fs: Initialize mlen for gen7 varying pull constant load messages. This makes the message length available at the IR level, which should save some guesswork in a future commit. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-12-27 03:08:10 +00:00
assert(inst->mlen);
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
assert(index.type == BRW_REGISTER_TYPE_UD);
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
intel/fs: Initialize mlen for gen7 varying pull constant load messages. This makes the message length available at the IR level, which should save some guesswork in a future commit. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-12-27 03:08:10 +00:00
uint32_t simd_mode, rlen;
i965/fs: Clean up remaining uses of dispatch_width in the generator. Most of these are bugs because the intended execution size of an instruction and the dispatch width of the shader aren't necessarily the same (especially in SIMD32 programs). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 08:13:33 +01:00
if (inst->exec_size == 16) {
i965/fs: Improve performance of varying-index uniform loads on IVB. Like we have done for the VS and for constant-index uniform loads, we use the sampler engine to get caching in front of the L3 to avoid tickling the IVB L3 bug. This is also a bit of a functional change, as we're now loading a vec4 instead of a single dword, though we're not taking advantage of the other 3 components of the vec4 (yet). With the driver hacked to always take the varying-index path for all uniforms, improves performance of my old GLSL demo by 315% +/- 2% (n=4). This a major fix for some blur shaders in compositors from the varying-index uniforms support I introduced in 9.1. v2: Move old offset computation into the pre-gen7 path. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=61554 NOTE: This is a candidate for the 9.1 branch.
2013-03-13 21:48:55 +00:00
rlen = 8;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD16;
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
} else {
i965/fs: Clean up remaining uses of dispatch_width in the generator. Most of these are bugs because the intended execution size of an instruction and the dispatch width of the shader aren't necessarily the same (especially in SIMD32 programs). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 08:13:33 +01:00
assert(inst->exec_size == 8);
i965/fs: Improve performance of varying-index uniform loads on IVB. Like we have done for the VS and for constant-index uniform loads, we use the sampler engine to get caching in front of the L3 to avoid tickling the IVB L3 bug. This is also a bit of a functional change, as we're now loading a vec4 instead of a single dword, though we're not taking advantage of the other 3 components of the vec4 (yet). With the driver hacked to always take the varying-index path for all uniforms, improves performance of my old GLSL demo by 315% +/- 2% (n=4). This a major fix for some blur shaders in compositors from the varying-index uniforms support I introduced in 9.1. v2: Move old offset computation into the pre-gen7 path. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=61554 NOTE: This is a candidate for the 9.1 branch.
2013-03-13 21:48:55 +00:00
rlen = 4;
simd_mode = BRW_SAMPLER_SIMD_MODE_SIMD8;
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
}
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
if (index.file == BRW_IMMEDIATE_VALUE) {
i965/fs: Track the binding table size in brw_wm_prog_data. By tracking the maximum surface index used by the shader, we know just how small we can make the binding table. Since it depends entirely on the shader program, we can just compute it once at compile time, rather than at binding table emit time (which happens during drawing). v2: Store binding_table_size, rather than max_surface_index, for consistency with the VS (which needs to be able to represent 0 surfaces). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
2013-08-15 03:49:33 +01:00
i965: Make 'dw1' and 'bits' unnamed structures in brw_reg. Generated by sed -i -e 's/\.bits\././g' *.c *.h *.cpp sed -i -e 's/dw1\.//g' *.c *.h *.cpp and then reverting changes to comments in gen7_blorp.cpp and brw_fs_generator.cpp. There wasn't any utility offered by forcing the programmer to list these to access their fields. Removing them will reduce churn in future commits. This is C11 (and gcc has apparently supported it for sometime "compatibility with other compilers") See https://gcc.gnu.org/onlinedocs/gcc/Unnamed-Fields.html Reviewed-by: Emil Velikov <emil.velikov@collabora.co.uk> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-10-23 03:41:30 +01:00
uint32_t surf_index = index.ud;
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
brw_inst *send = brw_next_insn(p, BRW_OPCODE_SEND);
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_inst_set_sfid(devinfo, send, BRW_SFID_SAMPLER);
i965/fs: Use the UW type for the destination of VARYING_PULL_CONSTANT_LOAD instructions Using a floating-point type doesn't usually cause hangs on my HSW, but the simulator complains about it quite a bit. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-08 23:26:24 +01:00
brw_set_dest(p, send, retype(dst, BRW_REGISTER_TYPE_UW));
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
brw_set_src0(p, send, offset);
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_set_desc(p, send,
intel/fs: Initialize mlen for gen7 varying pull constant load messages. This makes the message length available at the IR level, which should save some guesswork in a future commit. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-12-27 03:08:10 +00:00
brw_message_desc(devinfo, inst->mlen, rlen, false) |
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_sampler_desc(devinfo, surf_index,
0, /* LD message ignores sampler unit */
GEN5_SAMPLER_MESSAGE_SAMPLE_LD,
simd_mode, 0));
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
} else {
struct brw_reg addr = vec1(retype(brw_address_reg(0), BRW_REGISTER_TYPE_UD));
brw_push_insn_state(p);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
/* a0.0 = surf_index & 0xff */
brw_inst *insn_and = brw_next_insn(p, BRW_OPCODE_AND);
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_exec_size(p->devinfo, insn_and, BRW_EXECUTE_1);
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
brw_set_dest(p, insn_and, addr);
brw_set_src0(p, insn_and, vec1(retype(index, BRW_REGISTER_TYPE_UD)));
brw_set_src1(p, insn_and, brw_imm_ud(0x0ff));
i965: Don't disable exec masking for sampler message sends. This was telling the sampler to do texture fetches for *all* channels in the non-constant surface index case, what could have reduced throughput unnecessarily when some of the channels were disabled by control flow. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-26 15:24:03 +00:00
brw_pop_insn_state(p);
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 13:44:24 +00:00
/* dst = send(offset, a0.0 | <descriptor>) */
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_send_indirect_message(
i965: Factor out logic to build a send message instruction with indirect descriptor. This is going to be useful because the Gen7+ uniform and varying pull constant, texturing, typed and untyped surface read, write, and atomic generation code on the vec4 and fs back-end all require the same logic to handle conditionally indirect surface indices. In pseudocode: | if (surface.file == BRW_IMMEDIATE_VALUE) { | inst = brw_SEND(p, dst, payload); | set_descriptor_control_bits(inst, surface, ...); | } else { | inst = brw_OR(p, addr, surface, 0); | set_descriptor_control_bits(inst, ...); | inst = brw_SEND(p, dst, payload); | set_indirect_send_descriptor(inst, addr); | } This patch abstracts out this frequently recurring pattern so we can now write: | inst = brw_send_indirect_message(p, sfid, dst, payload, surface) | set_descriptor_control_bits(inst, ...); without worrying about handling the immediate and indirect surface index cases explicitly. v2: Rebase. Improve documentatation and commit message. (Topi) Preserve UW destination type cargo-cult. (Topi, Ken, Matt) Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2015-03-19 13:44:24 +00:00
p, BRW_SFID_SAMPLER, retype(dst, BRW_REGISTER_TYPE_UW),
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
offset, addr,
intel/fs: Initialize mlen for gen7 varying pull constant load messages. This makes the message length available at the IR level, which should save some guesswork in a future commit. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-12-27 03:08:10 +00:00
brw_message_desc(devinfo, inst->mlen, rlen, false) |
intel/eu: Use descriptor constructors for sampler messages. v2: Use SET_BITS macro instead of left shift (Ken). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-06-02 23:15:15 +01:00
brw_sampler_desc(devinfo,
0 /* surface */,
0 /* sampler */,
GEN5_SAMPLER_MESSAGE_SAMPLE_LD,
simd_mode,
0));
i965/fs: Generate indirect sends for nonconstant UBO array accesses Signed-off-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-02 03:27:21 +01:00
}
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
}
i965/fs: Add FS_OPCODE_MOV_DISPATCH_TO_FLAGS to fragment shader backend. In order to compute centroid varyings correctly, the fragment shader needs to be able to load the current pixel/sample mask into a flag register. This patch adds an opcode to the fragment shader back-end to do this; the opcode gets translated into the instruction mov(1) f0<1>UW g1.14<0,1,0>UW { align1 WE_all } Since this instruction clobbers f0, instruction scheduling has to treat it the same as instructions that have a conditional modifier. Reviewed-by: Eric Anholt <eric@anholt.net>
2012-06-18 22:50:04 +01:00
i965/fs: add generator support for pixel interpolator query V5: - Split into separate opcodes - Pass message data in src1 immediate - Put noperspective bit in fs_inst rather than adding any junk to backend_instruction Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-18 08:13:13 +00:00
void
fs_generator::generate_pixel_interpolator_query(fs_inst *inst,
struct brw_reg dst,
struct brw_reg src,
struct brw_reg msg_data,
unsigned msg_type)
{
intel/fs: Rework INTERPOLATE_AT_PER_SLOT_OFFSET This reworks INTERPOLATE_AT_PER_SLOT_OFFSET to work more like an ALU operation and less like a send. This is less code over-all and, as a side-effect, it now properly handles execution groups and lowering so SIMD32 support just falls out. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 02:06:13 +01:00
const bool has_payload = inst->src[0].file != BAD_FILE;
i965/fs: Handle non-const sample number in interpolateAtSample If a non-const sample number is given to interpolateAtSample it will now generate an indirect send message with the sample ID similar to how non-const sampler array indexing works. Previously non-const values were ignored and instead it ended up using a constant 0 value. The generator will try to determine if the sample ID is dynamically uniform via nir_src_is_dynamically_uniform. If not it will query the pixel interpolator in a loop, once for each different live sample number. The next live sample number is found using emit_uniformize. If multiple live channels have the same sample number then they will be handled in a single iteration of the loop. The loop is necessary because the indirect send message doesn't seem to have a way to specify a different value for each fragment. This fixes the following two Piglit tests: arb_gpu_shader5-interpolateAtSample-nonconst arb_gpu_shader5-interpolateAtSample-dynamically-nonuniform v2: Handle dynamically non-uniform sample ids. v3: Remove the BREAK instruction and predicate the WHILE directly. Make the tokens arrays const. (Matt Turner) v4: Iterate over the live channels instead of each possible sample number. v5: Don't special case immediate values in brw_pixel_interpolator_query. Make a better wrapper for the function to set up the PI send instruction. Ensure that the SHL instructions are scalar. (Francisco Jerez). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-07-17 14:40:03 +01:00
assert(msg_data.type == BRW_REGISTER_TYPE_UD);
intel/fs: Rework INTERPOLATE_AT_PER_SLOT_OFFSET This reworks INTERPOLATE_AT_PER_SLOT_OFFSET to work more like an ALU operation and less like a send. This is less code over-all and, as a side-effect, it now properly handles execution groups and lowering so SIMD32 support just falls out. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 02:06:13 +01:00
assert(inst->size_written % REG_SIZE == 0);
i965/fs: add generator support for pixel interpolator query V5: - Split into separate opcodes - Pass message data in src1 immediate - Put noperspective bit in fs_inst rather than adding any junk to backend_instruction Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-18 08:13:13 +00:00
brw_pixel_interpolator_query(p,
retype(dst, BRW_REGISTER_TYPE_UW),
intel/fs: Rework INTERPOLATE_AT_PER_SLOT_OFFSET This reworks INTERPOLATE_AT_PER_SLOT_OFFSET to work more like an ALU operation and less like a send. This is less code over-all and, as a side-effect, it now properly handles execution groups and lowering so SIMD32 support just falls out. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 02:06:13 +01:00
/* If we don't have a payload, what we send doesn't matter */
has_payload ? src : brw_vec8_grf(0, 0),
i965/fs: add generator support for pixel interpolator query V5: - Split into separate opcodes - Pass message data in src1 immediate - Put noperspective bit in fs_inst rather than adding any junk to backend_instruction Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-18 08:13:13 +00:00
inst->pi_noperspective,
msg_type,
i965/fs: Handle non-const sample number in interpolateAtSample If a non-const sample number is given to interpolateAtSample it will now generate an indirect send message with the sample ID similar to how non-const sampler array indexing works. Previously non-const values were ignored and instead it ended up using a constant 0 value. The generator will try to determine if the sample ID is dynamically uniform via nir_src_is_dynamically_uniform. If not it will query the pixel interpolator in a loop, once for each different live sample number. The next live sample number is found using emit_uniformize. If multiple live channels have the same sample number then they will be handled in a single iteration of the loop. The loop is necessary because the indirect send message doesn't seem to have a way to specify a different value for each fragment. This fixes the following two Piglit tests: arb_gpu_shader5-interpolateAtSample-nonconst arb_gpu_shader5-interpolateAtSample-dynamically-nonuniform v2: Handle dynamically non-uniform sample ids. v3: Remove the BREAK instruction and predicate the WHILE directly. Make the tokens arrays const. (Matt Turner) v4: Iterate over the live channels instead of each possible sample number. v5: Don't special case immediate values in brw_pixel_interpolator_query. Make a better wrapper for the function to set up the PI send instruction. Ensure that the SHL instructions are scalar. (Francisco Jerez). Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2015-07-17 14:40:03 +01:00
msg_data,
intel/fs: Rework INTERPOLATE_AT_PER_SLOT_OFFSET This reworks INTERPOLATE_AT_PER_SLOT_OFFSET to work more like an ALU operation and less like a send. This is less code over-all and, as a side-effect, it now properly handles execution groups and lowering so SIMD32 support just falls out. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 02:06:13 +01:00
has_payload ? 2 * inst->exec_size / 8 : 1,
i965/fs: Replace fs_inst::regs_written with ::size_written field in bytes. The previous regs_written field can be recovered by rewriting each rvalue reference of regs_written like 'x = i.regs_written' to 'x = DIV_ROUND_UP(i.size_written, reg_unit)', and each lvalue reference like 'i.regs_written = x' to 'i.size_written = x * reg_unit'. For the same reason as in the previous patches, this doesn't attempt to be particularly clever about simplifying the result in the interest of keeping the rather lengthy patch as obvious as possible. I'll come back later to clean up any ugliness introduced here. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-09-07 21:38:20 +01:00
inst->size_written / REG_SIZE);
i965/fs: add generator support for pixel interpolator query V5: - Split into separate opcodes - Pass message data in src1 immediate - Put noperspective bit in fs_inst rather than adding any junk to backend_instruction Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-18 08:13:13 +00:00
}
i965: Add FS backend for builtin gl_SampleID V2: - Update comments - Add compute_sample_id variables in brw_wm_prog_key - Add a special backend instruction to compute sample_id. V3: - Make changes to support simd16 mode. Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-25 00:17:08 +01:00
/* Sets vstride=1, width=4, hstride=0 of register src1 during
* the ADD instruction.
*/
void
fs_generator::generate_set_sample_id(fs_inst *inst,
struct brw_reg dst,
struct brw_reg src0,
struct brw_reg src1)
{
assert(dst.type == BRW_REGISTER_TYPE_D ||
dst.type == BRW_REGISTER_TYPE_UD);
assert(src0.type == BRW_REGISTER_TYPE_D ||
src0.type == BRW_REGISTER_TYPE_UD);
intel/fs: Fix codegen of FS_OPCODE_SET_SAMPLE_ID for SIMD32. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-04-26 03:28:21 +01:00
const struct brw_reg reg = stride(src1, 1, 4, 0);
const unsigned lower_size = MIN2(inst->exec_size,
devinfo->gen >= 8 ? 16 : 8);
for (unsigned i = 0; i < inst->exec_size / lower_size; i++) {
brw_inst *insn = brw_ADD(p, offset(dst, i * lower_size / 8),
offset(src0, (src0.vstride == 0 ? 0 : (1 << (src0.vstride - 1)) *
(i * lower_size / (1 << src0.width))) *
type_sz(src0.type) / REG_SIZE),
suboffset(reg, i * lower_size / 4));
brw_inst_set_exec_size(devinfo, insn, cvt(lower_size) - 1);
brw_inst_set_group(devinfo, insn, inst->group + lower_size * i);
brw_inst_set_compression(devinfo, insn, lower_size > 8);
i965/brw_reg: Add a firsthalf function and use it in the generator Right now, this function is a no-op but it indicates that we intend to only use the first half of the 16-wide register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-13 20:23:47 +01:00
}
i965: Add FS backend for builtin gl_SampleID V2: - Update comments - Add compute_sample_id variables in brw_wm_prog_key - Add a special backend instruction to compute sample_id. V3: - Make changes to support simd16 mode. Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-25 00:17:08 +01:00
}
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
void
intel/compiler: Silence unused parameter warnings in generate_foo methods Since all of the fs_generator::generate_foo methods take a fs_inst * as the first parameter, just remove the name to quiet the compiler. src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_barrier(fs_inst*, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:743:41: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_barrier(fs_inst *inst, struct brw_reg src) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_discard_jump(fs_inst*)’: src/intel/compiler/brw_fs_generator.cpp:1326:46: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_discard_jump(fs_inst *inst) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_pack_half_2x16_split(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1675:54: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_pack_half_2x16_split(fs_inst *inst, ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_shader_time_add(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1743:49: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_shader_time_add(fs_inst *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_set_simd4x2_header_gen9(brw_codegen*, brw::vec4_instruction*, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1412:52: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_mov_indirect(brw_codegen*, brw::vec4_instruction*, brw_reg, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1430:41: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp:1432:63: warning: unused parameter ‘length’ [-Wunused-parameter] struct brw_reg indirect, struct brw_reg length) ^~~~~~ Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2018-03-29 00:29:45 +01:00
fs_generator::generate_pack_half_2x16_split(fs_inst *,
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
struct brw_reg dst,
struct brw_reg x,
struct brw_reg y)
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
assert(dst.type == BRW_REGISTER_TYPE_UD);
i965: Fix assignment instead of comparison in asserts. Fixes side effect in assertion defects reported by Coverity. Note: This is a candidate for the 9.1 branch. Signed-off-by: Vinson Lee <vlee@freedesktop.org> Reviewed-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-26 07:27:50 +00:00
assert(x.type == BRW_REGISTER_TYPE_F);
assert(y.type == BRW_REGISTER_TYPE_F);
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
/* From the Ivybridge PRM, Vol4, Part3, Section 6.27 f32to16:
*
* Because this instruction does not have a 16-bit floating-point type,
* the destination data type must be Word (W).
*
* The destination must be DWord-aligned and specify a horizontal stride
* (HorzStride) of 2. The 16-bit result is stored in the lower word of
* each destination channel and the upper word is not modified.
*/
i965/fs: Replace ud_reg_to_w() with a more general helper function. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 15:58:49 +00:00
struct brw_reg dst_w = spread(retype(dst, BRW_REGISTER_TYPE_W), 2);
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
i965/fs: Replace ud_reg_to_w() with a more general helper function. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-04 15:58:49 +00:00
/* Give each 32-bit channel of dst the form below, where "." means
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
* unchanged.
* 0x....hhhh
*/
brw_F32TO16(p, dst_w, y);
/* Now the form:
* 0xhhhh0000
*/
brw_SHL(p, dst, dst, brw_imm_ud(16u));
/* And, finally the form of packHalf2x16's output:
* 0xhhhhllll
*/
brw_F32TO16(p, dst_w, x);
}
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 22:28:11 +00:00
void
intel/compiler: Silence unused parameter warnings in generate_foo methods Since all of the fs_generator::generate_foo methods take a fs_inst * as the first parameter, just remove the name to quiet the compiler. src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_barrier(fs_inst*, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:743:41: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_barrier(fs_inst *inst, struct brw_reg src) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_discard_jump(fs_inst*)’: src/intel/compiler/brw_fs_generator.cpp:1326:46: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_discard_jump(fs_inst *inst) ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_pack_half_2x16_split(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1675:54: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_pack_half_2x16_split(fs_inst *inst, ^~~~ src/intel/compiler/brw_fs_generator.cpp: In member function ‘void fs_generator::generate_shader_time_add(fs_inst*, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_fs_generator.cpp:1743:49: warning: unused parameter ‘inst’ [-Wunused-parameter] fs_generator::generate_shader_time_add(fs_inst *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_set_simd4x2_header_gen9(brw_codegen*, brw::vec4_instruction*, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1412:52: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp: In function ‘void generate_mov_indirect(brw_codegen*, brw::vec4_instruction*, brw_reg, brw_reg, brw_reg, brw_reg)’: src/intel/compiler/brw_vec4_generator.cpp:1430:41: warning: unused parameter ‘inst’ [-Wunused-parameter] vec4_instruction *inst, ^~~~ src/intel/compiler/brw_vec4_generator.cpp:1432:63: warning: unused parameter ‘length’ [-Wunused-parameter] struct brw_reg indirect, struct brw_reg length) ^~~~~~ Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2018-03-29 00:29:45 +01:00
fs_generator::generate_shader_time_add(fs_inst *,
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 22:28:11 +00:00
struct brw_reg payload,
struct brw_reg offset,
struct brw_reg value)
{
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 22:28:11 +00:00
brw_push_insn_state(p);
i965: Put '_default_' in the name of functions that set default state. Eventually we're going to use functions to set bits on an instruction. Putting 'default' in the name of functions that alter default state will help distinguins them. This patch was generated entirely mechanically, by the following: for file in brw*.{cpp,c,h}; do sed -i \ -e 's/brw_set_mask_control/brw_set_default_mask_control/g' \ -e 's/brw_set_saturate/brw_set_default_saturate/g' \ -e 's/brw_set_access_mode/brw_set_default_access_mode/g' \ -e 's/brw_set_compression_control/brw_set_default_compression_control/g' \ -e 's/brw_set_predicate_control/brw_set_default_predicate_control/g' \ -e 's/brw_set_predicate_inverse/brw_set_default_predicate_inverse/g' \ -e 's/brw_set_flag_reg/brw_set_default_flag_reg/g' \ -e 's/brw_set_acc_write_control/brw_set_default_acc_write_control/g' \ $file; done No manual changes were done after running that command. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-06-01 00:57:02 +01:00
brw_set_default_mask_control(p, true);
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 22:28:11 +00:00
assert(payload.file == BRW_GENERAL_REGISTER_FILE);
struct brw_reg payload_offset = retype(brw_vec1_grf(payload.nr, 0),
offset.type);
struct brw_reg payload_value = retype(brw_vec1_grf(payload.nr + 1, 0),
value.type);
assert(offset.file == BRW_IMMEDIATE_VALUE);
if (value.file == BRW_GENERAL_REGISTER_FILE) {
value.width = BRW_WIDTH_1;
value.hstride = BRW_HORIZONTAL_STRIDE_0;
value.vstride = BRW_VERTICAL_STRIDE_0;
} else {
assert(value.file == BRW_IMMEDIATE_VALUE);
}
/* Trying to deal with setup of the params from the IR is crazy in the FS8
* case, and we don't really care about squeezing every bit of performance
* out of this path, so we just emit the MOVs from here.
*/
brw_MOV(p, payload_offset, offset);
brw_MOV(p, payload_value, value);
i965: Make a brw_stage_prog_data for storing the SURF_INDEX information. It would be nice to be able to pack our binding table so that programs that use 1 render target don't upload an extra BRW_MAX_DRAW_BUFFERS - 1 binding table entries. To do that, we need the compiled program to have information on where its surfaces go. v2: Rename size to size_bytes to be more explicit. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-02 22:07:40 +01:00
brw_shader_time_add(p, payload,
i965/fs: Remove direct fs_generator brw_wm_prog_data dependence Instead we store a brw_stage_prog_data pointer, and cast it to brw_wm_prog_data for fragment shader specific code paths. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-29 20:50:46 +01:00
prog_data->binding_table.shader_time_start);
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 22:28:11 +00:00
brw_pop_insn_state(p);
}
i965: Set shader name for generator from call site fs_generator no longer knows what stage it's generating code for, so we have to set the debug name of the shader from the call site. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-28 02:40:47 +00:00
void
fs_generator::enable_debug(const char *shader_name)
{
debug_flag = true;
this->shader_name = shader_name;
}
i965: Refactor fs_generator API We split out SIMD8 and SIMD16 generation into seperate calls to new method generate_code(), which returns the start offset for the generated code. A new get_assembly() method returns the generated code. This avoids asserting MESA_SHADER_FRAGMENT and accessing wm_prog_data in the generator. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-14 00:28:08 +00:00
int
fs_generator::generate_code(const cfg_t *cfg, int dispatch_width)
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
{
i965: Refactor fs_generator API We split out SIMD8 and SIMD16 generation into seperate calls to new method generate_code(), which returns the start offset for the generated code. A new get_assembly() method returns the generated code. This avoids asserting MESA_SHADER_FRAGMENT and accessing wm_prog_data in the generator. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-14 00:28:08 +00:00
/* align to 64 byte boundary. */
while (p->next_insn_offset % 64)
brw_NOP(p);
this->dispatch_width = dispatch_width;
i965: Move annotation info into generate code. Suggested by Ken as a way to cut down lines of code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:42:32 +01:00
int start_offset = p->next_insn_offset;
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
int spill_count = 0, fill_count = 0;
i965/vec4/fs: Count loops in shader debug Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
2014-08-06 09:27:58 +01:00
int loop_count = 0;
i965: Move annotation info into generate code. Suggested by Ken as a way to cut down lines of code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:42:32 +01:00
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
struct disasm_info *disasm_info = disasm_initialize(devinfo, cfg);
i965: Move annotation info into generate code. Suggested by Ken as a way to cut down lines of code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:42:32 +01:00
i965: Pass a cfg pointer to generate_{code,assembly}. The loop over all instructions is now two-fold, over all of the blocks and all of the instructions in each block. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-07-12 05:16:13 +01:00
foreach_block_and_inst (block, fs_inst, inst, cfg) {
intel/fs: Add a generic SEND opcode Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 20:06:14 +00:00
struct brw_reg src[4], dst;
i965: Don't use brw_set_conditionalmod in the FS and vec4 compilers. brw_set_conditionalmod and brw_next_insn work together to set the conditional modifier for the next instruction, then turn it off. The Gen8+ generators don't implement this: we just set it for all future instructions, and whack it for each fs_inst/vec4_instruction. Both approaches work out because we only set conditional_mod on IR instructions like CMP, AND, and so on, which correspond to exactly one assembly instruction. The Gen8 generators would break if we had an IR instruction that generated multiple instructions, and the Gen4-7 EU emit layer would do...something. To safeguard against this, assert that we only generated one instruction if conditional_mod is set, and just set the flag directly on that instruction rather than altering default state. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 00:41:32 +01:00
unsigned int last_insn_offset = p->next_insn_offset;
i965/fs: Apply conditional mod specially to split MAD/LRP. Otherwise we'll apply the conditional mod to only one of SIMD8 instructions and trigger an assertion. NoDDClr/NoDDChk have the same problem but we never apply those to these instructions, so I'm leaving them for a later time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-12-30 20:56:13 +00:00
bool multiple_instructions_emitted = false;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965/fs: Don't follow pow with an instruction with two dest regs. Beginning with commit 7b208a73, Unigine Valley began hanging the GPU on Gen >= 8 platforms. Evidently that commit allowed the scheduler to make different choices that somehow finally ran afoul of a hardware bug in which POW and FDIV instructions may not be followed by an instruction with two destination registers (including compressed instructions). I presume the conditions are more complex than that, but the internal hardware bug report (BDWGFX bug_de 1696294) does not contain much more information. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=94924 Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> [v1] Tested-by: Mark Janes <mark.a.janes@intel.com> [v1] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-03 07:32:13 +01:00
/* From the Broadwell PRM, Volume 7, "3D-Media-GPGPU", in the
* "Register Region Restrictions" section: for BDW, SKL:
*
* "A POW/FDIV operation must not be followed by an instruction
* that requires two destination registers."
*
* The documentation is often lacking annotations for Atom parts,
* and empirically this affects CHV as well.
*/
if (devinfo->gen >= 8 &&
i965/fs: Don't apply POW/FDIV workaround on Gen10+ The documentation says it applies only to Gens 8 and 9. Reviewed-by: Scott D Phillips <scott.d.phillips@intel.com>
2017-08-25 23:52:27 +01:00
devinfo->gen <= 9 &&
i965/fs: Don't follow pow with an instruction with two dest regs. Beginning with commit 7b208a73, Unigine Valley began hanging the GPU on Gen >= 8 platforms. Evidently that commit allowed the scheduler to make different choices that somehow finally ran afoul of a hardware bug in which POW and FDIV instructions may not be followed by an instruction with two destination registers (including compressed instructions). I presume the conditions are more complex than that, but the internal hardware bug report (BDWGFX bug_de 1696294) does not contain much more information. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=94924 Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> [v1] Tested-by: Mark Janes <mark.a.janes@intel.com> [v1] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-05-03 07:32:13 +01:00
p->nr_insn > 1 &&
brw_inst_opcode(devinfo, brw_last_inst) == BRW_OPCODE_MATH &&
brw_inst_math_function(devinfo, brw_last_inst) == BRW_MATH_FUNCTION_POW &&
inst->dst.component_size(inst->exec_size) > REG_SIZE) {
brw_NOP(p);
last_insn_offset = p->next_insn_offset;
}
i965: Print disassembly after compaction. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-19 18:20:37 +01:00
if (unlikely(debug_flag))
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
disasm_annotate(disasm_info, inst, p->next_insn_offset);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965/fs: Simplify per-instruction compression control setup in generator. By using the new compression/group control interface. This will allow easier extension to support arbitrary channel enable groups at the IR level. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 02:48:04 +01:00
/* If the instruction writes to more than one register, it needs to be
* explicitly marked as compressed on Gen <= 5. On Gen >= 6 the
* hardware figures out by itself what the right compression mode is,
* but we still need to know whether the instruction is compressed to
* set up the source register regions appropriately.
*
* XXX - This is wrong for instructions that write a single register but
* read more than one which should strictly speaking be treated as
* compressed. For instructions that don't write any registers it
* relies on the destination being a null register of the correct
* type and regioning so the instruction is considered compressed
* or not accordingly.
*/
i965/eu: Remove brw_codegen::compressed and ::compressed_stack. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 23:25:28 +01:00
const bool compressed =
inst->dst.component_size(inst->exec_size) > REG_SIZE;
brw_set_default_compression(p, compressed);
i965/fs: Expose arbitrary channel execution groups to the IR. This generalizes the current fs_inst::force_sechalf flag to allow specifying channel enable groups other than 0 or 8. At some point it will likely make sense to fix the vec4 generator to support arbitrary execution groups and then move the definition of fs_inst::group into backend_instruction (e.g. so we can do FP64 in the VEC4 back-end). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-21 00:14:13 +01:00
brw_set_default_group(p, inst->group);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
for (unsigned int i = 0; i < inst->sources; i++) {
i965/fs: double regioning parameters and execsize for DF in IVB/BYT In IVB and BYT, both regioning parameters and execution sizes are measured as 32-bits element size. So when we have something like: mov(8) g2<1>DF g3<4,4,1>DF We are not actually moving 8 doubles (our intention), but 4 doubles. We need to double the parameters to cope with this issue. However, horizontal strides don't behave as they're supposed to on IVB for DF regions, they will cause each 32-bit half of DF sources to be strided individually, and doubling the value won't make any difference. v2: - Use devinfo directly (Matt). - Use Baytrail instead of Valleview (Matt). - Use IvyBridge instead of Ivy (Matt) - Double the exec_size in code emission (Curro) v3: - Change hstride doubling by an assert and fix commit log (Curro). - Substitute remaining compiler->devinfo by devinfo (Curro). v4: - Fix comment (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-07-18 08:27:56 +01:00
src[i] = brw_reg_from_fs_reg(devinfo, inst,
&inst->src[i], compressed);
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
/* The accumulator result appears to get used for the
* conditional modifier generation. When negating a UD
* value, there is a 33rd bit generated for the sign in the
* accumulator value, so now you can't check, for example,
* equality with a 32-bit value. See piglit fs-op-neg-uvec4.
*/
assert(!inst->conditional_mod ||
inst->src[i].type != BRW_REGISTER_TYPE_UD ||
!inst->src[i].negate);
}
i965/fs: double regioning parameters and execsize for DF in IVB/BYT In IVB and BYT, both regioning parameters and execution sizes are measured as 32-bits element size. So when we have something like: mov(8) g2<1>DF g3<4,4,1>DF We are not actually moving 8 doubles (our intention), but 4 doubles. We need to double the parameters to cope with this issue. However, horizontal strides don't behave as they're supposed to on IVB for DF regions, they will cause each 32-bit half of DF sources to be strided individually, and doubling the value won't make any difference. v2: - Use devinfo directly (Matt). - Use Baytrail instead of Valleview (Matt). - Use IvyBridge instead of Ivy (Matt) - Double the exec_size in code emission (Curro) v3: - Change hstride doubling by an assert and fix commit log (Curro). - Substitute remaining compiler->devinfo by devinfo (Curro). v4: - Fix comment (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-07-18 08:27:56 +01:00
dst = brw_reg_from_fs_reg(devinfo, inst,
&inst->dst, compressed);
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
i965/fs: Set default access mode to Align1 for all instructions in the generator. Currently the generator code for most opcodes honours the default access mode (which should typically be Align1 in the scalar back-end), but generate_code() doesn't set it explicitly which means that the access mode from a previous instruction could leak into the following ones if you did something special and weren't careful enough to save and restore the previous access mode. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 02:41:28 +01:00
brw_set_default_access_mode(p, BRW_ALIGN_1);
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
brw_set_default_predicate_control(p, inst->predicate);
brw_set_default_predicate_inverse(p, inst->predicate_inverse);
intel/fs: Add the group to the flag subreg number on SNB and older We want consistent behavior in the meaning of the flag_subreg field between SNB and IVB+. v2 (Jason Ekstrand): - Add some extra commentary Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Matt Turner <mattst88@gmail.com>
2018-05-18 04:51:24 +01:00
/* On gen7 and above, hardware automatically adds the group onto the
* flag subregister number. On Sandy Bridge and older, we have to do it
* ourselves.
*/
const unsigned flag_subreg = inst->flag_subreg +
(devinfo->gen >= 7 ? 0 : inst->group / 16);
brw_set_default_flag_reg(p, flag_subreg / 2, flag_subreg % 2);
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
brw_set_default_saturate(p, inst->saturate);
brw_set_default_mask_control(p, inst->force_writemask_all);
brw_set_default_acc_write_control(p, inst->writes_accumulator);
i965/fs: double regioning parameters and execsize for DF in IVB/BYT In IVB and BYT, both regioning parameters and execution sizes are measured as 32-bits element size. So when we have something like: mov(8) g2<1>DF g3<4,4,1>DF We are not actually moving 8 doubles (our intention), but 4 doubles. We need to double the parameters to cope with this issue. However, horizontal strides don't behave as they're supposed to on IVB for DF regions, they will cause each 32-bit half of DF sources to be strided individually, and doubling the value won't make any difference. v2: - Use devinfo directly (Matt). - Use Baytrail instead of Valleview (Matt). - Use IvyBridge instead of Ivy (Matt) - Double the exec_size in code emission (Curro) v3: - Change hstride doubling by an assert and fix commit log (Curro). - Substitute remaining compiler->devinfo by devinfo (Curro). v4: - Fix comment (Curro). Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-07-18 08:27:56 +01:00
unsigned exec_size = inst->exec_size;
if (devinfo->gen == 7 && !devinfo->is_haswell &&
(get_exec_type_size(inst) == 8 || type_sz(inst->dst.type) == 8)) {
exec_size *= 2;
}
brw_set_default_exec_size(p, cvt(exec_size) - 1);
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
i965/fs: do not require force_writemask_all with exec_size 4 So far we only used instructions with this size in situations where we did not operate per-channel and we wanted to ignore the execution mask, but gen7 fp64 will need to emit code with a width of 4 that needs normal execution masking. v2: - Modify the assert instead of deleting it (Curro) Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-06-20 11:13:14 +01:00
assert(inst->force_writemask_all || inst->exec_size >= 4);
i965/fs: Expose arbitrary channel execution groups to the IR. This generalizes the current fs_inst::force_sechalf flag to allow specifying channel enable groups other than 0 or 8. At some point it will likely make sense to fix the vec4 generator to support arbitrary execution groups and then move the definition of fs_inst::group into backend_instruction (e.g. so we can do FP64 in the VEC4 back-end). Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-21 00:14:13 +01:00
assert(inst->force_writemask_all || inst->group % inst->exec_size == 0);
i965: Move compression decisions before brw_reg_from_fs_reg(). brw_reg_from_fs_reg() needs to know whether the instruction will be compressed or not. Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=95370 Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-05-16 23:09:17 +01:00
assert(inst->base_mrf + inst->mlen <= BRW_MAX_MRF(devinfo->gen));
assert(inst->mlen <= BRW_MAX_MSG_LENGTH);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
switch (inst->opcode) {
case BRW_OPCODE_MOV:
brw_MOV(p, dst, src[0]);
break;
case BRW_OPCODE_ADD:
brw_ADD(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_MUL:
brw_MUL(p, dst, src[0], src[1]);
break;
i965/fs: add support for BRW_OPCODE_AVG in fs_generator Needed for compiling blorp blit programs. Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-17 14:39:16 +00:00
case BRW_OPCODE_AVG:
brw_AVG(p, dst, src[0], src[1]);
break;
i965/fs: Fix 32-bit integer multiplication. The MUL opcode does a 16bit * 32bit multiply, and we need to do the MACH to get the top 16bit * 32bit added in. Fixes fs-op-mult-int-*, fs-op-mult-ivec* Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-08-16 06:36:18 +01:00
case BRW_OPCODE_MACH:
brw_MACH(p, dst, src[0], src[1]);
break;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
i965/fs: Add support for generating the LINE instruction. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-04-02 01:25:12 +01:00
case BRW_OPCODE_LINE:
brw_LINE(p, dst, src[0], src[1]);
break;
i965/fs: Add support for generating MADs. Improves nexuiz performance 0.65% +/- .10% (n=5) on my gen6, and .39% +/- .11% (n=10) on gen7. No statistically significant performance difference on warsow (n=5, but only one shader has MADs). v2: Add support for MADs in 16-wide by using compression control. v3: Don't generate MADs when it will force an immediate to be moved to a temp. (it's not clear whether this is a win or not, but it should result in less questionable change to codegen compared to v2). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v2)
2012-02-06 23:59:11 +00:00
case BRW_OPCODE_MAD:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 6);
Revert "Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+"" This reverts commit 2d0457203871c843ebfc90fb895b65a9b14cd9bb. Acked-by: Scott D Phillips <scott.d.phillips@intel.com>
2018-01-05 17:46:11 +00:00
if (devinfo->gen < 10)
brw_set_default_access_mode(p, BRW_ALIGN_16);
i965/fs: Drop lowering code for a few three-source instructions from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:51:50 +01:00
brw_MAD(p, dst, src[0], src[1], src[2]);
i965/fs: Add support for generating MADs. Improves nexuiz performance 0.65% +/- .10% (n=5) on my gen6, and .39% +/- .11% (n=10) on gen7. No statistically significant performance difference on warsow (n=5, but only one shader has MADs). v2: Add support for MADs in 16-wide by using compression control. v3: Don't generate MADs when it will force an immediate to be moved to a temp. (it's not clear whether this is a win or not, but it should result in less questionable change to codegen compared to v2). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v2)
2012-02-06 23:59:11 +00:00
break;
i965/fs: Use the LRP instruction for ir_triop_lrp when possible. v2 [mattst88]: - Add BRW_OPCODE_LRP to list of CSE-able expressions. - Fix op_var[] array size. - Rename arguments to emit_lrp to (x, y, a) to clear confusion. - Add LRP function to brw_fs.cpp/.h. - Corrected comment about LRP instruction arguments in emit_lrp. v3 [mattst88]: - Duplicate MAD code for LRP instead of using a function pointer. - Check for != GRF instead of == IMM in emit_lrp. - Lower LRP on gen < 6. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Eric Anholt <eric@anholt.net> Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> 1
2012-12-02 08:08:15 +00:00
case BRW_OPCODE_LRP:
intel/compiler: Lower flrp32 on Gen11+ The LRP instruction is no more. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 00:20:41 +01:00
assert(devinfo->gen >= 6 && devinfo->gen <= 10);
Revert "Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+"" This reverts commit 2d0457203871c843ebfc90fb895b65a9b14cd9bb. Acked-by: Scott D Phillips <scott.d.phillips@intel.com>
2018-01-05 17:46:11 +00:00
if (devinfo->gen < 10)
brw_set_default_access_mode(p, BRW_ALIGN_16);
i965/fs: Drop lowering code for a few three-source instructions from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:51:50 +01:00
brw_LRP(p, dst, src[0], src[1], src[2]);
i965/fs: Use the LRP instruction for ir_triop_lrp when possible. v2 [mattst88]: - Add BRW_OPCODE_LRP to list of CSE-able expressions. - Fix op_var[] array size. - Rename arguments to emit_lrp to (x, y, a) to clear confusion. - Add LRP function to brw_fs.cpp/.h. - Corrected comment about LRP instruction arguments in emit_lrp. v3 [mattst88]: - Duplicate MAD code for LRP instead of using a function pointer. - Check for != GRF instead of == IMM in emit_lrp. - Lower LRP on gen < 6. Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Eric Anholt <eric@anholt.net> Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> 1
2012-12-02 08:08:15 +00:00
break;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
case BRW_OPCODE_FRC:
brw_FRC(p, dst, src[0]);
break;
case BRW_OPCODE_RNDD:
brw_RNDD(p, dst, src[0]);
break;
case BRW_OPCODE_RNDE:
brw_RNDE(p, dst, src[0]);
break;
case BRW_OPCODE_RNDZ:
brw_RNDZ(p, dst, src[0]);
break;
case BRW_OPCODE_AND:
brw_AND(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_OR:
brw_OR(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_XOR:
brw_XOR(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_NOT:
brw_NOT(p, dst, src[0]);
break;
case BRW_OPCODE_ASR:
brw_ASR(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_SHR:
brw_SHR(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_SHL:
brw_SHL(p, dst, src[0], src[1]);
break;
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
case BRW_OPCODE_F32TO16:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
brw_F32TO16(p, dst, src[0]);
break;
case BRW_OPCODE_F16TO32:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
brw_F16TO32(p, dst, src[0]);
break;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
case BRW_OPCODE_CMP:
i965/fs: Drop Gen7 CMP SIMD unrolling workaround from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:59:18 +01:00
if (inst->exec_size >= 16 && devinfo->gen == 7 && !devinfo->is_haswell &&
dst.file == BRW_ARCHITECTURE_REGISTER_FILE) {
/* For unknown reasons the WaCMPInstFlagDepClearedEarly workaround
* implemented in the compiler is not sufficient. Overriding the
* type when the destination is the null register is necessary but
* not sufficient by itself.
*/
assert(dst.nr == BRW_ARF_NULL);
dst.type = BRW_REGISTER_TYPE_D;
i965/fs: Implement the WaCMPInstFlagDepClearedEarly work-around. Prevents piglit regressions from the next patch. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-02-04 01:38:49 +00:00
}
i965/fs: Drop Gen7 CMP SIMD unrolling workaround from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:59:18 +01:00
brw_CMP(p, dst, inst->conditional_mod, src[0], src[1]);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
case BRW_OPCODE_SEL:
brw_SEL(p, dst, src[0], src[1]);
break;
i965/fs: Add infrastructure for generating CSEL instructions. v2 (idr): Don't allow CSEL with a non-float src2. v3 (idr): Add CSEL to fs_inst::flags_written. Suggested by Matt. v4 (idr): Only set BRW_ALIGN_16 on Gen < 10 (suggested by Matt). Don't reset the access mode afterwards (suggested by Samuel and Matt). Add support for CSEL not modifying the flags to more places (requested by Matt). Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> [v3] Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-11-23 04:12:17 +00:00
case BRW_OPCODE_CSEL:
assert(devinfo->gen >= 8);
if (devinfo->gen < 10)
brw_set_default_access_mode(p, BRW_ALIGN_16);
brw_CSEL(p, dst, src[0], src[1], src[2]);
break;
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
case BRW_OPCODE_BFREV:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
brw_BFREV(p, retype(dst, BRW_REGISTER_TYPE_UD),
i965: Normalize types for FBL, FBH, etc Allows the instructions to be compacted. The documentation claims that some of these only accept UD types, even though the type doesn't change the operation performed. Just normalize the types to ensure we get instruction compaction. The only functional changes are for FBL and CBIT (always use UD types) and FBH (always use the same types). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-30 23:10:17 +01:00
retype(src[0], BRW_REGISTER_TYPE_UD));
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
case BRW_OPCODE_FBH:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965: Normalize types for FBL, FBH, etc Allows the instructions to be compacted. The documentation claims that some of these only accept UD types, even though the type doesn't change the operation performed. Just normalize the types to ensure we get instruction compaction. The only functional changes are for FBL and CBIT (always use UD types) and FBH (always use the same types). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-30 23:10:17 +01:00
brw_FBH(p, retype(dst, src[0].type), src[0]);
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
case BRW_OPCODE_FBL:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965: Normalize types for FBL, FBH, etc Allows the instructions to be compacted. The documentation claims that some of these only accept UD types, even though the type doesn't change the operation performed. Just normalize the types to ensure we get instruction compaction. The only functional changes are for FBL and CBIT (always use UD types) and FBH (always use the same types). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-30 23:10:17 +01:00
brw_FBL(p, retype(dst, BRW_REGISTER_TYPE_UD),
retype(src[0], BRW_REGISTER_TYPE_UD));
i965: Use LZD to implement nir_op_ufind_msb This uses one less instruction. v2: Move emit_find_msb_using_lzd out of the visitor classes. Suggested by Curro. Signed-off-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-06-21 23:14:03 +01:00
break;
case BRW_OPCODE_LZD:
brw_LZD(p, dst, src[0]);
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
case BRW_OPCODE_CBIT:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965: Normalize types for FBL, FBH, etc Allows the instructions to be compacted. The documentation claims that some of these only accept UD types, even though the type doesn't change the operation performed. Just normalize the types to ensure we get instruction compaction. The only functional changes are for FBL and CBIT (always use UD types) and FBH (always use the same types). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-30 23:10:17 +01:00
brw_CBIT(p, retype(dst, BRW_REGISTER_TYPE_UD),
retype(src[0], BRW_REGISTER_TYPE_UD));
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
i965: Generate code for ir_binop_carry and ir_binop_borrow. Using the ADDC and SUBB instructions on Gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-09-19 21:01:08 +01:00
case BRW_OPCODE_ADDC:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965: Generate code for ir_binop_carry and ir_binop_borrow. Using the ADDC and SUBB instructions on Gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-09-19 21:01:08 +01:00
brw_ADDC(p, dst, src[0], src[1]);
break;
case BRW_OPCODE_SUBB:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965: Generate code for ir_binop_carry and ir_binop_borrow. Using the ADDC and SUBB instructions on Gen7. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-09-19 21:01:08 +01:00
brw_SUBB(p, dst, src[0], src[1]);
break;
i965/fs: Add support for the MAC instruction. This allows us to generate the MAC (multiply-accumulate) instruction, which can be used to implement some expressions in fewer instructions than doing a series of MUL and ADDs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Juha-Pekka Heikkila <juhapekka.heikkila@gmail.com>
2014-03-28 13:28:32 +00:00
case BRW_OPCODE_MAC:
brw_MAC(p, dst, src[0], src[1]);
break;
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
case BRW_OPCODE_BFE:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
Revert "Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+"" This reverts commit 2d0457203871c843ebfc90fb895b65a9b14cd9bb. Acked-by: Scott D Phillips <scott.d.phillips@intel.com>
2018-01-05 17:46:11 +00:00
if (devinfo->gen < 10)
brw_set_default_access_mode(p, BRW_ALIGN_16);
i965/fs: Drop lowering code for a few three-source instructions from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:51:50 +01:00
brw_BFE(p, dst, src[0], src[1], src[2]);
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
case BRW_OPCODE_BFI1:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
i965/fs: Remove manual unrolling of BFI instructions from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 04:02:29 +01:00
brw_BFI1(p, dst, src[0], src[1]);
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
case BRW_OPCODE_BFI2:
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen >= 7);
Revert "Revert "i965/fs: Use align1 mode on ternary instructions on Gen10+"" This reverts commit 2d0457203871c843ebfc90fb895b65a9b14cd9bb. Acked-by: Scott D Phillips <scott.d.phillips@intel.com>
2018-01-05 17:46:11 +00:00
if (devinfo->gen < 10)
brw_set_default_access_mode(p, BRW_ALIGN_16);
i965/fs: Remove manual unrolling of BFI instructions from the generator. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 04:02:29 +01:00
brw_BFI2(p, dst, src[0], src[1], src[2]);
i965/fs: Add support for bit instructions. Don't bother scalarizing ir_binop_bfm, since its results are identical for all channels. v2: Subtract result of FBH from 31 (unless an error) to convert MSB counts to LSB counts. v3: Use op0->clone() in ir_triop_bfi to prevent (var_ref channel_expressions) from appearing multiple times in the IR. Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> [v2]
2013-04-10 03:22:34 +01:00
break;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
case BRW_OPCODE_IF:
if (inst->src[0].file != BAD_FILE) {
/* The instruction has an embedded compare (only allowed on gen6) */
i965/fs: Add a devinfo field to the generator and use it for gen checks Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 01:45:40 +01:00
assert(devinfo->gen == 6);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
gen6_IF(p, inst->conditional_mod, src[0], src[1]);
} else {
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 22:50:46 +01:00
brw_IF(p, brw_get_default_exec_size(p));
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
break;
case BRW_OPCODE_ELSE:
brw_ELSE(p);
break;
case BRW_OPCODE_ENDIF:
brw_ENDIF(p);
break;
case BRW_OPCODE_DO:
intel/eu: Add some brw_get_default_ helpers This is much cleaner than everything that wants a default value poking at the bits of p->current directly. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-05-29 22:50:46 +01:00
brw_DO(p, brw_get_default_exec_size(p));
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
case BRW_OPCODE_BREAK:
i965: Don't make consumers of brw_CONT/brw_WHILE track if depth in loop. The codegen backends all had this same tracking, so just do it at the EU level. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 20:44:41 +00:00
brw_BREAK(p);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
case BRW_OPCODE_CONTINUE:
i965/eu: Merge brw_CONT and gen6_CONT. The only difference is setting PopCount on Gen4-5. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-08-04 22:26:26 +01:00
brw_CONT(p);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965: Don't make consumers of brw_WHILE do pre-gen6 BREAK/CONT patching. The EU code itself can just do this work, since all the consumers were duplicating it. Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
2011-12-06 20:30:03 +00:00
case BRW_OPCODE_WHILE:
brw_WHILE(p);
i965/vec4/fs: Count loops in shader debug Reviewed-by: Matt Turner <mattst88@gmail.com> Signed-off-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
2014-08-06 09:27:58 +01:00
loop_count++;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965: Rename math FS_OPCODE_* to SHADER_OPCODE_*. I want to just use the same enums in the VS.
2011-08-05 20:38:58 +01:00
case SHADER_OPCODE_RCP:
case SHADER_OPCODE_RSQ:
case SHADER_OPCODE_SQRT:
case SHADER_OPCODE_EXP2:
case SHADER_OPCODE_LOG2:
case SHADER_OPCODE_SIN:
case SHADER_OPCODE_COS:
i965: Assert that math instructions don't have conditional mod. The math function field is at the same location as conditional mod. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-21 20:34:22 +00:00
assert(inst->conditional_mod == BRW_CONDITIONAL_NONE);
i965/fs: Remove handcrafted math SIMD lowering from the generator. Most of this wouldn't have worked for SIMD32 and had various dispatch_width and compression control bugs. It's mostly dead now with SIMD lowering of math instructions turned on in the compiler. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:10:48 +01:00
if (devinfo->gen >= 6) {
assert(inst->mlen == 0);
assert(devinfo->gen >= 7 || inst->exec_size == 8);
gen6_math(p, dst, brw_math_function(inst->opcode),
src[0], brw_null_reg());
i965/fs: Split generate_math into gen4/gen6 and 1/2 operand variants. This mirrors the structure Eric used in the new VS backend, and seems simpler. In particular, the math1/math2 split will avoid having to figure out how many operands there are, as this is already known by the caller. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-18 19:55:42 +01:00
} else {
i965/fs: Remove handcrafted math SIMD lowering from the generator. Most of this wouldn't have worked for SIMD32 and had various dispatch_width and compression control bugs. It's mostly dead now with SIMD lowering of math instructions turned on in the compiler. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:10:48 +01:00
assert(inst->mlen >= 1);
assert(devinfo->gen == 5 || devinfo->is_g4x || inst->exec_size == 8);
gen4_math(p, dst,
brw_math_function(inst->opcode),
inst->base_mrf, src[0],
BRW_MATH_PRECISION_FULL);
i965/fs: Split generate_math into gen4/gen6 and 1/2 operand variants. This mirrors the structure Eric used in the new VS backend, and seems simpler. In particular, the math1/math2 split will avoid having to figure out how many operands there are, as this is already known by the caller. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-18 19:55:42 +01:00
}
break;
i965/fs: Implement integer quotient and remainder math operations. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Tested-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-09-29 01:37:54 +01:00
case SHADER_OPCODE_INT_QUOTIENT:
case SHADER_OPCODE_INT_REMAINDER:
i965/fs: Split generate_math into gen4/gen6 and 1/2 operand variants. This mirrors the structure Eric used in the new VS backend, and seems simpler. In particular, the math1/math2 split will avoid having to figure out how many operands there are, as this is already known by the caller. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-18 19:55:42 +01:00
case SHADER_OPCODE_POW:
i965: Assert that math instructions don't have conditional mod. The math function field is at the same location as conditional mod. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-21 20:34:22 +00:00
assert(inst->conditional_mod == BRW_CONDITIONAL_NONE);
i965/fs: Remove handcrafted math SIMD lowering from the generator. Most of this wouldn't have worked for SIMD32 and had various dispatch_width and compression control bugs. It's mostly dead now with SIMD lowering of math instructions turned on in the compiler. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:10:48 +01:00
if (devinfo->gen >= 6) {
assert(inst->mlen == 0);
assert((devinfo->gen >= 7 && inst->opcode == SHADER_OPCODE_POW) ||
inst->exec_size == 8);
i965/fs: Drop the generate_math[12]_gen7 methods. These functions are basically identical, so we should combine them. However, they're so trivial, we may as well just fold them into their only call sites. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
2014-06-07 10:27:43 +01:00
gen6_math(p, dst, brw_math_function(inst->opcode), src[0], src[1]);
i965/fs: Remove handcrafted math SIMD lowering from the generator. Most of this wouldn't have worked for SIMD32 and had various dispatch_width and compression control bugs. It's mostly dead now with SIMD lowering of math instructions turned on in the compiler. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-18 03:10:48 +01:00
} else {
assert(inst->mlen >= 1);
assert(inst->exec_size == 8);
gen4_math(p, dst, brw_math_function(inst->opcode),
inst->base_mrf, src[0],
BRW_MATH_PRECISION_FULL);
i965/fs: Split generate_math into gen4/gen6 and 1/2 operand variants. This mirrors the structure Eric used in the new VS backend, and seems simpler. In particular, the math1/math2 split will avoid having to figure out how many operands there are, as this is already known by the caller. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-08-18 19:55:42 +01:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
case FS_OPCODE_LINTERP:
intel/compiler/fs: Return multiple_instructions_emitted from generate_linterp If multiple instructions are emitted, special handling of things like conditional mod and NoDDClr/NoDDChk need to be performed. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-14 19:06:45 +01:00
multiple_instructions_emitted = generate_linterp(inst, dst, src);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965/fs: Combine pixel center calculation into one inst. The X and Y values come interleaved in g1 (.4-.11 inclusive), so we can calculate them together with a single add(32) instruction on some platforms like Broadwell and newer or in SIMD8 elsewhere. Note that I also moved the PIXEL_X/PIXEL_Y virtual opcodes from before LINTERP to after it. That's because the writes_accumulator_implicitly() function in backend_instruction tests for <= LINTERP for determining whether the instruction indeed writes the accumulator implicitly. The old FS_OPCODE_PIXEL_X/Y emitted ADD instructions, which did, but the new opcodes just emit MOVs, which don't. It doesn't matter, since we don't use these opcodes on Gen4/5 anymore, but in the case that we do... On Broadwell: total instructions in shared programs: 7192355 -> 7186224 (-0.09%) instructions in affected programs: 1190700 -> 1184569 (-0.51%) helped: 6131 On Haswell: total instructions in shared programs: 6155979 -> 6152800 (-0.05%) instructions in affected programs: 652362 -> 649183 (-0.49%) helped: 3179 Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-04-14 21:17:38 +01:00
case FS_OPCODE_PIXEL_X:
assert(src[0].type == BRW_REGISTER_TYPE_UW);
src[0].subnr = 0 * type_sz(src[0].type);
brw_MOV(p, dst, stride(src[0], 8, 4, 1));
break;
case FS_OPCODE_PIXEL_Y:
assert(src[0].type == BRW_REGISTER_TYPE_UW);
src[0].subnr = 4 * type_sz(src[0].type);
brw_MOV(p, dst, stride(src[0], 8, 4, 1));
break;
intel/fs: Add a generic SEND opcode Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-10-29 20:06:14 +00:00
case SHADER_OPCODE_SEND:
generate_send(inst, dst, src[0], src[1], src[2],
inst->ex_mlen > 0 ? src[3] : brw_null_reg());
break;
i965: Combine {VS,FS}_OPCODE_GET_BUFFER_SIZE opcodes. These are the same, we don't need a separate opcode enum per backend. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-12-11 01:03:32 +00:00
case SHADER_OPCODE_GET_BUFFER_SIZE:
i965/fs: Implement FS_OPCODE_GET_BUFFER_SIZE Signed-off-by: Samuel Iglesias Gonsalvez <siglesias@igalia.com> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-04-13 15:55:49 +01:00
generate_get_buffer_size(inst, dst, src[0], src[1]);
break;
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TEX:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
case FS_OPCODE_TXB:
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXD:
case SHADER_OPCODE_TXF:
i965: Add infrastucture for sample lod-zero operations. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-05-04 23:46:45 +01:00
case SHADER_OPCODE_TXF_LZ:
i965: rename tex_ms to tex_cms Prepares for the introduction of non-compressed multi-sampled lookup used in the blorp programs. v2: now also taking into account gen8 Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-10 14:36:31 +00:00
case SHADER_OPCODE_TXF_CMS:
i965/fs/skl+: Use ld2dms_w instead of ld2dms In order to support 16x MSAA, skl+ has a wider version of ld2dms that takes two parameters for the MCS data. The MCS data retrieved from the ld_mcs instruction already returns 4 or 8 registers and is documented to return zeroes for the mcsh value when the sample count is less than 16. v2: Use get_lowered_simd_width to fall back to SIMD8 instructions when the message length would be too long in SIMD16. Reviewed-by: Ben Widawsky <ben@bwidawsk.net>
2015-09-08 15:52:09 +01:00
case SHADER_OPCODE_TXF_CMS_W:
i965/fs: introduce non-compressed equivalent of tex_cms v2: introduces 'SHADER_OPCODE_TXF_UMS' also for gen8 Signed-off-by: Topi Pohjolainen <topi.pohjolainen@intel.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-12-10 14:38:15 +00:00
case SHADER_OPCODE_TXF_UMS:
i965: Add shader opcode for sampling MCS surface Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Paul Berry <stereotype441@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-29 21:32:16 +00:00
case SHADER_OPCODE_TXF_MCS:
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXL:
i965: Add infrastucture for sample lod-zero operations. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-05-04 23:46:45 +01:00
case SHADER_OPCODE_TXL_LZ:
i965: Rename texturing ops from FS_OPCODE to SHADER_OPCODE, except TXB. We'll be reusing most of these for the VS shortly. The one exception is TXB (texturing with LOD bias), which is explicitly forbidden in the VS. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2011-10-26 20:58:37 +01:00
case SHADER_OPCODE_TXS:
i965/fs: Generate LOD sampler message from ir_lod. v2: Support Ironlake as well. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-06 22:47:01 +00:00
case SHADER_OPCODE_LOD:
i965: add SHADER_OPCODE_TG4 Adds the Gen7 message IDs, a new SHADER_OPCODE_TG4 pseudo-op, and low-level support for emitting it via generate_tex(). V3: Updated for changes in master. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-31 09:31:12 +01:00
case SHADER_OPCODE_TG4:
i965: Add SHADER_OPCODE_TG4_OFFSET for gather with nonconstant offsets. The generator code ends up clearer this way than if we had to sniff via the message length. Implemented via the gather4_po message in hardware, which is present in Gen7 and later. Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2013-10-08 09:42:10 +01:00
case SHADER_OPCODE_TG4_OFFSET:
i965: add support for textureSamples function Signed-off-by: Ilia Mirkin <imirkin@alum.mit.edu> [v2: kayden-supplied code in fs_nir replacing need for logical opcode] Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-08-12 01:37:32 +01:00
case SHADER_OPCODE_SAMPLEINFO:
i965/fs: Plumb separate surfaces and samplers through from NIR Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-02-06 02:24:02 +00:00
generate_tex(inst, dst, src[0], src[1], src[2]);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
intel: Use TXS for image_size when we have a typed surface Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2018-08-16 17:01:24 +01:00
case SHADER_OPCODE_IMAGE_SIZE:
generate_tex(inst, dst, src[0], src[1], brw_imm_ud(0));
break;
i965/fs: Handle derivative quality decisions in the front-end. Kristian noted that there's very little use of brw_wm_prog_key in the generator, and that it basically just generates what it's told, without caring about what stage it's handling. One exception to this is derivative handling. When handling dFdxCoarse and dFdxFine, we packed an enum value in a second source register, explicitly telling the generator what to do. For dFdx, we specified an enum value of "please use the hint", then checked the program key in the generator level code. A natural method is to define separate FS_OPCODE_DD[XY]_{COARSE,FINE} opcodes, and have the front-end (which already decides what IR to generate based on the program key) decide which dPdx/dPdy should correspond to. This consolidates the decision making in one place. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-08 09:39:14 +00:00
case FS_OPCODE_DDX_COARSE:
case FS_OPCODE_DDX_FINE:
intel/compiler/fs: Pass fs_inst to generate_ddx/ddy instead of opcode In a future patch, generate_ddy will want to inspect inst->exec_size. Change generate_ddx as well for consistency. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 23:41:40 +01:00
generate_ddx(inst, dst, src[0]);
i965/fs: Handle derivative quality decisions in the front-end. Kristian noted that there's very little use of brw_wm_prog_key in the generator, and that it basically just generates what it's told, without caring about what stage it's handling. One exception to this is derivative handling. When handling dFdxCoarse and dFdxFine, we packed an enum value in a second source register, explicitly telling the generator what to do. For dFdx, we specified an enum value of "please use the hint", then checked the program key in the generator level code. A natural method is to define separate FS_OPCODE_DD[XY]_{COARSE,FINE} opcodes, and have the front-end (which already decides what IR to generate based on the program key) decide which dPdx/dPdy should correspond to. This consolidates the decision making in one place. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-11-08 09:39:14 +00:00
break;
case FS_OPCODE_DDY_COARSE:
case FS_OPCODE_DDY_FINE:
intel/compiler/fs: Pass fs_inst to generate_ddx/ddy instead of opcode In a future patch, generate_ddy will want to inspect inst->exec_size. Change generate_ddx as well for consistency. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-06-15 23:41:40 +01:00
generate_ddy(inst, dst, src[0]);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965: Merge together opcodes for SHADER_OPCODE_GEN4_SCRATCH_READ/WRITE I'm going to be introducing gen7 variants, and the previous naming was going to get confusing. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:45:06 +01:00
case SHADER_OPCODE_GEN4_SCRATCH_WRITE:
generate_scratch_write(inst, src[0]);
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
spill_count++;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965: Merge together opcodes for SHADER_OPCODE_GEN4_SCRATCH_READ/WRITE I'm going to be introducing gen7 variants, and the previous naming was going to get confusing. Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:45:06 +01:00
case SHADER_OPCODE_GEN4_SCRATCH_READ:
generate_scratch_read(inst, dst);
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
fill_count++;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:51:22 +01:00
case SHADER_OPCODE_GEN7_SCRATCH_READ:
generate_scratch_read_gen7(inst, dst);
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
fill_count++;
i965/fs: Use the gen7 scratch read opcode when possible. This avoids a lot of message setup we had to do otherwise. Improves GLB2.7 performance with register spilling force enabled by 1.6442% +/- 0.553218% (n=4). v2: Use BRW_PREDICATE_NONE, improve a comment (by Paul). Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-16 19:51:22 +01:00
break;
i965: Introduce a MOV_INDIRECT opcode. The geometry and tessellation control shader stages both read from multiple URB entries (one per vertex). The thread payload contains several URB handles which reference these separate memory segments. In GLSL, these inputs are represented as per-vertex arrays; the outermost array index selects which vertex's inputs to read. This array index does not necessarily need to be constant. To handle that, we need to use indirect addressing on GRFs to select which of the thread payload registers has the appropriate URB handle. (This is before we can even think about applying the pull model!) This patch introduces a new opcode which performs a MOV from a source using VxH indirect addressing (which allows each of the 8 SIMD channels to select distinct data.) Based on a patch by Jason Ekstrand. v2: Rename from INDIRECT_THREAD_PAYLOAD_MOV to MOV_INDIRECT; make it a bit more generic. Use regs_read() instead of hacking up the register allocator. (Suggested by Jason Ekstrand.) v3: Fix regs_read() to be more accurate for small unaligned regions. Also rebase on Matt's work. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> [v3] Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com> [v1]
2015-11-08 02:58:34 +00:00
case SHADER_OPCODE_MOV_INDIRECT:
generate_mov_indirect(inst, dst, src[0], src[1]);
break;
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
case SHADER_OPCODE_URB_READ_SIMD8:
i965: Add a SHADER_OPCODE_URB_READ_SIMD8_PER_SLOT opcode. We need to use per-slot offsets when there's non-uniform indexing, as each SIMD channel could have a different index. We want to use them for any non-constant index (even if uniform), as it lives in the message header instead of the descriptor, allowing us to set offsets in GRFs rather than immediates. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Abdiel Janulgue <abdiel.janulgue@linux.intel.com>
2015-11-07 09:37:33 +00:00
case SHADER_OPCODE_URB_READ_SIMD8_PER_SLOT:
i965: Introduce a new SHADER_OPCODE_URB_READ_SIMD8 opcode. In scalar mode, geometry shader inputs can easily take up hundreds of registers. This makes pushing VUE entries impractical; we'll need to resort to the pull model in some cases. To support this, we introduce a new opcode corresponding to the "URB Read SIMD8" message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-09-29 22:32:02 +01:00
generate_urb_read(inst, dst, src[0]);
break;
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
case SHADER_OPCODE_URB_WRITE_SIMD8:
i965: Introduce new SHADER_OPCODE_URB_WRITE_SIMD8_MASKED/PER_SLOT opcodes. In the vec4 backend, we have a vec4_instruction::urb_write_flags field. There are many kinds of flags for SIMD4x2 messages. However, there are really only two (per-slot offset, use channel masks) for SIMD8 messages. Rather than adding a boolean flag for per-slot offsets (polluting all instructions), I decided to just make three new opcodes. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2015-05-06 08:04:10 +01:00
case SHADER_OPCODE_URB_WRITE_SIMD8_PER_SLOT:
case SHADER_OPCODE_URB_WRITE_SIMD8_MASKED:
case SHADER_OPCODE_URB_WRITE_SIMD8_MASKED_PER_SLOT:
i965: Add SIMD8 URB write low-level IR instruction This is all we need from the generator for SIMD8 vertex shaders. This opcode is just the send instruction, all the hard work will happen in the visitor using LOAD_PAYLOAD. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-21 07:00:50 +01:00
generate_urb_write(inst, src[0]);
break;
i965/fs: Rename the existing pull constant load opcode. We're going to use another send message for handling loads with a varying per-fragment array index.
2012-11-07 18:42:34 +00:00
case FS_OPCODE_UNIFORM_PULL_CONSTANT_LOAD:
i965/fs: Mark UBO uniform pull constant loads as force_writemask_all. This lets the rest of the backend know that the uniform pull constant load opcodes don't respect channel enables -- Without this the register allocator has no way to know that the return payload of a pull constant load is not per-channel and spills of the destination will be broken under non-uniform control flow. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-23 22:07:23 +01:00
assert(inst->force_writemask_all);
i965/fs: Rename the existing pull constant load opcode. We're going to use another send message for handling loads with a varying per-fragment array index.
2012-11-07 18:42:34 +00:00
generate_uniform_pull_constant_load(inst, dst, src[0], src[1]);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965/fs: Set up gen7 UBO loads as sends from GRFs. This gives the instruction scheduler a chance to schedule between the loads, whereas before it was restricted due to the dependencies between the MRFs for setting them up. For one shader in gles3conform, it goes from getting stuck in register allocation for as long as anybody's bothered to leave it running down to 23 seconds, thanks to the LIFO scheduling. Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-05 08:06:30 +00:00
case FS_OPCODE_UNIFORM_PULL_CONSTANT_LOAD_GEN7:
i965/fs: Mark UBO uniform pull constant loads as force_writemask_all. This lets the rest of the backend know that the uniform pull constant load opcodes don't respect channel enables -- Without this the register allocator has no way to know that the return payload of a pull constant load is not per-channel and spills of the destination will be broken under non-uniform control flow. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-23 22:07:23 +01:00
assert(inst->force_writemask_all);
i965/fs: Set up gen7 UBO loads as sends from GRFs. This gives the instruction scheduler a chance to schedule between the loads, whereas before it was restricted due to the dependencies between the MRFs for setting them up. For one shader in gles3conform, it goes from getting stuck in register allocation for as long as anybody's bothered to leave it running down to 23 seconds, thanks to the LIFO scheduling. Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-05 08:06:30 +00:00
generate_uniform_pull_constant_load_gen7(inst, dst, src[0], src[1]);
break;
i965/fs: Rename Gen4 physical varying pull constant load opcode. For consistency with the Gen7 variant. I'm not doing the same to the uniform pull constant message at this point because the non-GEN7 one is still overloaded to be either an expression-like logical instruction or a Gen4-specific physical send message. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-20 21:03:31 +01:00
case FS_OPCODE_VARYING_PULL_CONSTANT_LOAD_GEN4:
generate_varying_pull_constant_load_gen4(inst, dst, src[0]);
i965/fs: Add instruction emit for varying-index reads of uniforms. The gen7 send-from-GRF path is sufficiently different from the perspective of IR generation and optimization that I just made it a separate opcode. v2: fix whitespace, rebase on Ken's recent refactor.
2012-11-07 19:18:34 +00:00
break;
case FS_OPCODE_VARYING_PULL_CONSTANT_LOAD_GEN7:
generate_varying_pull_constant_load_gen7(inst, dst, src[0], src[1]);
break;
i965: Add optimization pass to let us use the replicate data message The data port has a SIMD16 'replicate data' message, which lets us write the same color for all 16 pixels by sending the four floats in the lower half of a register instead of sending 4 times 16 identical component values in 8 registers. The message comes with a lot of restrictions and could be made generally useful by recognizing when those restriction are satisfied. For now, this lets us enable the optimization when we know it's safe, but we don't enable it by default. The optimization works for simple color clear shaders only, but does recognized and support multiple render targets. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Acked-by: Kenneth Graunke <kenneth@whitecape.org>
2014-07-07 23:27:17 +01:00
case FS_OPCODE_REP_FB_WRITE:
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
case FS_OPCODE_FB_WRITE:
i965/fs: Add a an optional source to the FS_OPCODE_FB_WRITE instruction Previously, we were use the base_mrf parameter of fs_inst to store the MRF location. In preparation for doing FB writes from the GRF, we now also allow you to set inst->base_mrf to -1 and provide a source register. Signed-off-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-09-16 23:16:20 +01:00
generate_fb_write(inst, src[0]);
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
break;
i965/fs: Add FS_OPCODE_MOV_DISPATCH_TO_FLAGS to fragment shader backend. In order to compute centroid varyings correctly, the fragment shader needs to be able to load the current pixel/sample mask into a flag register. This patch adds an opcode to the fragment shader back-end to do this; the opcode gets translated into the instruction mov(1) f0<1>UW g1.14<0,1,0>UW { align1 WE_all } Since this instruction clobbers f0, instruction scheduling has to treat it the same as instructions that have a conditional modifier. Reviewed-by: Eric Anholt <eric@anholt.net>
2012-06-18 22:50:04 +01:00
i965/fs: Define framebuffer read virtual opcode. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2016-07-22 00:52:33 +01:00
case FS_OPCODE_FB_READ:
generate_fb_read(inst, dst, src[0]);
break;
i965/fs: Improve performance of shaders that start out with a discard. I had tried this in the past, but ran into trouble with applications that sample from undiscarded pixels in the same subspan. To fix that issue, only jump to the end for an entire subspan at a time. Improves GLbenchmark 2.7 (1024x768) performance by 7.9 +/- 1.5% (n=8). v2: Drop the br variable in the jump instruction -- if I ever do jumps pre-gen6, it'll be a different code block anyway since we don't have HALT until gen6. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2012-12-06 18:15:08 +00:00
case FS_OPCODE_DISCARD_JUMP:
generate_discard_jump(inst);
break;
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 22:10:52 +00:00
case SHADER_OPCODE_SHADER_TIME_ADD:
i965/fs: Switch shader_time writes to using GRFs. This avoids conflicts between shader_time and FB writes, so we can include more of the program under our profiling. This does mean hiding more of the message setup from the optimizer, which doesn't have a way to handle multi-reg sends from GRFs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-03-19 22:28:11 +00:00
generate_shader_time_add(inst, src[0], src[1], src[2]);
i965: Add a debug flag for counting cycles spent in each compiled shader. This can be used for two purposes: Using hand-coded shaders to determine per-instruction timings, or figuring out which shader to optimize in a whole application. Note that this doesn't cover the instructions that set up the message to the URB/FB write -- we'd need to convert the MRF usage in these instructions to GRFs so that our offsets/times don't overwrite our shader outputs. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org> (v1) v2: Check the timestamp reset flag in the VS, which is apparently getting set fairly regularly in the range we watch, resulting in negative numbers getting added to our 32-bit counter, and thus large values added to our uint64_t. v3: Rebase on reladdr changes, removing a new safety check that proved impossible to satisfy. Add a comment to the AOP defs from Ken's review, and put them in a slightly more sensible spot. v4: Check timestamp reset in the FS as well.
2012-11-27 22:10:52 +00:00
break;
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 12:30:28 +01:00
case SHADER_OPCODE_MEMORY_FENCE:
i965: Add ARB_fragment_shader_interlock support. Adds suppport for ARB_fragment_shader_interlock. We achieve the interlock and fragment ordering by issuing a memory fence via sendc. Signed-off-by: Plamena Manolova <plamena.manolova@intel.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2018-04-27 15:06:56 +01:00
brw_memory_fence(p, dst, BRW_OPCODE_SEND);
break;
case SHADER_OPCODE_INTERLOCK:
/* The interlock is basically a memory fence issued via sendc */
brw_memory_fence(p, dst, BRW_OPCODE_SENDC);
i965: Add memory fence opcode. Acked-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2015-04-23 12:30:28 +01:00
break;
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 23:09:33 +01:00
case SHADER_OPCODE_FIND_LIVE_CHANNEL: {
const struct brw_reg mask =
i965/ir: Pass identity mask to brw_find_live_channel() in the packed dispatch case. This avoids emitting a few extra instructions required to take the dispatch mask into account when it's known to be tightly packed. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-09-16 01:24:10 +01:00
brw_stage_has_packed_dispatch(devinfo, stage,
prog_data) ? brw_imm_ud(~0u) :
stage == MESA_SHADER_FRAGMENT ? brw_vmask_reg() :
brw_dmask_reg();
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 23:09:33 +01:00
brw_find_live_channel(p, dst, mask);
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 12:42:53 +01:00
break;
i965/fs: Take Dispatch/Vector mask into account in FIND_LIVE_CHANNEL On at least Sky Lake, ce0 does not contain the full story as far as enabled channels goes. It is possible to have completely disabled channels where the corresponding bits in ce0 are 1. In order to get the correct execution mask, you have to mask off those channels which were disabled from the beginning by taking the AND of ce0 with either sr0.2 or sr0.3 depending on the shader stage. Failure to do so can result in FIND_LIVE_CHANNEL returning a completely dead channel. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Cc: Francisco Jerez <currojerez@riseup.net> [ Francisco Jerez: Fix a couple of typos, add mask register type assertion, clarify reason why ce0 can have bits set for disabled channels, clarify that this may only be a problem when thread dispatch doesn't pack channels tightly in the SIMD thread. Apply same treatment to Align16 path. ] Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-09-14 23:09:33 +01:00
}
i965: Introduce the FIND_LIVE_CHANNEL pseudo-opcode. This instruction calculates the index of an arbitrary channel enabled in the current execution mask. It's expected to be used as input for the BROADCAST opcode, but it's implemented as a separate instruction rather than being baked into BROADCAST because FIND_LIVE_CHANNEL has no dependencies so it can always be CSE'ed with other instances of the same instruction within a basic block. v2: Whitespace fixes. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-23 12:42:53 +01:00
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 18:14:24 +00:00
case SHADER_OPCODE_BROADCAST:
i965/ir: Make BROADCAST emit an unmasked single-channel move. Alternatively we could have extended the current semantics to 32-wide mode by changing brw_broadcast() to emit multiple indexed MOV instructions in the generator copying the selected value to all destination registers, but it seemed rather silly to waste EU cycles unnecessarily copying the exact same value 32 times in the GRF. The vstride change in the Align16 path is required to avoid assertions in validate_reg() since the change causes the execution size of the MOV and SEL instructions to be equal to the source region width. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-05-19 08:10:03 +01:00
assert(inst->force_writemask_all);
i965: Introduce the BROADCAST pseudo-opcode. The BROADCAST instruction picks the channel from its first source given by an index passed in as second source. This will be used in situations where all channels from the same SIMD thread have to agree on the value of something, e.g. a surface binding table index. This is in particular the case for UBO, sampler and image arrays, which can be indexed dynamically with the restriction that all active SIMD channels access the same index, provided to the shared unit as part of a single scalar field of the message descriptor. Simply taking the index value from the first channel as we were doing until now is incorrect, because it might contain an uninitialized value if the channel had previously been disabled by non-uniform control flow. v2: Minor style fixes. Improve commit message. Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-02-20 18:14:24 +00:00
brw_broadcast(p, dst, src[0], src[1]);
break;
i965/fs: Add support for nir_intrinsic_shuffle Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-08-29 17:21:32 +01:00
case SHADER_OPCODE_SHUFFLE:
generate_shuffle(inst, dst, src[0], src[1]);
break;
intel/fs: Add a couple of simple helper opcodes Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-09-01 05:45:30 +01:00
case SHADER_OPCODE_SEL_EXEC:
assert(inst->force_writemask_all);
brw_set_default_mask_control(p, BRW_MASK_DISABLE);
brw_MOV(p, dst, src[1]);
brw_set_default_mask_control(p, BRW_MASK_ENABLE);
brw_MOV(p, dst, src[0]);
break;
intel/fs: Add support for subgroup quad operations NIR has code to lower these away for us but we can do significantly better in many cases with register regioning and SIMD4x2. Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-09-01 23:18:02 +01:00
case SHADER_OPCODE_QUAD_SWIZZLE:
assert(src[1].file == BRW_IMMEDIATE_VALUE);
assert(src[1].type == BRW_REGISTER_TYPE_UD);
intel/fs: Implement quad swizzles on ICL+. Align16 is no longer a thing, so a new implementation is provided using Align1 instead. Not all possible swizzles can be represented as a single Align1 region, but some fast paths are provided for frequently used swizzles that can be represented efficiently in Align1 mode. Fixes ~90 subgroup quad swap Vulkan CTS tests. Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-12-06 22:11:34 +00:00
generate_quad_swizzle(inst, dst, src[0], src[1].ud);
intel/fs: Add support for subgroup quad operations NIR has code to lower these away for us but we can do significantly better in many cases with register regioning and SIMD4x2. Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-09-01 23:18:02 +01:00
break;
intel/fs: Add a couple of simple helper opcodes Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-09-01 05:45:30 +01:00
case SHADER_OPCODE_CLUSTER_BROADCAST: {
assert(src[0].type == dst.type);
assert(!src[0].negate && !src[0].abs);
assert(src[1].file == BRW_IMMEDIATE_VALUE);
assert(src[1].type == BRW_REGISTER_TYPE_UD);
assert(src[2].file == BRW_IMMEDIATE_VALUE);
assert(src[2].type == BRW_REGISTER_TYPE_UD);
const unsigned component = src[1].ud;
const unsigned cluster_size = src[2].ud;
struct brw_reg strided = stride(suboffset(src[0], component),
cluster_size, cluster_size, 0);
if (type_sz(src[0].type) > 4 &&
(devinfo->is_cherryview || gen_device_info_is_9lp(devinfo))) {
/* IVB has an issue (which we found empirically) where it reads
* two address register components per channel for indirectly
* addressed 64-bit sources.
*
* From the Cherryview PRM Vol 7. "Register Region Restrictions":
*
* "When source or destination datatype is 64b or operation is
* integer DWord multiply, indirect addressing must not be
* used."
*
* To work around both of these, we do two integer MOVs insead of
* one 64-bit MOV. Because no double value should ever cross a
* register boundary, it's safe to use the immediate offset in the
* indirect here to handle adding 4 bytes to the offset and avoid
* the extra ADD to the register file.
*/
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 0),
subscript(strided, BRW_REGISTER_TYPE_D, 0));
brw_MOV(p, subscript(dst, BRW_REGISTER_TYPE_D, 1),
subscript(strided, BRW_REGISTER_TYPE_D, 1));
} else {
brw_MOV(p, dst, strided);
}
break;
}
i965: Add FS backend for builtin gl_SampleID V2: - Update comments - Add compute_sample_id variables in brw_wm_prog_key - Add a special backend instruction to compute sample_id. V3: - Make changes to support simd16 mode. Signed-off-by: Anuj Phogat <anuj.phogat@gmail.com> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2013-10-25 00:17:08 +01:00
case FS_OPCODE_SET_SAMPLE_ID:
generate_set_sample_id(inst, dst, src[0], src[1]);
break;
i965/fs/gen7: Emit code for GLSL 3.00 pack/unpack operations (v4) v2: Remove lewd comment. [for idr] v3: - Optimize away tmp register for packHalf2x16. [for anholt, paul] - Improve comments. [for anholt, paul] - Reduce near-duplicate code by removing vec4_visitor emit_pack/unpack methods. [for chadv] v4: Factor our UD/W register conversion into helper function. [for anholt] Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> (v2) Signed-off-by: Chad Versace <chad.versace@linux.intel.com>
2013-01-09 19:46:42 +00:00
case FS_OPCODE_PACK_HALF_2x16_SPLIT:
generate_pack_half_2x16_split(inst, dst, src[0], src[1]);
break;
i965: Fix INTEL_DEBUG=shader_time for fragment shaders with discards. "discard" instructions generate HALT instructions which jump to a final HALT near the end of the shader. Previously, fs_generator created this final jump target when it saw the first FS_OPCODE_FB_WRITE, causing it to jump right before the FB write epilogue. This is normally good. However, INTEL_DEBUG=shader_time also has an epilogue section which records the final timestamp. The frontend emits IR for this just before FS_OPCODE_FB_WRITE. Unfortunately, this led to the following ordering: 1. Shader Time Epilogue 2. Final HALT (where discards jump) 3. Framebuffer Write Epilogue This meant that discarded pixels completely skipped the shader time epilogue, causing no ending timestamp to be written. This obviously led to inaccurate results. This patch adds a new FS_OPCODE_PLACEHOLDER_HALT in the IR stream just before any epilogue sections. This is where the final HALT should be generated, and makes it easy to ensure the correct ordering: 1. Final HALT 2. Shader Time Epilogue 3. Framebuffer Write Epilogue For shaders that don't discard, this opcode compiles away to nothing. The scheduler adds barrier dependencies to make sure that it doesn't get moved above any FS_OPCODE_DISCARD_JUMP instructions. One 8-wide shader in GLBenchmark 2.7 dropped from 2291.67 Gcycles to a mere 5.13 Gcycles. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-28 06:19:39 +00:00
case FS_OPCODE_PLACEHOLDER_HALT:
/* This is the place where the final HALT needs to be inserted if
* we've emitted any discards. If not, this will emit no code.
*/
i965: Print disassembly after compaction. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-19 18:20:37 +01:00
if (!patch_discard_jumps_to_fb_writes()) {
i965/fs: Don't modify ann_count if not debugging. If we make ann_count non-zero, annotation_finalize() won't bail. Not modifying it seems to make the code more clear than would modifying annotation_finalize().
2014-05-25 18:30:13 +01:00
if (unlikely(debug_flag)) {
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
disasm_info->use_tail = true;
i965/fs: Don't modify ann_count if not debugging. If we make ann_count non-zero, annotation_finalize() won't bail. Not modifying it seems to make the code more clear than would modifying annotation_finalize().
2014-05-25 18:30:13 +01:00
}
i965: Print disassembly after compaction. Reviewed-by: Eric Anholt <eric@anholt.net>
2014-05-19 18:20:37 +01:00
}
i965: Fix INTEL_DEBUG=shader_time for fragment shaders with discards. "discard" instructions generate HALT instructions which jump to a final HALT near the end of the shader. Previously, fs_generator created this final jump target when it saw the first FS_OPCODE_FB_WRITE, causing it to jump right before the FB write epilogue. This is normally good. However, INTEL_DEBUG=shader_time also has an epilogue section which records the final timestamp. The frontend emits IR for this just before FS_OPCODE_FB_WRITE. Unfortunately, this led to the following ordering: 1. Shader Time Epilogue 2. Final HALT (where discards jump) 3. Framebuffer Write Epilogue This meant that discarded pixels completely skipped the shader time epilogue, causing no ending timestamp to be written. This obviously led to inaccurate results. This patch adds a new FS_OPCODE_PLACEHOLDER_HALT in the IR stream just before any epilogue sections. This is where the final HALT should be generated, and makes it easy to ensure the correct ordering: 1. Final HALT 2. Shader Time Epilogue 3. Framebuffer Write Epilogue For shaders that don't discard, this opcode compiles away to nothing. The scheduler adds barrier dependencies to make sure that it doesn't get moved above any FS_OPCODE_DISCARD_JUMP instructions. One 8-wide shader in GLBenchmark 2.7 dropped from 2291.67 Gcycles to a mere 5.13 Gcycles. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Eric Anholt <eric@anholt.net>
2013-03-28 06:19:39 +00:00
break;
i965/fs: add generator support for pixel interpolator query V5: - Split into separate opcodes - Pass message data in src1 immediate - Put noperspective bit in fs_inst rather than adding any junk to backend_instruction Signed-off-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2013-11-18 08:13:13 +00:00
case FS_OPCODE_INTERPOLATE_AT_SAMPLE:
generate_pixel_interpolator_query(inst, dst, src[0], src[1],
GEN7_PIXEL_INTERPOLATOR_LOC_SAMPLE);
break;
case FS_OPCODE_INTERPOLATE_AT_SHARED_OFFSET:
generate_pixel_interpolator_query(inst, dst, src[0], src[1],
GEN7_PIXEL_INTERPOLATOR_LOC_SHARED_OFFSET);
break;
case FS_OPCODE_INTERPOLATE_AT_PER_SLOT_OFFSET:
generate_pixel_interpolator_query(inst, dst, src[0], src[1],
GEN7_PIXEL_INTERPOLATOR_LOC_PER_SLOT_OFFSET);
break;
i965/cs: Add generator support for CS_OPCODE_CS_TERMINATE v2: * Don't rely on brw_eu* to generate the send instruction. We now generate the send here, and drop the "i965/cs: Add support for the SEND message that terminates a CS thread" brw_eu* patch. Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-08-27 19:33:25 +01:00
case CS_OPCODE_CS_TERMINATE:
generate_cs_terminate(inst, src[0]);
break;
i965/fs: Implement support for ir_barrier Signed-off-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2014-08-27 19:32:08 +01:00
case SHADER_OPCODE_BARRIER:
generate_barrier(inst, src[0]);
break;
i965: enable the emission of the DIM instruction v2 (Matt): - Take a DF source argument for the DIM instruction emission in the visitors. - Indentation. Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-07-07 07:38:22 +01:00
case BRW_OPCODE_DIM:
assert(devinfo->is_haswell);
assert(src[0].type == BRW_REGISTER_TYPE_DF);
assert(dst.type == BRW_REGISTER_TYPE_DF);
brw_DIM(p, dst, retype(src[0], BRW_REGISTER_TYPE_F));
break;
i965/fs: Define new shader opcode to set rounding modes Although it is possible to emit them directly as AND/OR on brw_fs_nir, having a specific opcode makes it easier to remove duplicate settings later. v2: (Curro) - Set thread control to 'switch' when using the control register - Use a single SHADER_OPCODE_RND_MODE opcode taking an immediate with the rounding mode. - Avoid magic numbers setting rounding mode field at control register. v3: (Curro) - Remove redundant and add missing whitespace lines. - Match printing instruction to IR opcode "rnd_mode" v4: (Topi Pohjolainen) - Fix code style. Signed-off-by: Alejandro Piñeiro <apinheiro@igalia.com> Signed-off-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com> Reviewed-by: Francisco Jerez <currojerez@riseup.net> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-07-01 07:12:59 +01:00
case SHADER_OPCODE_RND_MODE:
assert(src[0].file == BRW_IMMEDIATE_VALUE);
brw_rounding_mode(p, (brw_rnd_mode) src[0].d);
break;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
default:
i965/fs: Remove the GL context from the generator Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 22:51:18 +01:00
unreachable("Unsupported opcode");
i965/fs: Add SHADER_OPCODE_LOAD_PAYLOAD. Will be used to simplify the handling of large virtual GRFs in SSA form. Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2014-05-28 02:47:40 +01:00
case SHADER_OPCODE_LOAD_PAYLOAD:
i965: Use unreachable() instead of unconditional assert(). Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-06-29 22:54:01 +01:00
unreachable("Should be lowered by lower_load_payload()");
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965: Don't use brw_set_conditionalmod in the FS and vec4 compilers. brw_set_conditionalmod and brw_next_insn work together to set the conditional modifier for the next instruction, then turn it off. The Gen8+ generators don't implement this: we just set it for all future instructions, and whack it for each fs_inst/vec4_instruction. Both approaches work out because we only set conditional_mod on IR instructions like CMP, AND, and so on, which correspond to exactly one assembly instruction. The Gen8 generators would break if we had an IR instruction that generated multiple instructions, and the Gen4-7 EU emit layer would do...something. To safeguard against this, assert that we only generated one instruction if conditional_mod is set, and just set the flag directly on that instruction rather than altering default state. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 00:41:32 +01:00
i965/fs: Apply conditional mod specially to split MAD/LRP. Otherwise we'll apply the conditional mod to only one of SIMD8 instructions and trigger an assertion. NoDDClr/NoDDChk have the same problem but we never apply those to these instructions, so I'm leaving them for a later time. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-12-30 20:56:13 +00:00
if (multiple_instructions_emitted)
continue;
i965/fs: Add no_dd_{clear,check} fields to fs_inst. And plumb them through. Also make the assert in the generator look like the vec4 one. Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2014-06-29 07:31:04 +01:00
if (inst->no_dd_clear || inst->no_dd_check || inst->conditional_mod) {
assert(p->next_insn_offset == last_insn_offset + 16 ||
!"conditional_mod, no_dd_check, or no_dd_clear set for IR "
"emitting more than 1 instruction");
i965: Replace 'struct brw_instruction' with 'brw_inst'. Use this an an opportunity to clean up the formatting of some old code (brw_ADD, for instance). Signed-off-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-06-13 22:29:25 +01:00
brw_inst *last = &p->store[last_insn_offset / 16];
i965/fs: Add no_dd_{clear,check} fields to fs_inst. And plumb them through. Also make the assert in the generator look like the vec4 one. Reviewed-by: Kristian Høgsberg <krh@bitplanet.net>
2014-06-29 07:31:04 +01:00
i965: Don't overwrite the math function with conditional mod. Ben was asking about the undocumented restriction that the math instruction cannot use the dependency control hints. I went to reconfirm and disabled the is_math() check in opt_set_dependency_control() and saw that the disassembled math instructions with dependency hints had a bogus math function. We were mistakenly overwriting it by setting an empty conditional mod. Unfortunately, this wasn't the cause of the aforementioned problem (I reproduced it). This bug is benign, since we don't set dependeny hints on math instructions -- but maybe some day. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-21 20:20:53 +00:00
if (inst->conditional_mod)
i965: Make the brw_inst helpers take a device_info instead of a context Reviewed-by: Matt Turner <mattst88@gmail.com>
2015-04-15 02:00:06 +01:00
brw_inst_set_cond_modifier(p->devinfo, last, inst->conditional_mod);
brw_inst_set_no_dd_clear(p->devinfo, last, inst->no_dd_clear);
brw_inst_set_no_dd_check(p->devinfo, last, inst->no_dd_check);
i965: Don't use brw_set_conditionalmod in the FS and vec4 compilers. brw_set_conditionalmod and brw_next_insn work together to set the conditional modifier for the next instruction, then turn it off. The Gen8+ generators don't implement this: we just set it for all future instructions, and whack it for each fs_inst/vec4_instruction. Both approaches work out because we only set conditional_mod on IR instructions like CMP, AND, and so on, which correspond to exactly one assembly instruction. The Gen8 generators would break if we had an IR instruction that generated multiple instructions, and the Gen4-7 EU emit layer would do...something. To safeguard against this, assert that we only generated one instruction if conditional_mod is set, and just set the flag directly on that instruction rather than altering default state. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Matt Turner <mattst88@gmail.com>
2014-05-31 00:41:32 +01:00
}
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965: Pass start_offset to brw_set_uip_jip(). Without this, we would pass over the instructions in the SIMD8 program (which is located earlier in the buffer) when brw_set_uip_jip() is called to handle the SIMD16 program. The assertion about compacted control flow was bogus: halt, cont, break cannot be compacted because they have both JIP and UIP. Instead, we should never see a compacted instruction in this code at all. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2016-08-29 23:57:41 +01:00
brw_set_uip_jip(p, start_offset);
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
/* end of program sentinel */
disasm_new_inst_group(disasm_info, p->next_insn_offset);
i965: Move annotation info into generate code. Suggested by Ken as a way to cut down lines of code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:42:32 +01:00
i965: Add initial assembly validation pass. Initially just checks that sources are non-NULL, which would have alerted us to the problem fixed by commit 6c846dc5. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-06-29 22:08:51 +01:00
#ifndef NDEBUG
i965: Move common code out of #ifdef I'm going to change the call in a later patch and with the difference in indentation level it wasn't immediately obvious that the calls were identical. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 21:35:01 +00:00
bool validated =
i965: Add initial assembly validation pass. Initially just checks that sources are non-NULL, which would have alerted us to the problem fixed by commit 6c846dc5. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-06-29 22:08:51 +01:00
#else
if (unlikely(debug_flag))
i965: Move common code out of #ifdef I'm going to change the call in a later patch and with the difference in indentation level it wasn't immediately obvious that the calls were identical. Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 21:35:01 +00:00
#endif
i965: Pass pointer and end of assembly to brw_validate_instructions This will allow us to more easily run brw_validate_instructions() on shader programs we find in GPU hang error states. Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2017-04-29 01:05:44 +01:00
brw_validate_instructions(devinfo, p->store,
start_offset,
p->next_insn_offset,
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
disasm_info);
i965: Add initial assembly validation pass. Initially just checks that sources are non-NULL, which would have alerted us to the problem fixed by commit 6c846dc5. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-06-29 22:08:51 +01:00
i965: Emit compaction stats without walking the assembly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 22:56:41 +01:00
int before_size = p->next_insn_offset - start_offset;
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
brw_compact_instructions(p, start_offset, disasm_info);
i965: Emit compaction stats without walking the assembly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 22:56:41 +01:00
int after_size = p->next_insn_offset - start_offset;
i965: Move annotation info into generate code. Suggested by Ken as a way to cut down lines of code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:42:32 +01:00
if (unlikely(debug_flag)) {
i965: Set shader name for generator from call site fs_generator no longer knows what stage it's generating code for, so we have to set the debug name of the shader from the call site. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-28 02:40:47 +00:00
fprintf(stderr, "Native code for %s\n"
i965: dump scheduling cycle estimates The heuristic we're using is rather lame, since it assumes everything is non-uniform and loops execute 10 times, but it should be enough for measuring improvements in the scheduler that don't result in a change in the number of instructions. v2: - Switch loops and cycle counts to be compatible with older shader-db. - Make loop heuristic 10x to match with spilling code. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-06-06 15:55:21 +01:00
"SIMD%d shader: %d instructions. %d loops. %u cycles. %d:%d spills:fills. Promoted %u constants. Compacted %d to %d"
i965: Set shader name for generator from call site fs_generator no longer knows what stage it's generating code for, so we have to set the debug name of the shader from the call site. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-10-28 02:40:47 +00:00
" bytes (%.0f%%)\n",
i965: dump scheduling cycle estimates The heuristic we're using is rather lame, since it assumes everything is non-uniform and loops execute 10 times, but it should be enough for measuring improvements in the scheduler that don't result in a change in the number of instructions. v2: - Switch loops and cycle counts to be compatible with older shader-db. - Make loop heuristic 10x to match with spilling code. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-06-06 15:55:21 +01:00
shader_name, dispatch_width, before_size / 16, loop_count, cfg->cycle_count,
i965/fs: Print spills:fills and number of promoted constants. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com> Reviewed-by: Chris Forbes <chrisf@ijw.co.nz>
2015-03-16 19:18:31 +00:00
spill_count, fill_count, promoted_constants, before_size, after_size,
i965: Emit compaction stats without walking the assembly. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 22:56:41 +01:00
100.0f * (before_size - after_size) / before_size);
i965: Move program header printing to end of generate_code(). Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:46:55 +01:00
i965: Rewrite disassembly annotation code The old code used an array to store each "instruction group" (the new, better name than the old overloaded "annotation"), and required a memmove() to shift elements over in the array when we needed to split a group so that we could add an error message. This was confusing and difficult to get right, not the least of which was because the array has a tail sentinel not included in .ann_count. Instead use a linked list, a data structure made for efficient insertion. Acked-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2017-11-16 01:08:42 +00:00
dump_assembly(p->store, disasm_info);
i965: Move annotation info into generate code. Suggested by Ken as a way to cut down lines of code. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-05-25 18:42:32 +01:00
}
intel: fix disasm_info memory leaks Fixes: 4f82b1728719 ("i965: Rewrite disassembly annotation code") Cc: Matt Turner <mattst88@gmail.com> Signed-off-by: Tapani Pälli <tapani.palli@intel.com> Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2017-11-20 08:57:17 +00:00
ralloc_free(disasm_info);
i965: Add initial assembly validation pass. Initially just checks that sources are non-NULL, which would have alerted us to the problem fixed by commit 6c846dc5. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-06-29 22:08:51 +01:00
assert(validated);
i965: Refactor fs_generator API We split out SIMD8 and SIMD16 generation into seperate calls to new method generate_code(), which returns the start offset for the generated code. A new get_assembly() method returns the generated code. This avoids asserting MESA_SHADER_FRAGMENT and accessing wm_prog_data in the generator. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-14 00:28:08 +00:00
i965: Plumb compiler debug logging through a function pointer in brw_compiler v2 (Ken): Make shader_debug_log a printf-like function. v3 (Jason): Add a void * to pass the brw_context through Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-16 22:13:52 +01:00
compiler->shader_debug_log(log_data,
i965: dump scheduling cycle estimates The heuristic we're using is rather lame, since it assumes everything is non-uniform and loops execute 10 times, but it should be enough for measuring improvements in the scheduler that don't result in a change in the number of instructions. v2: - Switch loops and cycle counts to be compatible with older shader-db. - Make loop heuristic 10x to match with spilling code. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-06-06 15:55:21 +01:00
"%s SIMD%d shader: %d inst, %d loops, %u cycles, "
i965: Plumb compiler debug logging through a function pointer in brw_compiler v2 (Ken): Make shader_debug_log a printf-like function. v3 (Jason): Add a void * to pass the brw_context through Reviewed-by: Chris Forbes <chrisf@ijw.co.nz> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-04-16 22:13:52 +01:00
"%d:%d spills:fills, Promoted %u constants, "
i965: Drop extra newline from shader compile messages. Ilia changed shader-db's run.c to not expect messages to contain a newline in shader-db commit 51bbc8035.
2016-01-14 00:17:26 +00:00
"compacted %d to %d bytes.",
i965/fs/generator: Take an actual shader stage rather than a string Cc: "11.1" <mesa-stable@lists.freedesktop.org> Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Matt Turner <mattst88@gmail.com>
2016-01-15 04:27:51 +00:00
_mesa_shader_stage_to_abbrev(stage),
dispatch_width, before_size / 16,
i965: dump scheduling cycle estimates The heuristic we're using is rather lame, since it assumes everything is non-uniform and loops execute 10 times, but it should be enough for measuring improvements in the scheduler that don't result in a change in the number of instructions. v2: - Switch loops and cycle counts to be compatible with older shader-db. - Make loop heuristic 10x to match with spilling code. Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-06-06 15:55:21 +01:00
loop_count, cfg->cycle_count, spill_count,
fill_count, promoted_constants, before_size,
after_size);
i965: Give compile stats through KHR_debug. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2014-11-14 20:46:44 +00:00
i965: Refactor fs_generator API We split out SIMD8 and SIMD16 generation into seperate calls to new method generate_code(), which returns the start offset for the generated code. A new get_assembly() method returns the generated code. This avoids asserting MESA_SHADER_FRAGMENT and accessing wm_prog_data in the generator. Signed-off-by: Kristian Høgsberg <krh@bitplanet.net> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2014-11-14 00:28:08 +00:00
return start_offset;
i965/fs: Split the BRW native code emit to brw_fs_emit.cpp This is all separate from the visitor and the optimization passes which feed into it. Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2011-05-25 00:34:27 +01:00
}
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
const unsigned *
intel: Drop program size pointer from vec4/fs assembly getters. These days, we're just passing a pointer to a prog_data field, which we already have access to. We can just use it directly. (In the past, it was a pointer to a separate value.) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-02-27 00:34:55 +00:00
fs_generator::get_assembly()
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
{
intel: Drop program size pointer from vec4/fs assembly getters. These days, we're just passing a pointer to a prog_data field, which we already have access to. We can just use it directly. (In the past, it was a pointer to a separate value.) Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-02-27 00:34:55 +00:00
return brw_get_program(p, &prog_data->program_size);
i965/fs: Split final assembly code generation out of fs_visitor. Compiling shaders requires several main steps: 1. Generating FS IR from either GLSL IR or Mesa IR 2. Optimizing the IR 3. Register allocation 4. Generating assembly code This patch splits out step 4 into a separate class named "fs_generator." There are several reasons for doing so: 1. Future hardware has a different instruction encoding. Splitting this out will allow us to replace fs_generator (which relies heavily on the brw_eu_emit.c code and struct brw_instruction) with a new code generator that writes the new format. 2. It reduces the size of the fs_visitor monolith. (Arguably, a lot more should be split out, but that's left for "future work.") 3. Separate namespaces allow us to make helper functions for generating instructions in both classes: ADD() can exist in fs_visitor and create IR, while ADD() in fs_generator() can create brw_instructions. (Patches for this upcoming.) Furthermore, this patch changes the order of operations slightly. Rather than doing steps 1-4 for SIMD8, then 1-4 for SIMD16, we now: - Do steps 1-3 for SIMD8, then repeat 1-3 for SIMD16 - Generate final assembly code for both modes together This is because the frontend work can be done independently, but final assembly generation needs to pack both into a single program store to feed the GPU. Reviewed-by: Eric Anholt <eric@anholt.net> Reviewed-by: Paul Berry <stereotype441@gmail.com>
2012-11-09 09:05:47 +00:00
}