2013-12-12 08:30:16 +00:00
|
|
|
/*
|
|
|
|
* Copyright © 2013 Intel Corporation
|
|
|
|
*
|
|
|
|
* Permission is hereby granted, free of charge, to any person obtaining a
|
|
|
|
* copy of this software and associated documentation files (the "Software"),
|
|
|
|
* to deal in the Software without restriction, including without limitation
|
|
|
|
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
|
|
|
|
* and/or sell copies of the Software, and to permit persons to whom the
|
|
|
|
* Software is furnished to do so, subject to the following conditions:
|
|
|
|
*
|
|
|
|
* The above copyright notice and this permission notice (including the next
|
|
|
|
* paragraph) shall be included in all copies or substantial portions of the
|
|
|
|
* Software.
|
|
|
|
*
|
|
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
|
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
|
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
|
|
|
|
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
|
|
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
|
|
|
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
|
|
|
|
* IN THE SOFTWARE.
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include "brw_fs.h"
|
|
|
|
#include "brw_fs_live_variables.h"
|
|
|
|
#include "brw_cfg.h"
|
|
|
|
|
|
|
|
/** @file brw_fs_saturate_propagation.cpp
|
2014-12-16 19:30:12 +00:00
|
|
|
*
|
|
|
|
* Implements a pass that propagates the SAT modifier from a MOV.SAT into the
|
|
|
|
* instruction that produced the source of the MOV.SAT, thereby allowing the
|
|
|
|
* MOV's src and dst to be coalesced and the MOV removed.
|
|
|
|
*
|
|
|
|
* For instance,
|
|
|
|
*
|
|
|
|
* ADD tmp, src0, src1
|
|
|
|
* MOV.SAT dst, tmp
|
|
|
|
*
|
|
|
|
* would be transformed into
|
|
|
|
*
|
|
|
|
* ADD.SAT tmp, src0, src1
|
|
|
|
* MOV dst, tmp
|
2013-12-12 08:30:16 +00:00
|
|
|
*/
|
|
|
|
|
|
|
|
static bool
|
intel/compiler: split is_partial_write() into two variants
This function is used in two different scenarios that for 32-bit
instructions are the same, but for 16-bit instructions are not.
One scenario is that in which we are working at a SIMD8 register
level and we need to know if a register is fully defined or written.
This is useful, for example, in the context of liveness analysis or
register allocation, where we work with units of registers.
The other scenario is that in which we want to know if an instruction
is writing a full scalar component or just some subset of it. This is
useful, for example, in the context of some optimization passes
like copy propagation.
For 32-bit instructions (or larger), a SIMD8 dispatch will always write
at least a full SIMD8 register (32B) if the write is not partial. The
function is_partial_write() checks this to determine if we have a partial
write. However, when we deal with 16-bit instructions, that logic disables
some optimizations that should be safe. For example, a SIMD8 16-bit MOV will
only update half of a SIMD register, but it is still a complete write of the
variable for a SIMD8 dispatch, so we should not prevent copy propagation in
this scenario because we don't write all 32 bytes in the SIMD register
or because the write starts at offset 16B (wehere we pack components Y or
W of 16-bit vectors).
This is a problem for SIMD8 executions (VS, TCS, TES, GS) of 16-bit
instructions, which lose a number of optimizations because of this, most
important of which is copy-propagation.
This patch splits is_partial_write() into is_partial_reg_write(), which
represents the current is_partial_write(), useful for things like
liveness analysis, and is_partial_var_write(), which considers
the dispatch size to check if we are writing a full variable (rather
than a full register) to decide if the write is partial or not, which
is what we really want in many optimization passes.
Then the patch goes on and rewrites all uses of is_partial_write() to use
one or the other version. Specifically, we use is_partial_var_write()
in the following places: copy propagation, cmod propagation, common
subexpression elimination, saturate propagation and sel peephole.
Notice that the semantics of is_partial_var_write() exactly match the
current implementation of is_partial_write() for anything that is
32-bit or larger, so no changes are expected for 32-bit instructions.
Tested against ~5000 tests involving 16-bit instructions in CTS produced
the following changes in instruction counts:
Patched | Master | % |
================================================
SIMD8 | 621,900 | 706,721 | -12.00% |
================================================
SIMD16 | 93,252 | 93,252 | 0.00% |
================================================
As expected, the change only affects SIMD8 dispatches.
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2018-07-10 08:52:46 +01:00
|
|
|
opt_saturate_propagation_local(fs_visitor *v, bblock_t *block,
|
|
|
|
unsigned dispatch_width)
|
2013-12-12 08:30:16 +00:00
|
|
|
{
|
|
|
|
bool progress = false;
|
2014-09-08 20:05:25 +01:00
|
|
|
int ip = block->end_ip + 1;
|
2013-12-12 08:30:16 +00:00
|
|
|
|
2014-09-08 20:05:25 +01:00
|
|
|
foreach_inst_in_block_reverse(fs_inst, inst, block) {
|
|
|
|
ip--;
|
2013-12-12 08:30:16 +00:00
|
|
|
|
|
|
|
if (inst->opcode != BRW_OPCODE_MOV ||
|
i965/fs: Consider type mismatches in saturate propagation.
NIR considers bcsel to produce and consume unsigned types, leading to
SEL instructions operating on unsigned types when the data is really
floating-point. Previous to this patch, saturate propagation would
happily transform
(+f0) sel g20:UD, g30:UD, g40:UD
mov.sat g50:F, g20:F
into
(+f0) sel.sat g20:UD, g30:UD, g40:UD
mov g50:F, g20:F
But since the meaning of .sat is dependent on the type of the
destination register, this is not valid.
Instead, allow saturate propagation to change the types of dest/source
on instructions that are simply copying data in order to propagate the
saturate modifier.
Fixes bad code gen in 158 programs.
Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-10-14 10:23:25 +01:00
|
|
|
!inst->saturate ||
|
2015-10-27 00:09:25 +00:00
|
|
|
inst->dst.file != VGRF ||
|
i965/fs: Consider type mismatches in saturate propagation.
NIR considers bcsel to produce and consume unsigned types, leading to
SEL instructions operating on unsigned types when the data is really
floating-point. Previous to this patch, saturate propagation would
happily transform
(+f0) sel g20:UD, g30:UD, g40:UD
mov.sat g50:F, g20:F
into
(+f0) sel.sat g20:UD, g30:UD, g40:UD
mov g50:F, g20:F
But since the meaning of .sat is dependent on the type of the
destination register, this is not valid.
Instead, allow saturate propagation to change the types of dest/source
on instructions that are simply copying data in order to propagate the
saturate modifier.
Fixes bad code gen in 158 programs.
Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-10-14 10:23:25 +01:00
|
|
|
inst->dst.type != inst->src[0].type ||
|
2015-10-27 00:09:25 +00:00
|
|
|
inst->src[0].file != VGRF ||
|
2015-01-28 06:46:22 +00:00
|
|
|
inst->src[0].abs)
|
2013-12-12 08:30:16 +00:00
|
|
|
continue;
|
|
|
|
|
2014-10-29 20:58:45 +00:00
|
|
|
int src_var = v->live_intervals->var_from_reg(inst->src[0]);
|
2013-12-12 08:30:16 +00:00
|
|
|
int src_end_ip = v->live_intervals->end[src_var];
|
|
|
|
|
|
|
|
bool interfered = false;
|
2015-10-20 10:16:00 +01:00
|
|
|
foreach_inst_in_block_reverse_starting_from(fs_inst, scan_inst, inst) {
|
2018-10-19 23:33:50 +01:00
|
|
|
if (scan_inst->exec_size == inst->exec_size &&
|
|
|
|
regions_overlap(scan_inst->dst, scan_inst->size_written,
|
2016-09-02 03:34:18 +01:00
|
|
|
inst->src[0], inst->size_read(0))) {
|
intel/compiler: split is_partial_write() into two variants
This function is used in two different scenarios that for 32-bit
instructions are the same, but for 16-bit instructions are not.
One scenario is that in which we are working at a SIMD8 register
level and we need to know if a register is fully defined or written.
This is useful, for example, in the context of liveness analysis or
register allocation, where we work with units of registers.
The other scenario is that in which we want to know if an instruction
is writing a full scalar component or just some subset of it. This is
useful, for example, in the context of some optimization passes
like copy propagation.
For 32-bit instructions (or larger), a SIMD8 dispatch will always write
at least a full SIMD8 register (32B) if the write is not partial. The
function is_partial_write() checks this to determine if we have a partial
write. However, when we deal with 16-bit instructions, that logic disables
some optimizations that should be safe. For example, a SIMD8 16-bit MOV will
only update half of a SIMD register, but it is still a complete write of the
variable for a SIMD8 dispatch, so we should not prevent copy propagation in
this scenario because we don't write all 32 bytes in the SIMD register
or because the write starts at offset 16B (wehere we pack components Y or
W of 16-bit vectors).
This is a problem for SIMD8 executions (VS, TCS, TES, GS) of 16-bit
instructions, which lose a number of optimizations because of this, most
important of which is copy-propagation.
This patch splits is_partial_write() into is_partial_reg_write(), which
represents the current is_partial_write(), useful for things like
liveness analysis, and is_partial_var_write(), which considers
the dispatch size to check if we are writing a full variable (rather
than a full register) to decide if the write is partial or not, which
is what we really want in many optimization passes.
Then the patch goes on and rewrites all uses of is_partial_write() to use
one or the other version. Specifically, we use is_partial_var_write()
in the following places: copy propagation, cmod propagation, common
subexpression elimination, saturate propagation and sel peephole.
Notice that the semantics of is_partial_var_write() exactly match the
current implementation of is_partial_write() for anything that is
32-bit or larger, so no changes are expected for 32-bit instructions.
Tested against ~5000 tests involving 16-bit instructions in CTS produced
the following changes in instruction counts:
Patched | Master | % |
================================================
SIMD8 | 621,900 | 706,721 | -12.00% |
================================================
SIMD16 | 93,252 | 93,252 | 0.00% |
================================================
As expected, the change only affects SIMD8 dispatches.
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2018-07-10 08:52:46 +01:00
|
|
|
if (scan_inst->is_partial_var_write(dispatch_width) ||
|
i965/fs: Consider type mismatches in saturate propagation.
NIR considers bcsel to produce and consume unsigned types, leading to
SEL instructions operating on unsigned types when the data is really
floating-point. Previous to this patch, saturate propagation would
happily transform
(+f0) sel g20:UD, g30:UD, g40:UD
mov.sat g50:F, g20:F
into
(+f0) sel.sat g20:UD, g30:UD, g40:UD
mov g50:F, g20:F
But since the meaning of .sat is dependent on the type of the
destination register, this is not valid.
Instead, allow saturate propagation to change the types of dest/source
on instructions that are simply copying data in order to propagate the
saturate modifier.
Fixes bad code gen in 158 programs.
Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-10-14 10:23:25 +01:00
|
|
|
(scan_inst->dst.type != inst->dst.type &&
|
|
|
|
!scan_inst->can_change_types()))
|
2015-01-28 06:43:28 +00:00
|
|
|
break;
|
|
|
|
|
2014-06-29 02:00:27 +01:00
|
|
|
if (scan_inst->saturate) {
|
2013-12-12 08:30:16 +00:00
|
|
|
inst->saturate = false;
|
|
|
|
progress = true;
|
2016-03-07 03:03:56 +00:00
|
|
|
} else if (src_end_ip == ip || inst->dst.equals(inst->src[0])) {
|
2014-06-29 02:00:27 +01:00
|
|
|
if (scan_inst->can_do_saturate()) {
|
i965/fs: Consider type mismatches in saturate propagation.
NIR considers bcsel to produce and consume unsigned types, leading to
SEL instructions operating on unsigned types when the data is really
floating-point. Previous to this patch, saturate propagation would
happily transform
(+f0) sel g20:UD, g30:UD, g40:UD
mov.sat g50:F, g20:F
into
(+f0) sel.sat g20:UD, g30:UD, g40:UD
mov g50:F, g20:F
But since the meaning of .sat is dependent on the type of the
destination register, this is not valid.
Instead, allow saturate propagation to change the types of dest/source
on instructions that are simply copying data in order to propagate the
saturate modifier.
Fixes bad code gen in 158 programs.
Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-10-14 10:23:25 +01:00
|
|
|
if (scan_inst->dst.type != inst->dst.type) {
|
|
|
|
scan_inst->dst.type = inst->dst.type;
|
|
|
|
for (int i = 0; i < scan_inst->sources; i++) {
|
|
|
|
scan_inst->src[i].type = inst->dst.type;
|
|
|
|
}
|
|
|
|
}
|
2015-01-28 06:46:22 +00:00
|
|
|
|
|
|
|
if (inst->src[0].negate) {
|
|
|
|
if (scan_inst->opcode == BRW_OPCODE_MUL) {
|
|
|
|
scan_inst->src[0].negate = !scan_inst->src[0].negate;
|
|
|
|
inst->src[0].negate = false;
|
2015-02-11 01:45:28 +00:00
|
|
|
} else if (scan_inst->opcode == BRW_OPCODE_MAD) {
|
2017-11-20 22:24:57 +00:00
|
|
|
for (int i = 0; i < 2; i++) {
|
|
|
|
if (scan_inst->src[i].file == IMM) {
|
|
|
|
brw_negate_immediate(scan_inst->src[i].type,
|
|
|
|
&scan_inst->src[i].as_brw_reg());
|
|
|
|
} else {
|
|
|
|
scan_inst->src[i].negate = !scan_inst->src[i].negate;
|
|
|
|
}
|
|
|
|
}
|
2015-02-11 01:45:28 +00:00
|
|
|
inst->src[0].negate = false;
|
2015-01-29 22:17:20 +00:00
|
|
|
} else if (scan_inst->opcode == BRW_OPCODE_ADD) {
|
|
|
|
if (scan_inst->src[1].file == IMM) {
|
|
|
|
if (!brw_negate_immediate(scan_inst->src[1].type,
|
|
|
|
&scan_inst->src[1].as_brw_reg())) {
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
scan_inst->src[1].negate = !scan_inst->src[1].negate;
|
|
|
|
}
|
|
|
|
scan_inst->src[0].negate = !scan_inst->src[0].negate;
|
|
|
|
inst->src[0].negate = false;
|
2015-01-28 06:46:22 +00:00
|
|
|
} else {
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2014-06-29 02:00:27 +01:00
|
|
|
scan_inst->saturate = true;
|
|
|
|
inst->saturate = false;
|
|
|
|
progress = true;
|
|
|
|
}
|
2013-12-12 08:30:16 +00:00
|
|
|
}
|
|
|
|
break;
|
|
|
|
}
|
2014-03-17 17:39:43 +00:00
|
|
|
for (int i = 0; i < scan_inst->sources; i++) {
|
2015-10-27 00:09:25 +00:00
|
|
|
if (scan_inst->src[i].file == VGRF &&
|
2015-10-26 11:35:14 +00:00
|
|
|
scan_inst->src[i].nr == inst->src[0].nr &&
|
2016-09-01 20:42:20 +01:00
|
|
|
scan_inst->src[i].offset / REG_SIZE ==
|
|
|
|
inst->src[0].offset / REG_SIZE) {
|
2015-02-11 00:25:47 +00:00
|
|
|
if (scan_inst->opcode != BRW_OPCODE_MOV ||
|
|
|
|
!scan_inst->saturate ||
|
|
|
|
scan_inst->src[0].abs ||
|
2015-01-28 06:46:22 +00:00
|
|
|
scan_inst->src[0].negate ||
|
|
|
|
scan_inst->src[0].abs != inst->src[0].abs ||
|
|
|
|
scan_inst->src[0].negate != inst->src[0].negate) {
|
2015-02-11 00:25:47 +00:00
|
|
|
interfered = true;
|
|
|
|
break;
|
|
|
|
}
|
2013-12-12 08:30:16 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if (interfered)
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return progress;
|
|
|
|
}
|
|
|
|
|
|
|
|
bool
|
|
|
|
fs_visitor::opt_saturate_propagation()
|
|
|
|
{
|
|
|
|
bool progress = false;
|
|
|
|
|
2014-07-12 04:54:52 +01:00
|
|
|
calculate_live_intervals();
|
2013-12-12 08:30:16 +00:00
|
|
|
|
2014-07-12 06:31:39 +01:00
|
|
|
foreach_block (block, cfg) {
|
intel/compiler: split is_partial_write() into two variants
This function is used in two different scenarios that for 32-bit
instructions are the same, but for 16-bit instructions are not.
One scenario is that in which we are working at a SIMD8 register
level and we need to know if a register is fully defined or written.
This is useful, for example, in the context of liveness analysis or
register allocation, where we work with units of registers.
The other scenario is that in which we want to know if an instruction
is writing a full scalar component or just some subset of it. This is
useful, for example, in the context of some optimization passes
like copy propagation.
For 32-bit instructions (or larger), a SIMD8 dispatch will always write
at least a full SIMD8 register (32B) if the write is not partial. The
function is_partial_write() checks this to determine if we have a partial
write. However, when we deal with 16-bit instructions, that logic disables
some optimizations that should be safe. For example, a SIMD8 16-bit MOV will
only update half of a SIMD register, but it is still a complete write of the
variable for a SIMD8 dispatch, so we should not prevent copy propagation in
this scenario because we don't write all 32 bytes in the SIMD register
or because the write starts at offset 16B (wehere we pack components Y or
W of 16-bit vectors).
This is a problem for SIMD8 executions (VS, TCS, TES, GS) of 16-bit
instructions, which lose a number of optimizations because of this, most
important of which is copy-propagation.
This patch splits is_partial_write() into is_partial_reg_write(), which
represents the current is_partial_write(), useful for things like
liveness analysis, and is_partial_var_write(), which considers
the dispatch size to check if we are writing a full variable (rather
than a full register) to decide if the write is partial or not, which
is what we really want in many optimization passes.
Then the patch goes on and rewrites all uses of is_partial_write() to use
one or the other version. Specifically, we use is_partial_var_write()
in the following places: copy propagation, cmod propagation, common
subexpression elimination, saturate propagation and sel peephole.
Notice that the semantics of is_partial_var_write() exactly match the
current implementation of is_partial_write() for anything that is
32-bit or larger, so no changes are expected for 32-bit instructions.
Tested against ~5000 tests involving 16-bit instructions in CTS produced
the following changes in instruction counts:
Patched | Master | % |
================================================
SIMD8 | 621,900 | 706,721 | -12.00% |
================================================
SIMD16 | 93,252 | 93,252 | 0.00% |
================================================
As expected, the change only affects SIMD8 dispatches.
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2018-07-10 08:52:46 +01:00
|
|
|
progress = opt_saturate_propagation_local(this, block, dispatch_width) || progress;
|
2013-12-12 08:30:16 +00:00
|
|
|
}
|
|
|
|
|
2014-09-08 20:09:44 +01:00
|
|
|
/* Live intervals are still valid. */
|
2013-12-12 08:30:16 +00:00
|
|
|
|
|
|
|
return progress;
|
|
|
|
}
|