4e7a777093
aco: try forming clauses even if reg_pressure exceeds
...
This patch allows to form clauses even if the register pressure
is at the limit with the effect that VMEM instructions are less
scattered after the first clause in a Block.
It respects the previous clause size to avoid excessive moving
of VMEM instructions.
VMEM_CLAUSE_MAX_GRAB_DIST is further reduced to compensate
some of the effects.
Totals from 28922 (19.26% of 150170) affected shaders: (GFX10.3)
VGPRs: 1546568 -> 1523072 (-1.52%); split: -1.52%, +0.00%
CodeSize: 117524892 -> 117510288 (-0.01%); split: -0.08%, +0.07%
MaxWaves: 605554 -> 611120 (+0.92%)
Instrs: 22292568 -> 22291927 (-0.00%); split: -0.10%, +0.09%
Latency: 488975399 -> 490230904 (+0.26%); split: -0.06%, +0.32%
InvThroughput: 117842300 -> 116521653 (-1.12%); split: -1.15%, +0.03%
VClause: 541550 -> 522464 (-3.52%); split: -9.73%, +6.20%
SClause: 718185 -> 718298 (+0.02%); split: -0.00%, +0.02%
Copies: 1420603 -> 1386949 (-2.37%); split: -2.64%, +0.27%
Branches: 559559 -> 559278 (-0.05%); split: -0.06%, +0.01%
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10896 >
2021-09-27 14:29:16 +00:00
7e1faf9349
aco: make clause-forming depend on the number of moved instructions
...
This allows more aggressive clause-forming in presence of
larger def-use distances. To compensate for the effect,
VMEM_CLAUSE_MAX_GRAB_DIST was decreased.
Totals from 5788 (3.85% of 150170) affected shaders: (GFX10.3)
VGPRs: 483960 -> 475272 (-1.80%); split: -1.82%, +0.02%
CodeSize: 59661240 -> 59669084 (+0.01%); split: -0.01%, +0.02%
MaxWaves: 70408 -> 71450 (+1.48%); split: +1.51%, -0.03%
Instrs: 11222417 -> 11224479 (+0.02%); split: -0.01%, +0.03%
Latency: 349397104 -> 349298602 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 88584832 -> 87762262 (-0.93%); split: -0.93%, +0.00%
VClause: 168905 -> 177089 (+4.85%); split: -0.48%, +5.32%
SClause: 375795 -> 375767 (-0.01%); split: -0.01%, +0.01%
Copies: 840298 -> 840231 (-0.01%); split: -0.04%, +0.03%
Branches: 373265 -> 373278 (+0.00%); split: -0.00%, +0.00%
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10896 >
2021-09-27 14:29:16 +00:00
903999c119
aco: stop scheduling if clause-forming fails
...
This avoids unintended reordering of VMEM instructions.
It is also highly unlikely that we find more independent
instructions before previous clause-related instructions.
Totals from 1921 (1.28% of 150170) affected shaders: (GFX10.3)
VGPRs: 103832 -> 103736 (-0.09%); split: -0.10%, +0.01%
CodeSize: 8695560 -> 8706000 (+0.12%); split: -0.03%, +0.15%
Instrs: 1643752 -> 1646349 (+0.16%); split: -0.04%, +0.20%
Latency: 26755527 -> 26614645 (-0.53%); split: -0.67%, +0.14%
InvThroughput: 7226604 -> 7204809 (-0.30%); split: -0.39%, +0.08%
VClause: 46536 -> 46201 (-0.72%); split: -0.81%, +0.09%
SClause: 47910 -> 47769 (-0.29%); split: -0.43%, +0.14%
Copies: 94647 -> 94558 (-0.09%); split: -0.26%, +0.17%
Branches: 36843 -> 36847 (+0.01%); split: -0.00%, +0.01%
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10896 >
2021-09-27 14:29:16 +00:00
b23a9dd1f6
aco/scheduler: allow moving down VMEM stores to below VMEM loads
...
fossil-db (Vega10):
Totals from 93 (0.06% of 150305) affected shaders:
SGPRs: 4832 -> 4768 (-1.32%)
VGPRs: 4084 -> 4144 (+1.47%)
CodeSize: 316080 -> 317208 (+0.36%); split: -0.11%, +0.47%
MaxWaves: 589 -> 580 (-1.53%)
Instrs: 60229 -> 60511 (+0.47%); split: -0.15%, +0.61%
Latency: 636477 -> 540029 (-15.15%); split: -15.26%, +0.10%
InvThroughput: 293027 -> 283043 (-3.41%); split: -4.21%, +0.80%
VClause: 2557 -> 2716 (+6.22%); split: -0.35%, +6.57%
SClause: 1381 -> 1395 (+1.01%); split: -0.14%, +1.16%
Copies: 9424 -> 9728 (+3.23%); split: -0.74%, +3.97%
fossil-db (Sienna Cichlid):
Totals from 88 (0.06% of 150170) affected shaders:
VGPRs: 3840 -> 3872 (+0.83%)
CodeSize: 300544 -> 300960 (+0.14%); split: -0.09%, +0.23%
Instrs: 53714 -> 53871 (+0.29%); split: -0.05%, +0.35%
Latency: 489854 -> 462001 (-5.69%); split: -6.30%, +0.61%
InvThroughput: 100307 -> 95142 (-5.15%); split: -5.50%, +0.35%
VClause: 2322 -> 2564 (+10.42%); split: -0.39%, +10.81%
SClause: 1345 -> 1358 (+0.97%); split: -0.15%, +1.12%
Copies: 4113 -> 4351 (+5.79%); split: -0.66%, +6.44%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12211 >
2021-08-23 16:48:31 +00:00
8341af5109
radv, aco, ac/nir: Tweak position export scheduling for NGG culling.
...
The result is about +5-ish fps in Doom Eternal.
It turns out that the location of position exports matters more
than we thought, and it's actually better to keep them at the bottom
for culling shaders rather than schedule it up to the top.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525 >
2021-07-13 23:56:33 +00:00
1e2639026f
aco: Format.
...
Manually adjusted some comments for more intuitive line breaks.
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11258 >
2021-07-12 21:27:31 +00:00
0eea0e55ad
aco: add 'common/' and 'llvm/' prefix to #includes
...
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11271 >
2021-07-12 12:09:31 +00:00
59fdaa1985
aco: reorder and cleanup #includes
...
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11271 >
2021-07-12 12:09:31 +00:00
d64f5a3f9d
aco: move VMEM instructions below descriptor loads
...
This is to prevent sequences like:
a = descriptor_load()
vmem(a)
b = descriptor_load()
vmem(b)
and instead create:
a = descriptor_load()
b = descriptor_load()
vmem(a)
vmem(b)
fossil-db (GFX10.3):
Totals from 114521 (78.30% of 146267) affected shaders:
VGPRs: 4540352 -> 4540216 (-0.00%); split: -0.03%, +0.02%
CodeSize: 289864228 -> 289114652 (-0.26%); split: -0.29%, +0.03%
MaxWaves: 2940234 -> 2940338 (+0.00%); split: +0.00%, -0.00%
Instrs: 55112418 -> 54919910 (-0.35%); split: -0.38%, +0.03%
Latency: 956528393 -> 954682011 (-0.19%); split: -0.24%, +0.05%
InvThroughput: 229280830 -> 229238107 (-0.02%); split: -0.04%, +0.02%
VClause: 1141832 -> 1139002 (-0.25%); split: -0.63%, +0.38%
SClause: 2357840 -> 2225008 (-5.63%); split: -6.01%, +0.38%
Copies: 3316040 -> 3331519 (+0.47%); split: -0.31%, +0.77%
Branches: 1187212 -> 1186919 (-0.02%); split: -0.03%, +0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6489 >
2021-06-14 15:47:37 +00:00
bc71222cd9
aco: don't move descriptor loads below buffer loads
...
fossil-db (GFX10.3):
Totals from 52870 (36.15% of 146267) affected shaders:
VGPRs: 2109936 -> 2110056 (+0.01%); split: -0.01%, +0.01%
CodeSize: 134898056 -> 134812748 (-0.06%); split: -0.08%, +0.02%
MaxWaves: 1347354 -> 1347346 (-0.00%)
Instrs: 25598063 -> 25575415 (-0.09%); split: -0.11%, +0.02%
Latency: 432491613 -> 432047723 (-0.10%); split: -0.12%, +0.02%
InvThroughput: 90940977 -> 90927545 (-0.01%); split: -0.03%, +0.01%
VClause: 570039 -> 570019 (-0.00%); split: -0.05%, +0.04%
SClause: 1145076 -> 1139040 (-0.53%); split: -0.60%, +0.07%
Copies: 1513949 -> 1513102 (-0.06%); split: -0.32%, +0.26%
Branches: 524279 -> 524275 (-0.00%); split: -0.03%, +0.03%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6489 >
2021-06-14 15:47:37 +00:00
3c390e2eb6
aco/scheduler: Move cursor handling state to dedicated interfaces
...
This clarifies the semantics of the index variables compared to the previous
version, which used the same variables in a slightly different way depending
on whether they were used for downwards moves or upwards ones.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10885 >
2021-06-07 12:09:39 +02:00
81761a311e
aco/scheduler: Clean up register demand tracking
...
Refactoring total_demand and total_demand_clause to cover non-overlapping
instruction intervals makes the code easier to follow and allows the register
demand to be updated more efficiently in some cases.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10885 >
2021-06-07 12:09:39 +02:00
80ee9d3947
aco/scheduler: Verify register demand invariants in debug mode
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10644 >
2021-05-13 15:27:57 +00:00
50ba919d37
aco/scheduler: Fix register demand computation for upwards moves
...
The initial value needs to be taken from the instruction that is being
moved over, not the one to be moved.
Additionally the parameter of this function was removed because it was
misleading. Setting it to any value other than source_idx would cause
register_demand to be initialized incorrectly. (Instead, the maximum
demand among the covered instructions would need to be determined.)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10644 >
2021-05-13 15:27:57 +00:00
c528af1076
aco/scheduler: Fix register demand computation for downwards moves
...
Previously, changes in total_demand_clause were not always propagated to
total_demand. For instance, clause moves do not change the local register
demand at the end of a clause, yet they may still affect the total maximum.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 8235bc6411https://gitlab.freedesktop.org/mesa/mesa/-/issues/4533
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10644 >
2021-05-13 15:27:57 +00:00
dfa38fa0c7
aco: group loads from the same vertex binding into the same clause
...
In the future, we might have vertex attribute loads from the same binding
but with different descriptors. Since they will be loading from the same
buffer, we should continue grouping them into clauses.
No fossil-db changes.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7871 >
2021-05-10 12:09:14 +00:00
3d4c13f3b8
aco: add DeviceInfo
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8761 >
2021-02-15 13:44:22 +00:00
947bf0bd67
aco: don't decrease the vgpr_limit when encountering bpermute
...
Instead we recalculate vgpr_limit on demand, depending on
the number of needed shared VGPRs.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8921 >
2021-02-12 19:00:18 +00:00
e115b01948
aco: return references in instruction cast methods
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595 >
2021-01-22 14:12:33 +00:00
1d245cd18b
aco: use format-check methods
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595 >
2021-01-22 14:12:32 +00:00
70dbcfa1c9
aco: use instruction cast methods
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595 >
2021-01-22 14:12:32 +00:00
489aa8c7cb
aco: fix num_waves on GFX10+
...
There are half the SIMDs per CU and physical_vgprs should be 512 instead
of 256.
fossil-db (GFX10.3):
Totals from 3622 (2.60% of 139391) affected shaders:
VGPRs: 298192 -> 289732 (-2.84%); split: -3.43%, +0.59%
CodeSize: 29443432 -> 29458388 (+0.05%); split: -0.00%, +0.06%
MaxWaves: 21703 -> 23395 (+7.80%); split: +7.84%, -0.05%
Instrs: 5677920 -> 5681438 (+0.06%); split: -0.01%, +0.07%
Cycles: 280715524 -> 280895676 (+0.06%); split: -0.00%, +0.07%
VMEM: 981142 -> 981894 (+0.08%); split: +0.18%, -0.10%
SMEM: 243315 -> 243454 (+0.06%); split: +0.07%, -0.02%
VClause: 88991 -> 89767 (+0.87%); split: -0.02%, +0.89%
SClause: 200660 -> 200659 (-0.00%); split: -0.00%, +0.00%
Copies: 430729 -> 434160 (+0.80%); split: -0.07%, +0.86%
Branches: 158004 -> 158021 (+0.01%); split: -0.01%, +0.02%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8523 >
2021-01-20 16:46:54 +00:00
4ea0ce2f55
aco: remove can_reorder semantic in get_sync_info_with_hack
...
No fossil-db changes.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8416 >
2021-01-11 16:35:19 +00:00
b1e12747b9
aco: create VMEM clauses slightly more aggressive
...
Totals from 3325 (2.39% of 139391) affected shaders (NAVI10):
SGPRs: 331528 -> 331056 (-0.14%); split: -0.14%, +0.00%
VGPRs: 306164 -> 337764 (+10.32%); split: -0.02%, +10.34%
CodeSize: 38843180 -> 38865388 (+0.06%); split: -0.04%, +0.10%
MaxWaves: 18908 -> 17028 (-9.94%); split: +0.01%, -9.95%
Instrs: 7423908 -> 7427934 (+0.05%); split: -0.06%, +0.12%
Cycles: 527411756 -> 526388408 (-0.19%); split: -0.21%, +0.02%
VMEM: 1148421 -> 992660 (-13.56%); split: +0.10%, -13.67%
SMEM: 227337 -> 232380 (+2.22%); split: +2.26%, -0.04%
VClause: 146416 -> 111171 (-24.07%); split: -24.10%, +0.03%
SClause: 243674 -> 243689 (+0.01%); split: -0.00%, +0.01%
Copies: 663496 -> 660333 (-0.48%); split: -0.85%, +0.37%
Branches: 223725 -> 223721 (-0.00%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7903 >
2020-12-22 15:08:40 +01:00
ac40301dbb
aco: schedule position exports in the same pass as memory operations
...
No fossil-db changes.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7903 >
2020-12-22 15:08:40 +01:00
0287ebeef3
aco: fix def-use distance calculation when scheduling.
...
This change also increases the VMEM_MAX_MOVES
to mitigate some of the scheduling changes.
Totals from 34301 (24.61% of 139391) affected shaders:
SGPRs: 2515440 -> 2552304 (+1.47%); split: -1.25%, +2.71%
VGPRs: 1786676 -> 1794724 (+0.45%); split: -0.31%, +0.76%
CodeSize: 151079856 -> 151209828 (+0.09%); split: -0.06%, +0.15%
MaxWaves: 392454 -> 388966 (-0.89%); split: +0.39%, -1.28%
Instrs: 28870746 -> 28895907 (+0.09%); split: -0.09%, +0.17%
Cycles: 960450680 -> 961315796 (+0.09%); split: -0.09%, +0.18%
VMEM: 19027987 -> 19796223 (+4.04%); split: +7.49%, -3.45%
SMEM: 2434691 -> 2394829 (-1.64%); split: +2.80%, -4.43%
VClause: 551776 -> 543051 (-1.58%); split: -1.73%, +0.15%
SClause: 1230147 -> 1227637 (-0.20%); split: -1.40%, +1.20%
Copies: 1957640 -> 1963617 (+0.31%); split: -1.11%, +1.41%
Branches: 611747 -> 612504 (+0.12%); split: -0.11%, +0.23%
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7903 >
2020-12-22 15:08:40 +01:00
3f14140f48
aco: allow to schedule SALU/SMEM through exec changes
...
Totals from 16794 (12.05% of 139391) affected shaders (NAVI10):
SGPRs: 757760 -> 762048 (+0.57%); split: -0.39%, +0.95%
VGPRs: 402844 -> 402744 (-0.02%); split: -0.04%, +0.02%
CodeSize: 22290900 -> 22285068 (-0.03%); split: -0.06%, +0.04%
MaxWaves: 294163 -> 294222 (+0.02%); split: +0.03%, -0.01%
Instrs: 4190074 -> 4188513 (-0.04%); split: -0.08%, +0.04%
Cycles: 40685028 -> 40678640 (-0.02%); split: -0.03%, +0.02%
VMEM: 7711867 -> 7704315 (-0.10%); split: +0.28%, -0.38%
SMEM: 942472 -> 1007052 (+6.85%); split: +7.15%, -0.30%
VClause: 92990 -> 92974 (-0.02%); split: -0.03%, +0.01%
SClause: 263700 -> 263810 (+0.04%); split: -0.38%, +0.42%
Copies: 277467 -> 276988 (-0.17%); split: -0.37%, +0.20%
Branches: 45899 -> 45896 (-0.01%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7903 >
2020-12-22 15:08:40 +01:00
731f8fc9dd
aco: Use program->num_waves as maximum in scheduler.
...
The scheduler doesn't take SGPR use into account, which can be
a limiting factor on older GPUs. This patch fixes a CTS test crash
on GFX6.
CC: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8040 >
2020-12-11 13:32:26 +00:00
34bc9477de
aco: Clean up symbol names and comments related to NGG
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7094 >
2020-10-21 09:49:38 +00:00
86c227c10c
aco: Use strong typing to model SW<->HW stage mappings
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7094 >
2020-10-21 09:49:38 +00:00
97afb2a0a9
aco: remove unused radv_shader.h includes
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7061 >
2020-10-14 15:09:34 +00:00
408195ec53
aco: remove useless occurences of radv_nir_compiler_options
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7061 >
2020-10-14 15:09:34 +00:00
7b4c24eb67
aco: don't move memory accesses to before control barriers
...
Fixes random failures of dEQP-VK.image.qualifiers.volatile.cube_array.r32i
and similar tests on Vega.
fossil-db (Navi):
Totals from 6 (0.00% of 135946) affected shaders:
VMEM: 1218 -> 1110 (-8.87%); split: +2.46%, -11.33%
SMEM: 174 -> 189 (+8.62%)
Copies: 84 -> 87 (+3.57%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: cd392a10d0https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6174 >
2020-08-11 14:16:00 +01:00
07250a92da
aco: implement subgroup shader_clock on GFX10.3
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5546 >
2020-08-04 20:39:33 +01:00
37988b5b8e
aco: fix max_waves_per_simd on Polaris, VegaM and GFX10.3
...
fossil-db (Polaris):
Totals from 20263 (14.75% of 137414) affected shaders:
SGPRs: 871407 -> 871679 (+0.03%); split: -0.00%, +0.03%
VGPRs: 513828 -> 550028 (+7.05%); split: -1.68%, +8.72%
CodeSize: 18869680 -> 18828148 (-0.22%); split: -0.23%, +0.01%
MaxWaves: 162012 -> 162030 (+0.01%); split: +0.01%, -0.00%
Instrs: 3629172 -> 3618817 (-0.29%); split: -0.30%, +0.02%
Cycles: 15682244 -> 15638244 (-0.28%); split: -0.30%, +0.02%
VMEM: 10675942 -> 10673344 (-0.02%); split: +0.18%, -0.21%
SMEM: 1209717 -> 1206088 (-0.30%); split: +0.03%, -0.33%
VClause: 81780 -> 81227 (-0.68%); split: -0.73%, +0.06%
SClause: 231724 -> 231561 (-0.07%); split: -0.07%, +0.00%
Copies: 187126 -> 180831 (-3.36%); split: -3.62%, +0.26%
Branches: 26841 -> 26837 (-0.01%); split: -0.03%, +0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5546 >
2020-08-04 20:39:33 +01:00
cd392a10d0
radv/aco,aco: use scoped barriers
...
fossil-db (Navi):
Totals from 109 (0.08% of 132058) affected shaders:
SGPRs: 5416 -> 5424 (+0.15%)
CodeSize: 460500 -> 460508 (+0.00%); split: -0.07%, +0.07%
Instrs: 87278 -> 87272 (-0.01%); split: -0.09%, +0.09%
Cycles: 2241996 -> 2241852 (-0.01%); split: -0.04%, +0.04%
VMEM: 33868 -> 35539 (+4.93%); split: +5.14%, -0.20%
SMEM: 7183 -> 7184 (+0.01%); split: +0.36%, -0.35%
VClause: 1857 -> 1882 (+1.35%)
SClause: 2052 -> 2055 (+0.15%); split: -0.05%, +0.19%
Copies: 6377 -> 6380 (+0.05%); split: -0.02%, +0.06%
PreSGPRs: 3391 -> 3392 (+0.03%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4905 >
2020-07-28 16:56:34 +00:00
d1f992f3c2
aco: rework barriers and replace can_reorder
...
fossil-db (Navi):
Totals from 273 (0.21% of 132058) affected shaders:
CodeSize: 937472 -> 936556 (-0.10%)
Instrs: 158874 -> 158648 (-0.14%)
Cycles: 13563516 -> 13562612 (-0.01%)
VMEM: 85246 -> 85244 (-0.00%)
SMEM: 21407 -> 21310 (-0.45%); split: +0.05%, -0.50%
VClause: 9321 -> 9317 (-0.04%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4905 >
2020-07-28 16:56:34 +00:00
1bbb64f300
aco: add missing add_to_hazard_query
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4905 >
2020-07-28 16:56:34 +00:00
1f2fd9c62e
aco: don't reorder barriers in the scheduler
...
Unless we're reordering it around a barrier of the same type
No shader-db changes.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4880 >
2020-05-28 10:34:03 +00:00
0c0691d43e
aco: Const correctness for get_barrier_interaction.
...
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536 >
2020-04-24 17:58:57 +00:00
e4da482d9e
aco/ngg: Schedule position exports of NGG VS/TES.
...
Similarly to the HW VS stage, the HW NGG GS stage also
benefits from executing these exports as early as possible.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576 >
2020-04-07 11:29:35 +00:00
35e58314d8
aco: Treat s_setprio as a scheduling barrier.
...
We want to execute instructions after s_setprio in the given
priority, so we must prevent the scheduler from scheduling beyond
s_setprio, otherwise some instructions could be executed in a
different priority.
Rename hazard_fail_memtime to hazard_fail_unreorderable and include
s_setprio in the list of unreorderable opcodes.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3576 >
2020-04-07 11:29:35 +00:00
c51348bd9b
aco: move some register demand helpers into aco_live_var_analysis.cpp
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3914 >
2020-03-16 16:09:02 +00:00
625d8705f0
aco: don't stop scheduling at exports
...
This allows us to move v_cvt_pkrtz_f16_f32 instructions upwards, improving
schedules and (somewhat unintentionally) moving the exports slightly
closer together.
Totals from affected shaders:
SGPRS: 1030224https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3776 >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3776 >
2020-03-13 14:04:50 +00:00
6b4c31f814
aco: allow barriers to be skipped during scheduling
...
Much better scheduling apparently in 160 shaders
Totals from affected shaders:
SGPRS: 6272 -> 6344 (1.15 %)
VGPRS: 4832 -> 4844 (0.25 %)
Spilled SGPRs: 0 -> 0 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 467192 -> 467428 (0.05 %) bytes
LDS: 459 -> 459 (0.00 %) blocks
Max Waves: 1407 -> 1409 (0.14 %)
SMEM score: 9309.00 -> 11216.00 (20.49 %)
VMEM score: 26679.00 -> 33652.00 (26.14 %)
SMEM clauses: 1817 -> 1776 (-2.26 %)
VMEM clauses: 2286 -> 2288 (0.09 %)
Instructions: 86537 -> 86596 (0.07 %)
Cycles: 676260 -> 676568 (0.05 %)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3776 >
2020-03-13 14:04:50 +00:00
928ac97875
aco: add helpers for ensuring correct ordering while scheduling
...
Pipeline-db changes in 721 shaders.
Totals from affected shaders:
SGPRS: 42336 -> 42656 (0.76 %)
VGPRS: 38368 -> 38636 (0.70 %)
Spilled SGPRs: 11967 -> 11967 (0.00 %)
Spilled VGPRs: 0 -> 0 (0.00 %)
Scratch size: 0 -> 0 (0.00 %) dwords per thread
Code Size: 5268088 -> 5269840 (0.03 %) bytes
LDS: 1069 -> 1069 (0.00 %) blocks
Max Waves: 4473 -> 4447 (-0.58 %)
SMEM score: 41155.00 -> 41826.00 (1.63 %)
VMEM score: 146339.00 -> 147471.00 (0.77 %)
SMEM clauses: 24434 -> 24535 (0.41 %)
VMEM clauses: 16637 -> 16592 (-0.27 %)
Instructions: 996037 -> 996388 (0.04 %)
Cycles: 76476112 -> 75281416 (-1.56 %)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3776 >
2020-03-13 14:04:50 +00:00
2cd760847a
aco: add helpers for moving instructions for scheduling
...
No pipeline-db changes
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3776 >
2020-03-13 14:04:50 +00:00
71440ba0f5
aco: reorder VMEM operands in ACO IR
...
For all VMEM instructions, the resource constant is now
in operands[0]. For MIMG instructions, the sampler shares
operands[1] with write data in case this instruction writes memory.
Moving the VADDR to be the last operand for MIMG is the first step to
support Navi NSA encoding.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/3602 >
2020-01-29 18:45:23 +00:00
40bb81c9dd
radv/aco,aco: implement GS on GFX9+
...
v2: implement GFX10
v3: rebase
v7: rebase after shader args MR
v8: fix gs_vtx_offset usage on GFX9/GFX10
v8: use unreachable() instead of printing intrinsic
v8: rename output_state to ge_output_state
v8: fix formatting around nir_foreach_variable()
v8: rename some helpers in the scheduler
v8: rename p_memory_barrier_all to p_memory_barrier_common
v8: fix assertion comparing ctx.stage against vertex_geometry_gs
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2421 >
2020-01-24 13:35:07 +00:00
b5c9688516
aco: limit register usage for large work groups
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2020-01-10 12:10:37 +00:00
Daniel Schürmann
Rhys Perry
Timur Kristóf
Tony Wasserka
Samuel Pitoiset