mesa: Fix uf10_to_f32() scale factor in the E == 0 and M != 0 case.

GL_EXT_packed_float, 2.1.B Unsigned 10-Bit Floating-Point Numbers:

        0.0,                      if E == 0 and M == 0,
        2^-14 * (M / 32),         if E == 0 and M != 0,
        2^(E-15) * (1 + M/32),    if 0 < E < 31,
        INF,                      if E == 31 and M == 0, or
        NaN,                      if E == 31 and M != 0,

In the second case (E == 0 and M != 0), we were multiplying the mantissa
by 2^-20, when we should have been multiplying by 2^-19 (which is
2^(-14 + -5), or 2^-14 * 2^-5, or 2^-14 / 32).

The previous section defines the formula for 11-bit numbers, which is:

        2^-14 * (M / 64),         if E == 0 and M != 0,

In other words, we had accidentally copy and pasted the 11-bit code
to the 10-bit case, and neglected to change the exponent.

Fixes dEQP-GLES3.functional.pbo.renderbuffer.r11f_g11f_b10f_triangles
when run with surface dimensions of 1536x1152 or 1920x1080.

Cc: mesa-stable@lists.freedesktop.org
References: https://code.google.com/p/chrome-os-partner/issues/detail?id=56244
Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Stephane Marchesin <stephane.marchesin@gmail.com>
Reviewed-by: Antia Puentes <apuentes@igalia.com>
This commit is contained in:
Kenneth Graunke 2016-08-15 22:26:42 -07:00
parent 0ff57446e3
commit 01e99cba04
1 changed files with 1 additions and 1 deletions

View File

@ -191,7 +191,7 @@ static inline float uf10_to_f32(uint16_t val)
if (exponent == 0) {
if (mantissa != 0) {
const float scale = 1.0 / (1 << 20);
const float scale = 1.0 / (1 << 19);
f32.f = scale * mantissa;
}
} else if (exponent == 31) {