INTEGRATION_depth_camera fails due to precision issue

[darksylinc]

I had INTEGRATION_depth_camera consistently fail on me in #323 so I had to take a deep look.

Note: It is also failing in the ign-rendering5 branch, I just noticed it now because due to a setup error this sample wasn't launching.

Note2: This bugs repros with both GPU HW and LIBGL_ALWAYS_SOFTWARE=1.

At first I thought this was the same bug as #327 but that was not the case (although I suspect they are the same bug, more on this later), because the failure was on the blue channel:

ign/src/ign-rendering/test/integration/depth_camera.cc:257: Failure
Expected: (mb) > (0u), actual: 0 vs 0

// This one repeats A LOT
ign/src/ign-rendering/test/integration/depth_camera.cc:449: Failure
Expected: (b) > (0u), actual: 0 vs 0

After an hour of debugging I found the most horrendously bizarre problem.

The short version:

1. Colours are correctly packed by packFloat in ogre2/src/media/materials/programs/depth_camera_fs.glsl
2. The RGBA value is 0 0 143 255 which corresponds to floating point value: 8.21616e-40 stored in alpha
3. Another pass (ogre2/src/media/materials/programs/depth_camera_final_fs.glsl) copies the calculated alpha value 'as is'. However after that copy, the resulting value is 0.0f

This is the RenderDoc capture of the first rendered frame:

BadRepro.rdc.zip

If we go to pixel[135][255].a we'll see correctly the pattern 0 0 143 255 (Texture '81' in our capture, byteswapped in the picture)

But when we go to the results of the copy (stored in Texture '83' in our capture), we see the values are gone:

Environment

• OS Version: Ubuntu 18.04 LTS (with mainline PPA kernel 5.11)
• Source or binary build?
  Source, branch ign-gazebo5 (diagnostic was made in my PostFrame branch though, since it's easier to capture RenderDoc there)
• If this is a GUI or sensor rendering bug, describe your GPU and rendering system. Otherwise delete this section.
  • Rendering plugin: ogre2
    • running on real hardware
  • Rendering system info:

LANG=C glxinfo -B | grep -i '\(direct rendering\|opengl\|profile\)'  # might require installing mesa-utils package
direct rendering: Yes
    Preferred profile: core (0x1)
    Max core profile version: 4.6
    Max compat profile version: 4.6
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL vendor string: X.Org
OpenGL renderer string: Radeon RX 560 Series (POLARIS11, DRM 3.40.0, 5.11.0-051100-generic, LLVM 10.0.1)
OpenGL core profile version string: 4.6 (Core Profile) Mesa 20.1.3 (git-663fa46287)
OpenGL core profile shading language version string: 4.60
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL version string: 4.6 (Compatibility Profile) Mesa 20.1.3 (git-663fa46287)
OpenGL shading language version string: 4.60
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 20.1.3 (git-663fa46287)
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

ps aux | grep Xorg
root      1774  2.6  0.7 2973636 255116 tty7   Ssl+ 12:26   6:08 /usr/lib/xorg/Xorg -core :0 -seat seat0 -auth /var/run/lightdm/root/:0 -nolisten tcp vt7 -novtswitch
matias   10920  0.0  0.0  17988  1100 pts/5    S+   16:19   0:00 grep --color=auto Xorg


sudo env LANG=C X -version

X.Org X Server 1.20.8
X Protocol Version 11, Revision 0
Build Operating System: Linux 4.15.0-140-generic x86_64 Ubuntu
Current Operating System: Linux matias-ubuntu 5.11.0-051100-generic #202102142330 SMP Sun Feb 14 23:33:21 UTC 2021 x86_64
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.11.0-051100-generic root=UUID=98733a50-8c43-4a07-bceb-ed2cc319135b ro quiet splash amdgpu.dc=0 amdgpu.gttsize=8192 acpi_osi=Linux vt.handoff=1
Build Date: 08 April 2021  01:40:27PM
xorg-server-hwe-18.04 2:1.20.8-2ubuntu2.2~18.04.5 (For technical support please see http://www.ubuntu.com/support) 
Current version of pixman: 0.34.0
	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.

Description

• Expected behavior: Unit test should pass
• Actual behavior: Unit test fails

Steps to reproduce

1. Just run ign/build/ignition-rendering5/bin/INTEGRATION_depth_camera
2. I'm using Mesa 20.1.3 (git-663fa46287)

Notes

Is this a driver bug?

• Could be, yes. But I find it strange that LIBGL_ALWAYS_SOFTWARE fails on me too. Perhaps the problem has to do with interpolation during sampling. I checked it and samplers according to RenderDoc is set to "Point" "Mipmaps: None"
• But I've ran into similar problems in the past. For example Ogre ran into a D3D11 bug where an uint32 mask was stored in a float, but it would fail when the mask resulted in a NaN. Why? Because after inspecting the assembly the code looked something like this:

mov freg0, [mem] // loads NaN into reg0
mul freg0, 1.0f // multiplies freg0 against 1.0f, destroying the NaN's bits
mov ireg0, freg0 // moves from floating point register to integer register, we have garbage now

This was a compiler codegen error in fxc (D3D11's compiler), If the float is a valid value, this spurious multiplication was harmless. But if the float was a NaN, all the binary information was lost and when we reinterpreted the float back to uint32, the relevant data was lost. Bottom line, manipulating uints in floats is always risky in terms of GPU

Are other samples/unit tests affected?

Unknown.

Is this the real cause of #327 instead of RGB vs RGBA?

Unknown.

Why are the CI server tests unaffected?

Unknown. Could be simply due to different Mesa driver versions.
Or maybe the CI servers aren't properly launching the ogre2 test (e.g. failure to load libOgreMain.so.2.1.0 will count as a pass!)

Is this only a unit test bug?

No. This bug could be affecting whatever result where the alpha channel is too close to 0 (or possibly could affect other binary patterns such as NaN).

Solutions

I'm still researching this problem. One possibility could be to use texelFetch instead of texture() to bypass filtering completely. But it's still risky.

The sure solution would be to reinterpret the texture as RGBA32_UINT; sample it as uint4; then reinterpret the rgb to float, and copy the alpha as is.

IIRC Ogre 2.1 did not support texture reinterpretation, it was implemented in 2.2

Update:
Replacing vec4 p = texture(inputTexture, inPs.uv0); with vec4 p = texelFetch(inputTexture, ivec2( inPs.uv0 * 256.0 ), 0 ); did solve the problem at least for me (I hardcoded the resolution to 256 for a quick test)