WebGLRenderer: Render transmissiveObjects in two passes.

[mrdoob]

Description

Inspired by @drcmda recent tweets, I was curious to see how it would look like if we also rendered the transmissive objects (back sided) in the transmissive pass.

Before	After

Not a fan of calling this twice:

textures.updateMultisampleRenderTarget( _transmissionRenderTarget );
textures.updateRenderTargetMipmap( _transmissionRenderTarget );

But the results are pretty good!

I'll do a couple of links to test the performance in a bit.

[mrdoob]

It's much closer to what path tracers produce:

r149	three-gpu-pathtracer

[gkjohnson]

Very supportive of this as an option 😁 With the (eventual) addition of depth peeling you should be able to get a really nice render and get rid of some artifacts like these that are missing depth:

Here's a demo I made awhile ago that uses depth peeling with a transmission-like light attenuation - this uses the depth of the back faces to derive the thickness of the volume at each pixel:

https://twitter.com/garrettkjohnson/status/1433267075867295753

Lowering layers to 1 will show the above artifacts you'll see in more complicated models. This shows the use with 1, 2, 3, 4 layers:

[mrdoob]

Another interesting comparison:

r149	This PR
live link	live link

[mrdoob]

@gkjohnson

Very supportive of this as an option 😁

I guess it depends on the performance impact... I would like it to be on by default.

[mrdoob]

Depending on the model (and thickness > 0) we can see itself as if it was a reflection though...

So maybe we should only do this when material.side === DoubleSide... 😇

[Mugen87]

Another interesting comparison:

Do you mind adding stats.js to the amber examples?

Testing them on a M2 Pro or RTX 2070 shows no difference in performance but on a Pixel 4a you can see noticeable more stuttering when moving the camera around.

Does the PR version also contain the new texture filtering approach? If so, a comparison with r149 is a bit problematic since it does not demonstrate the real performance drop caused by this particular change.

[mrdoob]

Here they are:

https://threejs-gltf-mosquito.glitch.me/index-r150a.html
https://threejs-gltf-mosquito.glitch.me/index-r150b.html

	r150dev (a)	r150dev (b)
MacBook Air M1	60	60
Pixel 7	90	65

[Mugen87]

Thanks for updating! Here are my numbers:

	r150dev (a)	r150dev (b)
Mac mini M2 Pro	60	60
Lenovo Legion RTX 2070	144	144
Pixel 4a	26	17

[Mugen87]

If we decide to go for "best possible quality" in context of the PBR implementation, we should be aware that specific materials features like transmission will be more and more problematic to use on certain low- and potentially mid-end mobile devices.

On the other side, it's very difficult to provide a single PBR implementation that looks good and performs well on any device. I suspect that even mediump support won't save the day. At some point, you probably need different shader code for desktop and mobile.

[gkjohnson]

I'm losing about 10 fps on my Pixel 3, as well -- from ~37 fps to ~27 or so.

[donmccurdy]

... it's very difficult to provide a single PBR implementation that looks good and performs well on any device

+1. Even MeshStandardMaterial (with none of these features) is often too expensive for painting the screen on a mobile device. I think Mozilla Hubs switches down to MeshBasicMaterial; Lambert would be plausible too.

I'm interested in whether we could offer a "MeshStandardLiteMaterial" (better name needed?), with some careful approximations and feature limitations for better mobile performance. But as MeshPhysicalMaterial is our highest-end material for the latest PBR features ... the quality/performance tradeoff here seems acceptable.

[mrdoob]

I changed it so this only happens when a material is doublesided.

Hopelly these two links have the same performance:

https://threejs-gltf-mosquito.glitch.me/index-r150a.html
https://threejs-gltf-mosquito.glitch.me/index-r150c.html

	r150dev (a)	r150dev (c)
MacBook Air M1	60	60
Pixel 7	90	90

[LeviPesin]

a and c render in 4 FPS on my tablet (Samsung Galaxy Tab A 10.1), b renders in 2. On my laptop (HP Pavilion, AMD A8-7410 APU) they render in 26 and 20 FPS.

[mrdoob]

Considering that the current approach (only doing this for DoubleSided materials) doesn't introduce any performance regression (all models use FrontSide) I think it's worth merging this to be able to produce these renders when appropriate.

[DmitryUlyanov]

I'm interested in whether we could offer a "MeshStandardLiteMaterial"

+1, we (and many people out there) are developing three-js based product and aiming to support older devices. There is certainly demand for quality shaders, but it would be great to have support for lighter lower quality shaders as well.

Also with the filtering update in r150 MeshStandardMaterial is now not working on iOS 14. Previously three.js gracefully fall back to WebGL1 if WebGL2 is not available, but new filtering uses WebGL2-only functions e.g. textureLod resulting in an exception. EDIT: found that is solved already #25563

[elalish]

So, now that this only happens with doubleSide, your first two images (dragon and thickness test) no longer show any change. This also makes me wonder what the meaning of doubleSide should be in the glTF spec, particularly in the case of the volumetric extension (would love your thoughts @emackey).

I'm also curious why there is such a difference between three.js and three-gpu-pathtracer for e.g. the dragon, @gkjohnson may know. I would think that an attenuation color and a thickness texture should get us closer than a factor of 2 difference in output color. Did we get our units wrong somewhere?

[emackey]

So, now that this only happens with doubleSide, your first two images (dragon and thickness test) no longer show any change. This also makes me wonder what the meaning of doubleSide should be in the glTF spec

Specified in the spec for thinknessFactor here: https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Khronos/KHR_materials_volume#properties

"The doubleSided property has no effect on volume boundaries."

The Overview section mentions:

"The mesh to which the material is attached defines the boundaries of an homogeneous medium and therefore must be manifold."

Essentially, when thicknessFactor is nonzero, we've left thin-walled mode behind, and now the glTF mesh is believed to be a closed, manifold outer shell of polygons defining a watertight boundary around a 3D volume of material. glTF's doubleSided flag has no meaning here.

[gkjohnson]

I'm also curious why there is such a difference between three.js and three-gpu-pathtracer for e.g. the dragon, @gkjohnson may know. I would think that an attenuation color and a thickness texture should get us closer than a factor of 2 difference in output color. Did we get our units wrong somewhere?

Two differences that come to mind. Currently three.js is not modeling overlap of objects using something like depth peeling - so as shown here foreground objects will appear lighter. Three.js is also, of course, not modeling changes in ray distance traversal due to refraction or internal bounces which would cause the attenuation to increase. It would be best to do a comparison with an ior of 1.0 if we want to make sure the math is correct - but without those few things I wouldn't expect the visuals with a higher ior to exactly match.

[elalish]

Yeah, I'd expect differences, but not of this magnitude. I feel like either the thickness texture on this model is in the wrong units or we're interpreting it wrong. I think it's seriously off by a factor of 2.

[emackey]

Try the AttenuationTest model, it has measurable thickness.

[emackey]

Specifically, the latest version of AttenuationTest has been updated since the screenshot shown in the original post on this PR. The current version features a "Sample Color" row with flat squares instead of the cubes shown above, and this correction makes it much easier to identify what shade of color is the correct color based on depth.

[elalish]

FYI, after talking on the PBR call, we believe the thickness texture of the dragon model is likely incorrect, so let's fix it and compare again before trying to match the path tracers any better.