Improve MeshPhysicalMaterial transmission support #21884

takahirox · 2021-05-24T09:32:28Z

Related issue: #16977, #21000

Description

Sorry for the late but this is a PR for improving MeshPhysicalMaterial transmission support.

The screenshot of a new glTF transmission example.

Changes

Add thickness, thicknessMap, attenuationDistance, and attenuationColor properties to MeshPhysicalMaterial.
Add webgl_loader_gltf_transmission example.
Add transmission_(pars_)fragment.glsl.js which are primarily based on glTF-Sample-Viewer shader codes. Integrate transmissionmap_(pars_)fragment.glsl.js with them.
Add transmissive objects render path to WebGLRenderer. Refer to Intent to implement/contribute Enterprise PBR improvements #16977 (comment) for the detail.

Notes

This PR doesn't include KHR_materials_ior and KHR_materials_volume extension supports in GLTFLoader. I would make a follow up PR if this change is merged.
Please feel free to make follow up PRs for cleaning up, optimization, and improvement. /cc @whatisor
Regarding the shader codes, I almost just copied from glTF-Sample-Viewer shader codes. I applied this part but haven't this part yet (because I haven't perfectly understood that part yet). Please feel free to make a follow up PR to apply that part.
I haven't finely compared with the reference screenshot yet. I would be happy if you help.

Limitations

I haven't checked yet but probably it doesn't work with XR yet. Follow up PR is welcome.

WestLangley · 2021-05-24T12:13:41Z

@takahirox Thanks for this! :-)

The previous transmission implementation -- which was just a stub -- looked like this when transmission was set to 1.

Now, it darkens when transmission is set to 1... More light should transmit, not less, no?

cx20 · 2021-05-24T15:52:47Z

I used raw.githack.com to run this sample. However, the result was not as shown in the thumbnail on the left.

https://rawcdn.githack.com/takahirox/three.js/d6df4b39033dcf42c139dcef9c700b18880774d1/examples/index.html?q=gltf#webgl_loader_gltf_transmission

The environment tested was Windows 10 + RTX 2060 + Chrome 90.

mrdoob · 2021-05-24T16:58:02Z

@cx20 raw.githack.com doesn't work in this case. I'll try to merge this ASAP.

elalish · 2021-05-24T17:36:08Z

src/renderers/WebGLRenderer.js

+	// transmission
+
+	let _transmissionRenderTarget = null;
+	const _transmissionSamplerSize = new Vector2( 1024, 1024 ); // Should be configurable?


Shouldn't this be the same as the output render dimensions? Without refraction, the transmission lines up exactly with the viewport.

elalish · 2021-05-24T17:41:54Z

src/renderers/WebGLRenderer.js

 		const transparentObjects = currentRenderList.transparent;

 		if ( opaqueObjects.length > 0 ) renderObjects( opaqueObjects, scene, camera );
+		if ( transmissiveObjects.length > 0 ) renderTransmissiveObjects( opaqueObjects, transmissiveObjects, scene, camera );


Isn't there some way we could not have to render the opaque objects twice? It seems like we could just copy the previous buffer into the tranmissionSamplerMap.

With WebGL2 we'll be able to avoid rendering twice, but with WebGL 1 we can't have antialias when rendering into a render target 😕

Well, wouldn't we still be able to take the normal anti-aliased framebuffer we just rendered the opaque objects into and copy that into the transmissionSamplerMap for the next pass? Instead of rendering it all over again? This would also remove the question around what size the samplerMap should be.

Can WebGL 1 generate mipmaps for non-power-of two textures? The current implementation uses the lower level mips for roughness transmission.

AFAIK, it can't.

Ah yes, forgot about that. Could we sample it into a power-of-two texture, instead of a pure copy?

elalish · 2021-05-24T17:47:00Z

src/renderers/shaders/ShaderChunk/transmission_fragment.glsl.js

+	vec3 f0 = vec3( pow( ior - 1.0, 2.0 ) / ( pow( ior + 1.0, 2.0 ) ) );
+	vec3 f90 = vec3( 1.0 );
+
+	vec3 f_transmission = totalTransmission * getIBLVolumeRefraction(


Wow, this isn't just tranmission support, it's also volume and ior? Awesome!

elalish · 2021-05-24T17:57:15Z

src/loaders/MaterialLoader.js

+		if ( json.transmission !== undefined ) material.transmission = json.transmission;
+		if ( json.thickness !== undefined ) material.thickness = json.thickness;
+		if ( json.attenuationDistance !== undefined ) material.attenuationDistance = json.attenuationDistance;
+		if ( json.attenuationColor !== undefined && material.attenuationColor !== undefined ) material.attenuationColor.setHex( json.attenuationColor );


Ah, I see. This can support the glTF volume and ior extensions (which are just going in now for ratification), but we haven't yet added these to GLTFLoader. That will be exciting!

takahirox · 2021-05-24T21:02:51Z

@WestLangley

Thanks for the report. Yeah, I think transmission = 1 should look more transmissive. I'll take a look...

WestLangley · 2021-05-24T21:28:15Z

@takahirox I understand your use of the glFT fidelity test case, but IMHO, I'd leave that for glTF, and instead focus on enhancing the GUI (and model, if necessary) in the current three.js transmission test case.

By being able to vary the parameters, one can see if the new shader is working correctly.

takahirox · 2021-05-26T05:15:27Z

Looking better?

mrdoob · 2021-05-26T09:55:19Z

Excellent work @takahirox!

I'm going to merge this and try using renderer.copyFramebufferToTexture() to avoid drawing the opaque list twice (thanks @elalish!)

mrdoob · 2021-05-26T09:55:28Z

Thanks!

cx20 · 2021-05-26T13:09:26Z

I tried the latest version of the three.js library and glTF Loader with gltf-test.
I think the result is very good.

Click on the screenshot below to see a sample.

model	three.js r128 + GLTF Loader result	three.js r129dev + GLTF Loader result
Transmission Test
Toy Car

WestLangley · 2021-05-26T15:43:16Z

@takahirox Can you recheck the metalness = 1 case? Pure metals are not transparent, so in that case, the value of transmission should have no effect.

mrdoob · 2021-05-26T18:34:07Z

@elalish

I tried using copyFramebufferToTexture but seems like the render target needs to be the same size as the framebuffer (which is not power of 2 and everything breaks...)

I guess we'll need WebGL 2 for that?

mrdoob · 2021-05-26T18:39:26Z

@takahirox Seems like Safari is not liking the glsl in this PR...

Any ideas?

takahirox · 2021-05-26T19:16:01Z

@mrdoob

Safari doesn't support WebGL2 and GLSL3 yet, does it? It seems I should have written texture2D and texture2DLodEXT instead of texture and textureLod in the shader for WebGL1 and GLSL1.

And we need to add the following lines if we use texture2DLodEXT.

#extension GL_EXT_shader_texture_lod : enable
#extension GL_OES_standard_derivatives : enable

https://developer.mozilla.org/en-US/docs/Web/API/EXT_shader_texture_lod

mrdoob · 2021-05-26T19:46:59Z

@takahirox Thanks! 90abd4f

takahirox · 2021-05-26T22:35:27Z

@WestLangley

@takahirox Can you recheck the metalness = 1 case? Pure metals are not transparent, so in that case, the value of transmission should have no effect.

Good catch. In the current shader, I first apply the transmissionSamplerMap to diffuseColor.

And then material.diffuseColor will be calculated from diffuseColor but it will be vec3(0.0) if metalness is 1.

The problem is diffuseColor has an effect to material.specularColor even if metalness is 1.

Perhaps material.specularColor should be calculated from the raw diffuseColor?

mrdoob · 2021-05-27T09:22:42Z

@cx20 Curious to see how these examples looks like after #21894 🤔

whatisor · 2021-05-27T09:32:13Z

@takahirox I tried to figure out how transparent Plastic material work.
For plastic material, environment reflection should be almost zero. Can the current transmission model work if environment reflection intensity is almost 0?

cx20 · 2021-05-27T12:40:32Z

@mrdoob I compared the results before and after the fix in #21894.
I think the metal band is no longer transparent.

model	before	after
Transmission Test

WestLangley · 2021-05-27T12:53:00Z

I think the metal band is no longer transparent.

Right, you can't tell. That test case is not very useful for debugging. It is too complex. What are the material settings, for example?

cx20 · 2021-05-27T13:44:47Z

@emackey Do you have any advice for us?

WestLangley · 2021-05-27T14:08:51Z

@cx20 A test case with a GUI, similar to https://threejs.org/examples/webgl_materials_physical_transmission.html, is much more useful for determining if the shader is physically plausible.

IMHO, of course. :-)

emackey · 2021-05-27T14:18:44Z

I agree this particular glTF model mixes test cases without fully indicating which is which.

In the sphere shown here, transmission is 1.0 for the entire sphere, roughness is 0.0, and the metallic channel is where the swirls are happening at full strength.

For what it's worth, @cx20's "after" image looks correct to me, because I can no longer see the checkerboard cloth through half of the swirls (those are the metallic ones). So I believe this matches the spec for transmission in glTF.

But given infinite time, yes there should be a clearer glTF test model, where a metallic transmissive sphere is compared to a metallic non-transmissive sphere, and a dozen other such tests.

WestLangley · 2021-05-27T15:05:36Z

But given infinite time, yes there should be a clearer glTF test model

We already have webgl_materials_physical_transmission.html which is being improved with a more-extensive GUI.

emackey · 2021-05-27T15:25:28Z

We already have webgl_materials_physical_transmission.html which is being improved with a more-extensive GUI.

That's great!

I'm looking at this from the Khronos side, where we (3D Formats) want to test visual consistency of glTF among a number of different rendering engines, including ThreeJS.

takahirox · 2021-05-27T19:40:40Z

@mrdoob @Mugen87

Would you mind if adding @whatisor next to us in the r129 change log for MeshPhysicalMaterial transmission improvement? It ended up I alone made the PR but I'm sure he worked in #21000, too, and helped us go forward. I want to respect his work.

mrdoob · 2021-05-27T22:14:37Z

@takahirox definitely! 👍

Mugen87 · 2021-05-28T06:35:14Z

@takahirox Updated the release notes!

WestLangley · 2021-06-10T14:02:27Z

With all due respect, I still do not think this implementation is physically plausible. A fully-transmissive, white material is not that dark or opaque. I stand by #21884 (comment).

https://threejs.org/examples/webgl_materials_physical_transmission.html

mrdoob · 2021-06-10T14:20:58Z

With all due respect, I still do not think this implementation is physically plausible. A fully-transmissive, white material is not that dark or opaque. I stand by #21884 (comment).

@whatisor any ideas?

takahirox · 2021-06-10T17:19:02Z

Trying to improve the transmission... Is this looking better?

whatisor · 2021-06-15T16:21:37Z

@takahirox I think your new tuning needs to consider Metallic factor.
Follow specification from gltf, metallic should not be affected by transmission (otherwise, it is partial plastic)

Therefore, for a material with metallicFactor=1.0, the value of transmissionFactor doesn't matter.

--https://github.com/KhronosGroup/glTF/blob/master/extensions/2.0/Khronos/KHR_materials_transmission/README.md--

takahirox · 2021-06-15T17:19:11Z

@whatisor

Thanks for the comment.

new tuning

It means #21975, doesn't it?

consider Metallic factor

Yes, it does. material.diffuseColor passed to getIBLVolumeRefraction() is diffuseColor.rgb * ( 1.0 - metalnessFactor );

The screenshot of the both metallic and transmission are 1. It is no transparent.

whatisor · 2021-06-18T03:17:45Z

oh, it seems to be from metallicRoughnessTexture , not factor itself.

"metallicFactor": 0.0, "metallicRoughnessTexture": { "index": 4 }

I remember reference result with metallic test.

takahirox · 2021-06-18T03:31:50Z

Ah, sorry the screenshots in #21884 (comment) are bit old.

This is the current screenshot.

whatisor · 2021-06-18T03:46:19Z

Super cool! I love this

takahirox added 3 commits May 23, 2021 18:54

Imptove MeshPhysicalMaterial transmission support

ed5e74a

Add webgl_loader_gltf_transmission example

afec60b

Update webgl_materials_physical_transmission screenshot

549354d

This was referenced May 24, 2021

Intent to implement/contribute Enterprise PBR improvements #16977

Closed

GLTFLoader: Support KHR_materials_transmission physically-based transparency #21000

Closed

Dispose transmissionRenderTarget in WebGLRenderer.dispose()

d6df4b3

elalish reviewed May 24, 2021

View reviewed changes

takahirox added 6 commits May 25, 2021 00:44

Remove _transmissionSamplerMap from MeshPhysicalMaterial

9a958ea

Fix transmissionMap

c27a451

Remove unnecessary sRGB-Linear conversion from transmission shader

3b9236f

Improve transmission shaders

9e23e20

Add more parameters to webgl_materials_physical_transmission example

10a93f7

Update screenshot

2b41ff8

mrdoob merged commit bbe6f1e into mrdoob:dev May 26, 2021

takahirox deleted the Transmission branch May 26, 2021 14:57

takahirox mentioned this pull request May 27, 2021

Improve metallic transmission #21894

Merged

This was referenced May 27, 2021

Editor: Added transmission support. #21896

Merged

ShaderChunk: Fixed transmission crash when using flat shading #21897

Merged

Mugen87 added this to the r129 milestone May 27, 2021

joshuaellis mentioned this pull request May 27, 2021

r129 three-types/three-ts-types#75

Closed

11 tasks

takahirox mentioned this pull request May 28, 2021

MeshPhysicalMaterial transmission doesn't work correctly in WebXR immersive mode #21911

Closed

rawwerks mentioned this pull request Jun 1, 2021

support for KHR_materials_transmission (re: Three.js R129) google/model-viewer#2445

Closed

10 tasks

takahirox mentioned this pull request Jun 11, 2021

Improve MeshPhysicalMaterial transmission support Part2 #21975

Merged

donmccurdy mentioned this pull request Jul 12, 2021

GLTFLoader: Add KHR_materials_volume and KHR_materials_ior extensions support #22117

Merged

Improve MeshPhysicalMaterial transmission support #21884

Improve MeshPhysicalMaterial transmission support #21884

Conversation

takahirox commented May 24, 2021 • edited Loading

WestLangley commented May 24, 2021

cx20 commented May 24, 2021

mrdoob commented May 24, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elalish May 24, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

takahirox commented May 24, 2021 • edited Loading

WestLangley commented May 24, 2021 • edited Loading

takahirox commented May 26, 2021

mrdoob commented May 26, 2021

mrdoob commented May 26, 2021

cx20 commented May 26, 2021

WestLangley commented May 26, 2021

mrdoob commented May 26, 2021

mrdoob commented May 26, 2021

takahirox commented May 26, 2021 • edited Loading

mrdoob commented May 26, 2021

takahirox commented May 26, 2021 • edited Loading

mrdoob commented May 27, 2021 • edited Loading

whatisor commented May 27, 2021

cx20 commented May 27, 2021 • edited Loading

WestLangley commented May 27, 2021

cx20 commented May 27, 2021

WestLangley commented May 27, 2021

emackey commented May 27, 2021

WestLangley commented May 27, 2021

emackey commented May 27, 2021

takahirox commented May 27, 2021 • edited Loading

mrdoob commented May 27, 2021

Mugen87 commented May 28, 2021

WestLangley commented Jun 10, 2021

mrdoob commented Jun 10, 2021

takahirox commented Jun 10, 2021 • edited Loading

whatisor commented Jun 15, 2021 • edited Loading

takahirox commented Jun 15, 2021

whatisor commented Jun 18, 2021

takahirox commented Jun 18, 2021

whatisor commented Jun 18, 2021

takahirox commented May 24, 2021 •

edited

Loading

elalish May 24, 2021 •

edited

Loading

takahirox commented May 24, 2021 •

edited

Loading

WestLangley commented May 24, 2021 •

edited

Loading

takahirox commented May 26, 2021 •

edited

Loading

takahirox commented May 26, 2021 •

edited

Loading

mrdoob commented May 27, 2021 •

edited

Loading

cx20 commented May 27, 2021 •

edited

Loading

takahirox commented May 27, 2021 •

edited

Loading

takahirox commented Jun 10, 2021 •

edited

Loading

whatisor commented Jun 15, 2021 •

edited

Loading