Material: Add alpha to coverage define, add clipping & alpha test anti aliasing

[gkjohnson]

Related issue: --

Description

This PR is a draft to add a #define for ALPHA_TO_COVERAGE to materials when material.alphaToCoverage = true and updates materials clipping planes to take advantage of the option to antialias clipped edges. I don't expect that the alphaToCoverage field will be toggled frequently so hopefully adding a new define in this case is okay. The define can eventually be used for other applications like smoothing aliased edges when alphaTest is true. If the feature is agreeable I can go ahead and make the rest of the needed changes.

A couple things to be aware of:

• clipping with alpha to coverage = true requires a few more shader operations per clip plane.
• clipping with alpha to coverage = true requires the derivatives extension (which shouldn't be a problem in WebGL2).
• transparent objects will still have an aliased clip edge. This could be addressed but would require a new define for transparent = true objects which is probably not worth it. Or we could just use the new clipping method code always which will work even if alpha to coverage = false but be slightly more expensive.
• all materials will have to be adjusted so the diffuseColor definition comes before the the clipping planes shader chunk.

Some updated demos with an "alphaToCoverage" GUI toggle:

webgl_clipping

webgl_clipping_intersection

webgl_clipping_stencil

And some images:

Before	After

[WestLangley]

A very informative post

https://bgolus.medium.com/anti-aliased-alpha-test-the-esoteric-alpha-to-coverage-8b177335ae4f

[gkjohnson]

The define can eventually be used for other applications like smoothing aliased edges when alphaTest is true.

Here's an example of how the define can be used to smooth alpha test materials using textures with smooth gradients. The alphaToCoverage setting will already help smooth out alpha test textures with a stark alpha contrast but as mentioned in the article @WestLangley posted it can still cause issues with mip maps and linear texture interpolation:

Settings	Screenshot
transparent = true
alphaTest = 0.5
alphaTest = 0.5, alphaToCoverage = true (dev branch)
alphaTest = 0.5, alphaToCoverage = true (with A2C define)

And here's a zoomed in shot of the difference in edge quality (left is dev, right is new alphaToCoverage + alphaTest):

[gkjohnson]

@mrdoob is this interesting to you? I'm happy to finish out adding support for clip plane edges and alpha test textures if so.

This Oculus VR guide recommends using alpha to coverage for alpha cutout materials. And here's an article (cached version linked because the main site has a cookie pop up that's difficult to navigate) that discusses Street Fighter 4 using alpha to coverage for fences and leaves:

Alpha to Coverage
It may be a little difficult to spot but if your video card is compatible, alpha to coverage is applied. Objects that use Alpha test will look different when MSAA is in use. This is mainly applied to objects such as fences and leaves.

[gkjohnson]

I think this should be ready, now. It includes the ability to use alpha to coverage to smooth edges when using material clipping planes and alpha test for alpha maps. I've updated a couple examples, as well, to show its capabilities.

You can see the differences here:

[mrdoob]

Very nice! Is there any case in which we wouldn't want to have this always enabled?

[gkjohnson]

Very nice! Is there any case in which we wouldn't want to have this always enabled?

Unfortunately when it's enabled for partially transparent objects you get this kind of banding where there should be smooth gradients since the opacity is mapped to one of the 4 (8?) MSAA coverage levels.

See this image from the table above:

[wmcmurray]

Very nice! Is there any case in which we wouldn't want to have this always enabled?

Also when a user want to display very sharply pixelated transparent texture (low resolution + alphaTest + NearestFilter), I suppose this feature would interfere with the ability to do that ? 🤔

[mrdoob]

Unfortunately when it's enabled for partially transparent objects you get this kind of banding where there should be smooth gradients since the opacity is mapped to one of the 4 (8?) MSAA coverage levels.

How about when transparent === false and alphaTest > 0 && alphaTest < 1?

[gkjohnson]

How about when transparent === false and alphaTest > 0 && alphaTest < 1?

With this PR Material.alphaToCoverage now enables the the SAMPLE_ALPHA_TO_COVERAGE gl parameter and adds a #define to the shaders so shader features like clipping and alphaTest can take advantage of it. AlphaToCoverage isn't only relevant in those cases, though. Users writing custom shaders may want to enable A2C without using alphatest.

And it can actually still be worthwhile to enable A2C when using an alpha map and not use alpha test. Some of the articles I referenced in this comment discuss how A2C can be used to alleviate some of the apparent volume loss when using alpha test with texture mipmaps, though there may need to be other changes needed to make this "just work".

That's all to say that A2C has a variety of uses beyond just the ones added here so I don't think this should be an automatically toggled.

[mrdoob]

Yes, sorry. I was basically reconsidering material.alphaToCoverage.

1. Does the dev/user needs to be aware of this or can we just take care of it for them?
2. Is that the best name for it? Could it be something like material.alphaTestSmooth = true|false?
3. Is this WebGL only or does it apply to WebGPU too? Does the name alphaToCoverage makes sense in WebGPU?

Mainly looking for API suggestions that make this more intuitive for new devs.

[gkjohnson]

Does the dev/user needs to be aware of this or can we just take care of it for them?

I don't know. I don't think we can but perhaps I'm just overly detail oriented. I actually removed the "alpha to coverage" toggle for the webgl_clipping_stencil example because it actually caused more artifacts at the seams of the clip planes no matter how I applied the alpha at the edges. It either often created a gap at the seam between the edges or created a doubling of edges where the planes intersected. I feel like there are cases where you'd want to disable this.

Is that the best name for it? Could it be something like material.alphaTestSmooth = true|false?

I hesitate to use names that obscure what's actually happening. "Alpha To Coverage" is a searchable term that gives information on what's actually happening at the GPU level and I think that's a good thing. Something like "alphaTestSmooth" also implies this only works in a single use case -- it doesn't. End users can write shaders with the alpha to coverage behavior in mind to get smooth edges with procedural fragment shaders and has benefits even when alphaTest isn't bein used.

Is this WebGL only or does it apply to WebGPU too? Does the name alphaToCoverage makes sense in WebGPU?

This API feature is referred to as "alpha to coverage" in Vulkan, DirectX, and WebGPU.

[gkjohnson]

@mrdoob I see it hasn't been added to any milestone -- is this blocked because it can't be implicitly enabled?

This PR just enhances the existing Material.alphaToCoverage flag and doesn't add any new APIs so if we wanted to reevaluate how alphaToCoverage is toggled I think that could happen in a future PR.

[alberto-taiuti-skydio]

I'm curious, what's the reason for moving the call to the clipping code below the calculation for the diffuse color, in many of the shaders?

[gkjohnson]

The clipping fragment code block now multiples into the diffuse color alpha to perform the alpha test AA.

[alberto-taiuti-skydio]

Ah, makes sense. Thanks for explaning

[gkjohnson]

@Mugen87 @mrdoob is there any way to see the output screenshot from CI to compare with the uploaded screenshots to see what the differences might be?

[Mugen87]

I remember a tool that did show the diff as red/pink pixels or something similar. However, I don't know anymore how it was used and whether it was ever integrated in the CI.

[LeviPesin]

However, I don't know anymore how it was used and whether it was ever integrated in the CI.

It is an essential part of the E2E test -- the difference screenshots are autogenerated, so you can just download the artefact from the run and see them there.

[gkjohnson]

It is an essential part of the E2E test -- the difference screenshots are autogenerated, so you can just download the artefact from the run and see them there.

Thanks - finding the artifacts in the Github UI is non trivial.

---

I've reverted example changes since it's possible there are small differences in how the CI and my local machine are performing A2C sampling:

Expected	Actual	Diff

You can see they're both anti-aliased at the stipe edgues but still look different.

The last issue is with webgl_nodes_materials_instance_uniform which I'm confused by beacause I don't think node materials touches anything in this PR? cc @sunag, any ideas? It looks like the env map is being sampled in sampled in screenspace for some reason in the "after" screenshot:

before	after

[sunag]

The last issue is with webgl_nodes_materials_instance_uniform which I'm confused by beacause I don't think node materials touches anything in this PR? cc @sunag, any ideas? It looks like the env map is being sampled in sampled in screenspace for some reason in the "after" screenshot:

WebGLNodeBuilder still works with simple code replacement, so the order of the compiled nodes with those of the replaced code must be the same, I pushed a small commit to correct this.

[gkjohnson]

@sunag thank you!

Well this is ready to merge, again, if we could.