WebGLRenderer: Support more than 8 morph targets.

[Mugen87]

Related issue: Fixed #21636.

Description

This PR adds support for using more than 8 morph targets/blend shapes.

The implementation utilizes DataTexture2DArray to encode each morph target into a separate layer of a texture array. This is more scalable than using just a single large texture however it requires WebGL 2. If no WebGL 2 is available, the previous code path is used.

[Mugen87]

Right now the new texture based implementation is always used with a WebGL2 context, Since at least on one system a problem with texture arrays was detected, it's probably more safe to:

a) only use the new code path when a geometry holds more than eight morph targets.
b) by using a flag for enabling the new code path.

I favor option a) so far.

[Mugen87]

I favor option a) so far.

Actually that is not a good idea. While implementing it I've realized that not the number of the geometry's morph targets is important but the number of active influences. However, it's not possible to compute this number just once. You have to do this per frame and thus switch between attributes and data textures if necessary.

To simplify things it's probably better to use a flag for enabling the new code path or use the data texture approach by default with WebGL 2.

[ManishJu]

If the cost is minimal why not always use the data textures. This is what babylon.js is doing. Any disadvantage of doing so ?

[Mugen87]

Any disadvantage of doing so ?

This approach did not work on at least one system (iMac (Retina 5K, 27'', ultimo 2014)). I'm afraid users are going to complain if they have the same issue and can't opt-out the data texture approach.

On the other hand this might be an issue only one a single system. It's hard to tell. I vote for a safe approach: Using it by default but allowing to disable the code path with a flag.

[ManishJu]

Using it by default but allowing to disable the code path with a flag.

makes sense

[Mugen87]

Updated the PR with the flag approach. If the new parameter disableMorphTargetTexture is set to true, the previous code path will be used.

[mrdoob]

@Mugen87 Do you get artifacts with babylon.js too? 🤔

[Mugen87]

No, it works fine there 😮 . It seems babylon.js generates the texture array differently. The sampling in the shader is also different.

Notice that the initial commit of the PR did not show any artifacts. They were introduced with this commit a9f5cdc which avoided POT textures. I don't know yet why a NPOT texture breaks the sampling on my iMac.

[Mugen87]

Using the babylon.js approach for computing the texture size fixed the bug on my iMac 🎉 .

For testing: https://raw.githack.com/Mugen87/three.js/4f7e0e9afc0689c6c28832c9c12095fbe4369e6e/examples/webgl_animation_skinning_morph.html

[WestLangley]

If the new parameter disableMorphTargetTexture is set to true, the previous code path will be used.

IMO, such user-set parameters are not appropriate. It is not the user's responsibility to specify -- or understand -- such implementation details.

Given your success, I assume that parameter can be removed. Right?

[Mugen87]

Yes, I wanted to suggest that after testing on some more devices.

I'll update the PR soon with something that can be merged.

[mrdoob]

Love the new screenshot for webgl_loader_mmd! 😁

[takahirox]

Love the new screenshot for webgl_loader_mmd! 😁

omg

[Mugen87]

Not sure what's happening to webgl_loader_mmd. When I locally render just the first frame when the loading process has been finished, everything looks fine. Investigating...

[Mugen87]

Puppeteer has a smaller max texture size than my desktop and notebook. That revealed an error with the MMD asset. After fixing the texture sampling, things should work now as expected.

Sorry for disintegrating Miku 😇 .

[Mugen87]

I guess this PR is good to go now!

[yaroslavnikiforov]

I tested these updates with the model which has a lot of shapes and it doesn't work. Only 8 shapes can be applied at the same time on one mesh as it was before. Do I need to pass some additional parameters?
isWebGL2 is true on my device.

[Mugen87]

Do you mind sharing the model in this thread?

You can also test with the following example that more than eight morph targets can be applied now: https://threejs.org/examples/webgl_loader_mmd_pose. Just use the example code with the engine changes from the PR.

Do I need to pass some additional parameters?

Nope. The new code is used by default assuming a WebGL 2 rendering context is available.

[yaroslavnikiforov]

Yes, it works! So what I was doing wrong is installing three.js as npm package likes this: git://github.com/Mugen87/three.js.git#dev3. But it is okay if using build/three.module.js from your branch. Thanks!

[Mugen87]

@yaroslavnikiforov Thanks for testing!

[borislavski]

Hi @Mugen87! This works only with webgl2 right? Is there a way to enable the new code path with webgl1 as well?

[Mugen87]

This works only with webgl2 right?

Correct!

Is there a way to enable the new code path with webgl1 as well?

No. The code relies on WebGL 2.

[mrdoob]

In order to mae this work in WebGL 1 we would need to use a long DataTexture and do the "paging" in the shader, right?

If so, do you know how big the texture would need to be? I'm on a Pixelbook Go right now and the max texture size is 16384.

[Mugen87]

gl_VertexID is not available in WebGL 1. So in order to find out the vertex index which is currently processed by the shader, we would need an additional attribute. Which also means a separate code path.

Besides, WebGL 1 does not have sampler2DArray. So for each morph target, we would need to allocate a separate texture unit which is a limited resource.

Considering these limitations, I think it's better to just support WebGL 2.

[mrdoob]

Sounds good! 👍

[mrdoob]

Thanks!

[Francois-Laberge-Bose]

Awesome work guys! Is someone going to add a demo showing of this new capabilities?

[Mugen87]

The Miku asset in https://threejs.org/examples/webgl_loader_mmd_pose already allows to configure more than 8 active morph targets (which is good for testing).

However I was hoping to find a new asset with a complex facial animation clip but with no luck so far.

[looeee]

Here's a model with an animation that has ~50 active morph targets, created using the iPhone Face Cap app. You can see it in action here (this is animated on the CPU).

iphone_face_cap.zip

I reached out to the creator of the app, Niels Jansson, and he says it's fine to use the model as long as it's credited.

[Mugen87]

I like this model especially since you can clearly see that it is not correctly animated with r132. It looks as expected with the new system.

@looeee Are you interested in making a new example with this asset? Maybe webgl_morphtargets_face?

[looeee]

Yeah sure, I'll do it tomorrow.

[mrdoob]

The md2 example seems to be glitchy now. @Mugen87 any ideas?

Screen.Recording.2021-09-22.at.10.15.03.PM.mov

[Mugen87]

Found the issue. The alpha values were not set correctly for the RGBA texture. I'll file a PR 👍 .

[0b5vr]

I've created a new issue related to this issue. tldr it sometimes uses gigantic amount of memory which have not seen in r132
#23095