MeshVelocityMaterial: render a scene's velocity field to a buffer (useful for motion blur, TRAA post effects)

[bhouston]

Related issue: #14050

Description

This adds a MeshVelocityMaterial to Three.js in the spirit of MeshNormalMaterial and MeshDepthMaterial. This enables one to render meshes into a velocity field buffer. This is useful for a number of advanced post effects including Motion Blur and Temporal Reprojection Anti-Aliasing.

This implementation only supports a few different methods of inducing velocity in the scene. Here is what it supports and what it doesn't:

• camera movement changes - SUPPORTED.
• node movement changes - SUPPORTED.
• skinning changes - NOT SUPPORTED.
• blend shape changes - NOT SUPPORTED.
• displacement changes - NOT SUPPORTED.

I have added a "velocity" mode to the channel example to show this off.

This is what it looks like:

three.js.examples.-.Google.Chrome.2022-03-24.17-46-50.mp4

I have separated out this MeshVelocityMaterial into its own PR so that people can give feedback on this design in isolation rather than combining it with TRAA in general or a Motion Blur post effect.

This contribution is funded by Threekit.

[WestLangley]

Setting material.needsUpdate = true; is not necessary when changing material.side.

However, toggling between BasicDepthPacking and RGBADepthPacking does not update the material.

EDiT: It appears the renderer treats parameters.depthPacking as a boolean, and it is not.

[bhouston]

I believe I have cleaned up this PR so that it meets the various suggestions made in its review. Let me know what else I should do.

I would like next to contribute a TRAA method that uses this material.

[AdaRoseCannon]

There is an upcoming WebXR proposal, to allow fewer frames to be rendered, called space warp being lead by @cabanier this would be required for that to work https://twitter.com/rcabanier/status/1537210613318811648

[Mugen87]

@bhouston I've updated the PR to remove the merge conflicts and to honor the remaining feedback. Just one question: Is it normal when the rendered velocity sometimes affects only parts of the mesh? This is a screenshot from the front:

I can see this behavior before my latest commits, too.

[bhouston]

If parts of the mesh are not moving left or right, then yes, they will not have colors.

[Mugen87]

Let's merge this! Having a velocity material in the examples is really useful. When the material is going be used in more and more components (like WebXR), we can still move MeshVelocityMaterial into the core.

[cabanier]

If parts of the mesh are not moving left or right, then yes, they will not have colors.

@bhouston Since the whole mesh is moving when you rotate it, shouldn't it all light up?

[cabanier]

I think I understand now. If one of the colors/velocities goes negative, it gets clipped to black.

[bhouston]

Honestly, it all depends on how things are being visualized. If it is hard to tell if it is correct with the current visualization method, maybe we need to switch to a different one? Maybe we could draw little lines to indicate flow or something on screen?

[WestLangley]

For visualization only, something along these lines... but better:

vec2 vel = ( ndcPositionCurrent - ndcPositionPrevious ) * 0.5;

vel *= 100.0; // values are too small to see

vel = vel * 0.5 + 0.5;

gl_FragColor = vec4( vel, 1.0, 1.0 );

Tip: Allowing panning in the example is illustrative.

[cabanier]

FYI I'm using your velocity shader to implement space warp on quest and it works great.