Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for skinning >4 bones per vertex with a bone weight texture #26222

Open
wants to merge 29 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
34d7408
Add support for skinning >4 bones per vertex with a bone weight texture
cstegel May 26, 2023
fc03f04
Fix examples; unused and uninitialized variables
cstegel Jun 9, 2023
fb59366
Add screenshots for 2 new skinning examples
cstegel Jun 9, 2023
94b7331
Fix access to potentially null member in SkinnedMesh
cstegel Jun 9, 2023
9183fd0
Increase e2e-test render timeout and parse time to avoid flaky tests
cstegel Jun 9, 2023
b000985
Add unit tests for new (Interleaved)BufferAttribute methods
cstegel Jun 9, 2023
461adde
Add env map and tone mapping for new >4 bone skinning examples
cstegel Jun 13, 2023
6aab6e7
Add artist attribution for the 16 per-vertex bone-skinning model
cstegel Jun 13, 2023
0de2851
Add bone weight texture support to all SkinnedMesh loaders
cstegel Jun 13, 2023
d9c777d
Add CC4 license attribution for new head model
cstegel Jun 13, 2023
4aebe19
Add directional light back into >4 bone skinning example for better v…
cstegel Jun 13, 2023
6d8e7f1
Rename "many bone influences" example to "weight-texture"
cstegel Jun 13, 2023
6b15533
Update docs for bone weight texture skinning
cstegel Jun 14, 2023
d47fa40
Add unit tests for SkinnedMesh creation
cstegel Jun 14, 2023
aa98ea6
Add unit tests for new BufferGeometry methods
cstegel Jun 14, 2023
86d53a3
Remove comment in shader chunk
cstegel Jun 17, 2023
5d46ce1
Fix tab/space issue in shader code
cstegel Jun 17, 2023
f4cfced
Remove flaky webgl_animation_skinning_performance from screenshot tests
cstegel Jun 20, 2023
aa34feb
Empty commit to re-trigger CI because unit tests didn't run
cstegel Jun 20, 2023
7721604
Merge branch 'dev' into more-bones
cstegel Jul 26, 2023
4ff7701
Restore old visuals after useLegacyLights default changed in dev
cstegel Jul 26, 2023
50a6da9
Update svg_sandbox screenshot because e2e test is flaky
cstegel Jul 26, 2023
d66eab4
Merge branch 'dev' into more-bones
cstegel Aug 7, 2023
b5e206e
Merge branch 'dev' into more-bones
cstegel Sep 11, 2023
be0b73d
Merge branch 'dev' into more-bones
cstegel Oct 17, 2023
fb8a84e
Merge branch 'dev' into more-bones
mrdoob Nov 9, 2023
8ca2e48
Merge branch 'dev' into more-bones
cstegel Jan 22, 2024
d3aa1b4
Merge branch 'dev' into more-bones
cstegel Apr 3, 2024
607c88a
Convert spaces to tabs
cstegel Apr 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions docs/api/en/constants/Loaders.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<base href="../../../" />
<script src="page.js"></script>
<link type="text/css" rel="stylesheet" href="page.css" />
</head>
<body>
<h1>Loader Constants</h1>

<h2>Skinning Modes</h2>

Controls whether or not a SkinnedMesh will create and use a bone index/weights texture instead of a vertex buffer.

<code>
THREE.BoneIndexWeightsTextureNever
THREE.BoneIndexWeightsTextureAlways
THREE.BoneIndexWeightsTextureAllow
</code>
<p>
[page:constant BoneIndexWeightsTextureNever] Never uses a bone index/weights texture. A single vertex buffer -- up to 4 weights per-vertex -- will always be used. If a SkinnedMesh has more than 4 weights per-vertex, the 4 highest-weighted bones will be used for each vertex.<br />
[page:constant BoneIndexWeightsTextureAlways] Always use a bone index/weights texture instead of a vertex buffer. Any number of vertex weights is supported per-vertex.<br />
[page:constant BoneIndexWeightsTextureAllow] (Default) Use a bone index/weights texture instead of a vertex buffer if a mesh has more than 4 weights per-vertex.<br />
</p>

<h2>Source</h2>

<p>
[link:https://github.com/mrdoob/three.js/blob/master/src/constants.js src/constants.js]
</p>
</body>
</html>
3 changes: 2 additions & 1 deletion docs/api/en/objects/SkinnedMesh.html
Original file line number Diff line number Diff line change
Expand Up @@ -83,12 +83,13 @@ <h2>Code Example</h2>

<h2>Constructor</h2>
<h3>
[name]( [param:BufferGeometry geometry], [param:Material material] )
[name]( [param:BufferGeometry geometry], [param:Material material], [param:Constant useBoneIndexWeightsTexture] )
</h3>
<p>
[page:BufferGeometry geometry] - an instance of [page:BufferGeometry].<br />
[page:Material material] - (optional) an instance of [page:Material].
Default is a new [page:MeshBasicMaterial].
[page:Constant useBoneIndexWeightsTexture] — (optional) Controls whether or not a bone weights texture will be created and used instead of a vertex buffer for skinning. Defaults to [page:Loaders THREE.BoneIndexWeightsTextureAllow].
</p>

<h2>Properties</h2>
Expand Down
6 changes: 5 additions & 1 deletion docs/examples/en/loaders/GLTFLoader.html
Original file line number Diff line number Diff line change
Expand Up @@ -163,9 +163,13 @@ <h2>Custom extensions</h2>

<h2>Constructor</h2>

<h3>[name]( [param:LoadingManager manager] )</h3>
<h3>[name]( [param:LoadingManager manager], [param:Object options] )</h3>
<p>
[page:LoadingManager manager] — The [page:LoadingManager loadingManager] for the loader to use. Default is [page:LoadingManager THREE.DefaultLoadingManager].
[page:Object options] — (optional) Change loading behavior. Properties:<br/>
<ul>
<li>useBoneIndexWeightsTexture — (optional) Controls whether or not a [page:SkinnedMesh SkinnedMesh] will create and use a bone weights texture instead of a vertex buffer. Defaults to [page:Loaders THREE.BoneIndexWeightsTextureAllow].</li>
</ul>
</p>
<p>
Creates a new [name].
Expand Down
6 changes: 5 additions & 1 deletion docs/examples/en/loaders/MMDLoader.html
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,13 @@ <h2>Examples</h2>

<h2>Constructor</h2>

<h3>[name]( [param:LoadingManager manager] )</h3>
<h3>[name]( [param:LoadingManager manager], [param:Object options] )</h3>
<p>
[page:LoadingManager manager] — The [page:LoadingManager loadingManager] for the loader to use. Default is [page:LoadingManager THREE.DefaultLoadingManager].
[page:Object options] — (optional) Change loading behavior. Properties:<br/>
<ul>
<li>useBoneIndexWeightsTexture — (optional) Controls whether or not a [page:SkinnedMesh SkinnedMesh] will create and use a bone weights texture instead of a vertex buffer. Defaults to [page:Loaders THREE.BoneIndexWeightsTextureAllow].</li>
</ul>
</p>
<p>
Creates a new [name].
Expand Down
1 change: 1 addition & 0 deletions docs/list.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"Core": "api/en/constants/Core",
"CustomBlendingEquation": "api/en/constants/CustomBlendingEquations",
"BufferAttributeUsage": "api/en/constants/BufferAttributeUsage",
"Loaders": "api/en/constants/Loaders",
"Materials": "api/en/constants/Materials",
"Renderer": "api/en/constants/Renderer",
"Textures": "api/en/constants/Textures"
Expand Down
2 changes: 2 additions & 0 deletions examples/files.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
"webgl_animation_skinning_blending",
"webgl_animation_skinning_additive_blending",
"webgl_animation_skinning_ik",
"webgl_animation_skinning_weight-texture",
"webgl_animation_skinning_morph",
"webgl_animation_skinning_performance",
"webgl_animation_multiple",
"webgl_camera",
"webgl_camera_array",
Expand Down
14 changes: 12 additions & 2 deletions examples/jsm/loaders/ColladaLoader.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,13 @@ import { TGALoader } from '../loaders/TGALoader.js';

class ColladaLoader extends Loader {

constructor( manager, options = {} ) {

super( manager );
this.options = options;

}

load( url, onLoad, onProgress, onError ) {

const scope = this;
Expand Down Expand Up @@ -80,6 +87,8 @@ class ColladaLoader extends Loader {

parse( text, path ) {

const options = this.options;

function getElementsByTagName( xml, name ) {

// Non recursive xml.getElementsByTagName() ...
Expand Down Expand Up @@ -3626,7 +3635,6 @@ class ColladaLoader extends Loader {
if ( object.isSkinnedMesh ) {

object.bind( skeleton, controller.skin.bindMatrix );
object.normalizeSkinWeights();

}

Expand Down Expand Up @@ -3814,7 +3822,9 @@ class ColladaLoader extends Loader {
case 'polylist':
if ( skinning ) {

object = new SkinnedMesh( geometry.data, material );
object = new SkinnedMesh( geometry.data, material, {
useBoneIndexWeightsTexture: options.useBoneIndexWeightsTexture
} );

} else {

Expand Down
13 changes: 8 additions & 5 deletions examples/jsm/loaders/FBXLoader.js
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,10 @@ let sceneGraph;

class FBXLoader extends Loader {

constructor( manager ) {
constructor( manager, options = {} ) {

super( manager );
this.options = options;

}

Expand Down Expand Up @@ -142,7 +143,7 @@ class FBXLoader extends Loader {

const textureLoader = new TextureLoader( this.manager ).setPath( this.resourcePath || path ).setCrossOrigin( this.crossOrigin );

return new FBXTreeParser( textureLoader, this.manager ).parse( fbxTree );
return new FBXTreeParser( textureLoader, this.manager, this.options ).parse( fbxTree );

}

Expand All @@ -151,10 +152,11 @@ class FBXLoader extends Loader {
// Parse the FBXTree object returned by the BinaryParser or TextParser and return a Group
class FBXTreeParser {

constructor( textureLoader, manager ) {
constructor( textureLoader, manager, options = {} ) {

this.textureLoader = textureLoader;
this.manager = manager;
this.options = options;

}

Expand Down Expand Up @@ -1310,8 +1312,9 @@ class FBXTreeParser {

if ( geometry.FBX_Deformer ) {

model = new SkinnedMesh( geometry, material );
model.normalizeSkinWeights();
model = new SkinnedMesh( geometry, material, {
useBoneIndexWeightsTexture: this.options.useBoneIndexWeightsTexture
} );

} else {

Expand Down
50 changes: 35 additions & 15 deletions examples/jsm/loaders/GLTFLoader.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import {
AnimationClip,
Bone,
BoneIndexWeightsTextureAllow,
Box3,
BufferAttribute,
BufferGeometry,
Expand Down Expand Up @@ -69,10 +70,13 @@ import { toTrianglesDrawMode } from '../utils/BufferGeometryUtils.js';

class GLTFLoader extends Loader {

constructor( manager ) {
constructor( manager, options = {} ) {

super( manager );

this.useBoneIndexWeightsTexture =
options.useBoneIndexWeightsTexture ?? BoneIndexWeightsTextureAllow;

this.dracoLoader = null;
this.ktx2Loader = null;
this.meshoptDecoder = null;
Expand Down Expand Up @@ -371,7 +375,8 @@ class GLTFLoader extends Loader {
requestHeader: this.requestHeader,
manager: this.manager,
ktx2Loader: this.ktx2Loader,
meshoptDecoder: this.meshoptDecoder
meshoptDecoder: this.meshoptDecoder,
useBoneIndexWeightsTexture: this.useBoneIndexWeightsTexture

} );

Expand Down Expand Up @@ -1952,15 +1957,15 @@ class GLTFDracoMeshCompressionExtension {

for ( const attributeName in gltfAttributeMap ) {

const threeAttributeName = ATTRIBUTES[ attributeName ] || attributeName.toLowerCase();
const threeAttributeName = getThreeAttributeName( attributeName );

threeAttributeMap[ threeAttributeName ] = gltfAttributeMap[ attributeName ];

}

for ( const attributeName in primitive.attributes ) {

const threeAttributeName = ATTRIBUTES[ attributeName ] || attributeName.toLowerCase();
const threeAttributeName = getThreeAttributeName( attributeName );

if ( gltfAttributeMap[ attributeName ] !== undefined ) {

Expand Down Expand Up @@ -2239,8 +2244,6 @@ const ATTRIBUTES = {
TEXCOORD_2: 'uv2',
TEXCOORD_3: 'uv3',
COLOR_0: 'color',
WEIGHTS_0: 'skinWeight',
JOINTS_0: 'skinIndex',
};

const PATH_PROPERTIES = {
Expand All @@ -2263,6 +2266,28 @@ const ALPHA_MODES = {
BLEND: 'BLEND'
};

function getThreeAttributeName( gltfAttributeName ) {

if ( ATTRIBUTES[ gltfAttributeName ] !== undefined) {

return ATTRIBUTES[ gltfAttributeName ];

} else if ( gltfAttributeName.startsWith( 'JOINTS_' ) ) {

const index = gltfAttributeName.substring(7);
return 'skinIndex' + ( index === '0' ? '' : index );

} else if ( gltfAttributeName.startsWith( 'WEIGHTS_' ) ) {

const index = gltfAttributeName.substring(8);
return 'skinWeight' + ( index === '0' ? '' : index );

}

return gltfAttributeName.toLowerCase();

}

/**
* Specification: https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/README.md#default-material
*/
Expand Down Expand Up @@ -3788,16 +3813,11 @@ class GLTFParser {

// .isSkinnedMesh isn't in glTF spec. See ._markDefs()
mesh = meshDef.isSkinnedMesh === true
? new SkinnedMesh( geometry, material )
? new SkinnedMesh( geometry, material, {
useBoneIndexWeightsTexture: parser.options.useBoneIndexWeightsTexture
} )
: new Mesh( geometry, material );

if ( mesh.isSkinnedMesh === true ) {

// normalize skin weights to fix malformed assets (see #15319)
mesh.normalizeSkinWeights();

}

if ( primitive.mode === WEBGL_CONSTANTS.TRIANGLE_STRIP ) {

mesh.geometry = toTrianglesDrawMode( mesh.geometry, TriangleStripDrawMode );
Expand Down Expand Up @@ -4678,7 +4698,7 @@ function addPrimitiveAttributes( geometry, primitiveDef, parser ) {

for ( const gltfAttributeName in attributes ) {

const threeAttributeName = ATTRIBUTES[ gltfAttributeName ] || gltfAttributeName.toLowerCase();
const threeAttributeName = getThreeAttributeName( gltfAttributeName );

// Skip attributes already provided by e.g. Draco extension.
if ( threeAttributeName in geometry.attributes ) continue;
Expand Down
11 changes: 7 additions & 4 deletions examples/jsm/loaders/MMDLoader.js
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,14 @@ import { MMDParser } from '../libs/mmdparser.module.js';
*/
class MMDLoader extends Loader {

constructor( manager ) {
constructor( manager, options = {} ) {

super( manager );

this.loader = new FileLoader( this.manager );

this.parser = null; // lazy generation
this.meshBuilder = new MeshBuilder( this.manager );
this.meshBuilder = new MeshBuilder( this.manager, options );
this.animationBuilder = new AnimationBuilder();

}
Expand Down Expand Up @@ -414,11 +414,12 @@ const NON_ALPHA_CHANNEL_FORMATS = [
*/
class MeshBuilder {

constructor( manager ) {
constructor( manager, options = {} ) {

this.crossOrigin = 'anonymous';
this.geometryBuilder = new GeometryBuilder();
this.materialBuilder = new MaterialBuilder( manager );
this.options = options;

}

Expand Down Expand Up @@ -448,7 +449,9 @@ class MeshBuilder {
.setResourcePath( resourcePath )
.build( data, geometry, onProgress, onError );

const mesh = new SkinnedMesh( geometry, material );
const mesh = new SkinnedMesh( geometry, material, {
useBoneIndexWeightsTexture: this.options.useBoneIndexWeightsTexture
} );

const skeleton = new Skeleton( initBones( mesh ) );
mesh.bind( skeleton );
Expand Down
Binary file added examples/models/gltf/HeadWithMax16Joints.glb
Binary file not shown.
Binary file modified examples/screenshots/svg_sandbox.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading