Animate glTF files

[daose]

Currently glTF only loads the 3d model with no animations. Need to put animations in so we can generate a more interesting sprite sheet.

Likely steps are:

1. read animation data (contains the number of key frames and animation data)
2. walk the node tree (instead of reading all primitives in one go)
3. apply the local transforms to each node & calculate the new positions/normals
4. form our geometry meshes out of that
5. repeat for the next keyframe

Resources:
https://raw.githubusercontent.com/KhronosGroup/glTF/master/specification/2.0/figures/gltfOverview-2.0.0b.png
https://github.com/KhronosGroup/glTF-Tutorials/blob/master/gltfTutorial/gltfTutorial_006_SimpleAnimation.md

[sunjay]

How will we handle armatures/skeletons? Will we have to calculate the mesh deformations ourselves? Most non-trivial animations aren't done with just simple transformations. We will have to put in the work to implement this kind of stuff even though it is much harder. (maybe as part of another issue?)

[daose]

Yeah this will only handle simple animations, seems like we will have to handle this stuff ourselves. The material rendering isn't finished either, but one step at a time!

[sunjay]

Some instructions for the person who ends up working on this:

Mentoring Instructions

1. Add code to the frame and end_frame methods in GltfFile to generate the transformed meshes for the given animation frame
   • The idea is that all the animation support will be built in here
   • Even rigging/skeleton support will be computed as part of this function
   • We may want to cache calculated meshes to avoid recomputing things all the time, but that can always be a separate issue--no need to prematurely optimize

spritec/src/loaders/gltf.rs

Lines 43 to 64 in 9df6e0b

	/// Return the particular frame of the given animation
	///
	/// The `animation` parameter is the name of the animation to select. Can be omitted if there
	/// is only a single animation or if there is no animation.
	///
	/// The `frame` parameter is the specific animation frame to render. The default is to render
	/// the first frame (or the loaded pose of the model if there is no animation)
	pub fn frame(&self, animation: Option<&str>, frame: Option<usize>) -> Model {
	//TODO: Use the `animation` and `frame` parameter when we support glTF animations. This
	// will probably involve refactoring this struct considerably (the interface should stay
	// stay the same though).
	self.model()
	}
	/// Returns the frame index of the last frame in the given animation. If animation is None,
	/// the file must have only a single animation.
	pub fn end_frame(&self, animation: Option<&str>) -> usize {
	//TODO: Use the `animation` parameter when we support glTF animations.
	0
	}

2. Test by adjusting the configuration file for bigboi

spritec/samples/bigboi/spritec.toml

Lines 1 to 20 in 9df6e0b

	# Configuration file for generating images with spritec
	#
	# Paths in this file are relative to the location of this configuration file
	[[spritesheets]]
	path = "../../spritesheet0.png"
	scale = 2
	[[spritesheets.animations]]
	frames = { gltf = "gltf/bigboi_rigged.gltf", start_frame = 0, end_frame = 3 }
	frame_width = 64
	frame_height = 64
	camera = "PerspectiveFront"
	outline = { thickness = 0.15 }
	[[spritesheets.animations]]
	frames = { gltf = "gltf/bigboi_rigged.gltf", start_frame = 4, end_frame = 7 }
	frame_width = 64
	frame_height = 64
	camera = "PerspectiveFront"

Right now the generated spritesheet just has the same pose 8 times:

When this is complete, it should look more like the other spritesheets generated from the .obj files:

3. Once spritesheets are working, make sure that adjusting the poses to use glTF rather than OBJ works too:

spritec/samples/bigboi/spritec.toml

Lines 125 to 132 in 9df6e0b

	[[poses]]
	# Or { gltf = "gltf/bigboi_rigged.gltf", frame = 0 }
	model = "obj/bigboi_rigged_000001.obj"
	path = "../../pose000001.png"
	width = 64
	height = 64
	scale = 2
	camera = "PerspectiveFront"

The actual details of how to accomplish all of this are up to you. This is just a high level overview of some possible steps. There may be more things you need to do that I haven't thought of.

Feel free to ask me any questions about this! 😄

[daose]

from #57 , next step is to walk the node hierarchy tree from a scene and get the global transform of each node instead of local

[daose]

goal we are aiming towards:

• pre-processing data from gltf-crate into our own representation of scene/nodes/animations
• each animation will output another scene with the correct global transforms of each node
• probably use arc so we're not unnecessarily copying data

[sunjay]

The renderer will need to be changed to accept a scene (probably the scene root node + lights). We will then traverse the scene hierarchy and accumulate the transforms before performing the drawing process.

Once @daose has the code for getting a scene root from a glTF file, I will work on refactoring our renderer to do the above steps.

While I am doing that, Bill and @JLSJamesShi can work on the second bullet point and figure out how to output a scene with all of the animations applied. As mentioned, we're going to use Arc and attempt to share as much data as possible so we aren't copying all the time.

By the end of this work we should only load from the glTF file once during construction and no longer need any handles to the glTF file in our scene graph.

[JLSJamesShi]

Next steps:

1. Load animation data from glTF and store it in the glTF backend in a way that allows for lookup based on name and time.
2. Add functionality such that if we are given a time, the entire hierarchy is updated based on that time.
3. Apply the geometry filter and load the results to the GPU.
4. Figure out caching to avoid unnecessary geometry processing and uploading.