Define declarativelly an imported mesh

[mariosanchez]

Hi!

I'm trying to use useLoader() to load a .glb file.

My goal is to be hable to load a mesh, and declare the diferent parts of the imported mesh as different meshes in order to redeclare them in a declarative way and then be hable to change it's materials and props with a state management system or component state:

Something like:

function MyComponent(props) {
  const url = '/file.glb';
  const [loaded, { meshes }] = useLoader(url);

  useMemo(() => {
    if (loaded) {
      return (
        <mesh name="parent">
          <mesh name="part1" source={meshes[1]}>
            <standardMaterial
              name="progress-mat"
              diffuseColor={props.part1Color}
            />
          </mesh>
          <mesh name="part2" source={meshes[2]}>
            <standardMaterial
              name="progress-mat"
              diffuseColor={props.part2Color}
            />
          </mesh>
          [...]
        </mesh>
      );
    }

    return null;
  }, [loaded]);

  return null;
}

The problem I'm facing is that I think useLoader appends the imported mesh to the scene immediatelly because I have not been hable to load a single part of the imported mesh or maybe I'm doing something wrong.

[brianzinn]

You want to declaratively have your meshes show up from your model and attach materials.
What you are trying to do (with perhaps some small changes in the package) could be done with:

<model ...>
  <standardMaterial key={'mesh0mat'} assignTo={'meshes[0].material'} diffuseColor={props.part1Color} />
  <standardMaterial key={'mesh1mat'} assignTo={'meshes[1].material'} diffuseColor={props.part2Color} />
</model>

What you are trying to do does not work currently the source is not a property to assign an existing mesh (but we could add that capability) - we had some earlier discussions with creating instanced meshes. It may be better to load the model inside an AssetContainer and then instantiate to a mesh? Otherwise as you have noticed it will display immediately... Let me know which way you would like to proceed and we can work together to get that as part of react-babylonjs :)

[mariosanchez]

Ey @brianzinn !

Sorry for the late response.

I would like to try the solution you are suggesting.

I'm a newbie in WebGL and Babylon to be honest, but I've been reading about the AssetContainer, and it sounds like it could be a solution. I will be glad to help in any way you think I might. I suppose I could do some sort of test with storybook by forking the library so we can check what you are proposing, but not sure about the tradeoffs. What would you choose?

To give you some context, I'm in some sort of spike to develope an application that allows you to visualize and modify a custom 3D model. I've been playing with three.js and react-three-fiber also, but I think babylon would solve better some features that I want to use and babylone can give me out of the box (glow effects for example). So I would like to turn in an off effects and change colors in a mesh/model.

One of the thinks I really liked of react-spring is that they have this sort of script that converts a gltf to a authomatic generated group of meshes in react-three-fiber https://github.com/react-spring/gltfjsx It was a really good dev experience, since I could just change the materials in a declarative way very easy, as I described in my first message.

[mariosanchez]

Ey @brianzinn,

I figured out how to achive what I was looking for.

I did an implementation of a loder hook using the AssetContainer, but maybe is not te best implementation to include it as a new asset in the library.

Would you want to share it with you an discuss on how can we do it properly?

Also, really interested in the fromObject mesh prop you mention here #66 (comment) so if I might help in any way, just tell me.

Cheers!

[brianzinn]

I think a fromObject would solve a bunch of current issues - I know how to add that and it’s not a big change. If you could share your code or do a PR then we can get that into the library. I’ll add the fromObject in the next few days.

[brianzinn]

hi @mariosanchez - I got the part in the reconciler done, so you can assign meshes and declaratively attach materials/textures.

const MyMesh = (props) => {
  const [mesh, setMesh] = useState(null);
  const scene = useBabylonScene();
  useMemo(() => {
    console.log('creating a box with scene', scene);
    setMesh(MeshBuilder.CreateBox('test', { size: 1}, scene));
  }, []);
  
  return (
    <>
      {mesh &&
        <mesh fromInstance={mesh} rotation={props.rotation}>
          <standardMaterial name='boxmat' diffuseColor={Color3.Blue()} specularColor={Color3.Black()} />
        </mesh>
      }
    </>
  )
}

That is on a branch called from-instance. You can try it out by using link locally. If you want to share your loader hook then maybe we can combine them to a working example with a .glb file. Also, would be cool to get React Suspense connected with a fallback loader...

[mariosanchez]

hi @brianzinn, I've trying the fromInstance prop and it works lovely so far. To put an example of how the code changed:

Before

import React, { useEffect, useRef } from 'react';

function Part({ partMesh, material }) {
  const partName = useRef(null);
  partMesh.setEnabled(true);

  useEffect(() => {
    partMesh.setParent(partName.current.hostInstance);
  }, [partName]);

  partMesh.material = material;

  return <transformNode name={`part_${partMesh.name}`} ref={partName} />;
}

export default Part;

After

import React from 'react';

function Part({ partMesh, material }) {
  partMesh.setEnabled(true);
  partMesh.material = material;

  return <mesh fromInstance={partMesh} />;
}

export default Part;

[mariosanchez]

About the solution I got for not immediatelly display the loaded meshes in the scene I got some inspiration in that post in the babylon forum https://forum.babylonjs.com/t/load-mesh-without-adding-to-scene/606/12 and the useLoader of the library. I think it's far from being a generic solution, and maybe other aproach could be better, but I will exemplify it.

I have this useAssetManager hook (didn't think much about the naming tbh), which hides the loaded meshes:

import { useEffect, useState } from 'react';
import '@babylonjs/loaders';
import { useBabylonScene } from 'react-babylonjs';
import { AssetsManager } from '@babylonjs/core';

export function useAssetManager(rootUrl, meshNames, sceneFilename) {
  const scene = useBabylonScene();

  const [loaded, setLoaded] = useState(false);
  const [result, setResult] = useState([]);

  const assetsManager = new AssetsManager(scene);

  useEffect(() => {
    loadMesh(assetsManager, rootUrl, meshNames, sceneFilename)
      .then(({ loadedMeshes }) => {
        setResult(
          loadedMeshes.map((mesh) => {
            mesh.setEnabled(false);
            return mesh;
          })
        );
        setLoaded(true);
      })
      .catch((error) => console.error(error));

    assetsManager.load();
  }, [rootUrl, sceneFilename]);

  return [loaded, result];
}

function loadMesh(assetsManager, rootUrl, meshNames, sceneFilename) {
  return new Promise((resolve, reject) => {
    const meshTask = assetsManager.addMeshTask(
      `load ${rootUrl}`,
      meshNames,
      rootUrl,
      sceneFilename
    );
    meshTask.onSuccess = function onMeshTaskSuccess(task) {
      resolve(task);
    };
    meshTask.onError = function onMesshTaskError(_, message, exception) {
      reject(exception ?? message);
    };
  });
}

Then I use it like this:

function Component() {
  const url = '/gltf/aMode.glb';
  const [loaded, meshes] = useAssetManager(url, [ /* a mesh names list */ ]);

  return useMemo(() => {
    if (loaded) {
      const parts = meshes.reduce(
        (partialParts, mesh) => ({ ...partialParts, [mesh.id]: mesh }),
        {}
      );

      return (
        <transformNode name="parent">
          <Part
            partMesh={parts['aPartName']}
            material={aMaterial}
          />
          <Part
            partMesh={parts['aPartName']}
            material={aMaterial}
          />
// ...
        </transformNode>
      );
    }

    return null;
  }, [loaded, state.parts]);
}

Using it like this you are not getting all the features of a AssetManager, I'm just using it this way to hide the mesh when it's loeaded.

Maybe ther is some sort of modification we could do in the existing loader to work the same way with an option parameter to hide or display the meshes? The problem I found with the original loader is that it duplicate the meshes it returns, the reference is no the same soy you end with two meshes in the scene that are independent, and that was not what I wanted.

Also, the AssetManager displays a loading screen by default that you probably just want to disable and put yours.

I feel like probably we want to define a proper interface for this feature before get a step further. What are yout thoughs?

[brianzinn]

I added a hook useAssetManager. It only supports binary tasks right now and it's not out yet on NPM, but I want to extend it and support your scenario. Just a start really - brings up a lot of questions. There is also a storybook:
https://brianzinn.github.io/react-babylonjs/?path=/story/hooks--use-asset-manager

[mariosanchez]

Hi @brianzinn

Thanks for your work! It's for sure a good start. And so cool you could add suspense support to it.

One doubt I have, is that you commented that useMemo is failing. Is it acctually throwing an error or it's just it has no effect because of you are throwing a promise?

Are you planning to ship this to npm together with the fromInstance ?

[brianzinn]

I did ship an NPM with fromInstance hopefully it worked (2.2.8). It's using a github action that I just made :)

You are correct that useMemo fails just when the error or promise is thrown - it does work properly when the promise resolves. At least that is what I discovered in my development experience (and with an endless loop before I added caching). I have a new branch that reports progress, so the technical stuff is out of the way, so I am now going to do a big cleanup. I want to have a hook also for SceneLoader - it actually has a better "progress" report generally (especially for big files). I may be integrating my first dependency - there is some discussion in issue 87 that will be about that.

I put it on a branch - going to try to get some time to clean that all up. Does that have all the functionality that you would need? You should be able to disable meshes as you are above. I added support for MeshTasks in this branch and a storybook example for progress:
https://github.com/brianzinn/react-babylonjs/compare/use-asset-manager-v2

[mariosanchez]

Yes it does. I will check the branch anyway, but keep going.

We could close the issue if you are okay with this :)

Thanks again!

[brianzinn]

Thanks for confirming it works! I'll leave open and will let you know when next version is ready - probably v3.0 as it will have breaking changes with switch over to Suspense. I like how you made a map of mesh ids (parts), I am thinking to add something like that to final API for task lookup by name.

[brianzinn]

hi @mariosanchez - going to close as 3.0 is out, but have a look at how I created the <Model .../> component:
https://github.com/brianzinn/react-babylonjs/blob/master/src/customComponents/Model.tsx

The hook returns a meshMap you can use to set props as you originally described. You need on that hook. Feel free to re-open if you have any more comments/questions. Thanks for your help getting this across the finish line (and patience)!!