[discussion and proposal]: InstancesMesh

[hookex]

@brianzinn

Hi, brianzinn. I'm busy with my works these days. But we can start to design InstancesMesh API.

Why use InstancesMesh

• Better Performace. This API can draw large amounts of meshes of the same geometry by using only one GPU API call.
  (Have you ever compared the performance of the MergedMesh and InstancesMesh?)

Features

• pass Instances data to one component. (InstancesData's length maybe change when update)
• property: position、scale、color、alpha
• events: click、[hover]. (The number of event registrations must be small)

API suggestion

// component name
<Instances/>

interface InstanceData {
    position: [number, number, number],
    scaling?: [number, number, number],
    color?: [number, number, number],
    alpha?: number,
}

interface InstancesProps {
    mesh: Babylon.Mesh | Ref<Babylon.Mesh>,
    data: InstanceData[],
    onClick: Function;
}

[brianzinn]

Hi Hooke!

I really like this idea. I was thinking an instance would work something like this:

import { useMesh } from 'react-babylonjs'

const [boxRef, box] = useMesh()
const instance3 = useRef(instanceRef)

return (
  <box ref={boxRef} />
  {box && (
    <Instance mesh={box} position={Vector3.Up()} />
    <Instance mesh={box} position={Vector3.Down() />
    <Instance ref={instanceRef} mesh={box} position={new Vector3(0, 0, 2) />
  )}
)

I think InstanceData is missing:

• rotation
• rotationQuaternion
• setPivotMatrix
• name

Your proposal looks like a really good way to instantiate a lot of objects easily, but doesn't allow extra control (like instance3 above) - this would also allow access to manually updating vertex data. It also seems less intuitive about how to update position. Do you intend to be able to update positions with your API? I think both ways could be supported. Let me know how I can help - I'll make a useMesh() hook and share with you tomorrow.

I think also it would be good to support loaded models - maybe with hooks.

[brianzinn]

So, this is something like the hook I was thinking of. It would force the re-render with some kind of trick and publish the ref for re-use. This kind of hook could be re-used if generic to clean up even some current stories like the PixiJS one for tetures. Anyway:

export const useMesh = <T extends AbstractMesh>(): [MutableRefObject<T>, T] => {
  const [unused, swapState] = useState(false)
  const ref = useRef<T>(undefined as any)
  
  useLayoutEffect(
    () => void swapState((value: boolean) => !value),
    [ref.current]
  )
  return [ref, ref.current]
}

Then I made a story like this:

const Instance = (props) => {
  console.log('creating')
  const createdInstance = props.mesh.hostInstance.createInstance(props.name);
  createdInstance.position = props.position;
  return null;
}

const InstancedMeshes = () => {
  const [boxRef, box] = useMesh()

  return (
    <>
      <box ref={boxRef} name='box1' position={new Vector3(2, 0, 0)} />
      {box &&
        <>
        {Array.from(new Array(50), (_, index) => index).map(number => (
            <Instance name={`box${number}`} mesh={box} position={new Vector3((number * 2) - 50, 2, 0)} />
        ))
        }
        </>
      }
    </>
  )
}

I just added the above to a story. Need to think about how it would handle all of the lifecycle methods and updates to properties. ie: we should be able to have something like:

const [position, setPosition] = useState(Vector3.Zero());

...
// this should update when state updates, so need same LifecycleListener as Mesh.
// I assign one in the react reconciler
<Instance name='box' mesh={mesh} position={position} />

Anyway, that's just an idea. Would really like to see what direction you were headed, because this looks really useful :)

[hookex]

useMesh()

Good idea! Maybe it is also suitable for tthe <material/> sharing.

<Instance/>

If one instanceMesh is a <instance/>, I'm afraid the amount of react elements can have a significant impact on overall performance. We might draw tens of thousands of instances at once.
If we use only one react element, for example: <Instances datas=[{},{},{}]>, the performance impact from react will be minimal.

But we can start in this way, maybe it can't support tens of thousands of instances, but it's performance is better than <mesh>, flexibility is better than <Instances>.

Here is another way of coding that more simple but less flexible.

return (
  <box>
        <Instance position={Vector3.Up()} />
        <Instance position={Vector3.Down() />
        <Instance ref={instanceRef}  position={new Vector3(0, 0, 2) />
  </box>
)

<Instances>

how to update position.
We can update position by diff data.

loaded models

There's still a lot of room for optimization. Maybe useLoader() hook is a good choice.

[hookex]

Then I made a story like this:

Thank you for writing a demo so quickly.
It seems a lot easier to implement than I thought.
It's update is like the mesh in babylon.js. (InstancedMesh)