Question on generating Instanced mesh declaratively based on parent props

[AaronGaddes]

Hi there, first off I'd like to say nice work on this library; it looks like it could be exactly what I'm after for a project I'm working on.

I am currently running into a bit of a wall with trying to create instanced meshes and also be able to update them based on props from the parent (or potentially from context). I'm trying to create a tile-based (online) game where there is a board of hexagonal tiles. The tiles themselves are defined via an array that will eventually be defined server-side and come in from props so I need a way to be able to update the instances when the prop changes (i.e. the colour of a tile changes). I intend on creating a react DOM layer above the canvas for some player controls, but don't want the entire scene to refresh and thus the player loose their position in the world.

I originally tried to just make each tile an individual disc mesh but it seems to be pretty bad for performance. I also tried creating the instances imperatively but can't figure out how I'd be able to update them when the props change without the entire scene being reset.

I seen a few previous discussions you've had around implementing instanced mesh but it doesn't look like they have been added yet. Are you still looking to implement them or if not anytime soon would you have any suggestions on how I might be able to implement the above with the current tools?

Here is a Playground link to what I'm looking at doing with instances:
https://playground.babylonjs.com/#XQ7IFU#15

[brianzinn]

You can do something like this perhaps?
https://codesandbox.io/s/unruffled-pasteur-uz5r7

I'd like to get a more declarative way using <instancedMesh createFrom={mesh} /> (not a real host component yet). Instances are actually 4/10 of current issues, so does deserve more attention. I just keep getting side tracked.

also, check out this new video series on a hex game, if you haven't seen it already:
https://forum.babylonjs.com/t/new-demo-video-series-hex-tile-game-board-with-procedurally-generated-islands/13581

[AaronGaddes]

Thank you for getting back to me to quickly. I ended up figuring out a way that is similar to your example but I encapsulated it into a GameMap component that I add as a child of <Scene>:

const tileColours: {[key: string]: Color4} = {
  "land": Color4.FromHexString("#81b29aff"),
  "water": Color4.FromHexString("#0096c7ff"),
  "mountain": Color4.FromHexString("#B2967Dff")
}

export const GameMap = ({mapGrid, onTileClick}: GameMapProps) => {

  const scene = useBabylonScene();

  const handleTileSelected = useCallback((tile) => {
    console.log(tile);
    onTileClick(tile);
  }, [onTileClick])
  
  useEffect(() => {
  
    // create an initial "hetTile" to add instances of
    const hexTile = MeshBuilder.CreateDisc("hexTile", {
        radius: mapGrid[0].width()/2,
        tessellation: 6
    }, scene);
  
    hexTile.isVisible = false;
  
    hexTile.registerInstancedBuffer("color", 4); // 4 is the stride size eg. 4 floats here

    const instances: InstancedMesh[] = [];
    for (let i = 0; i < mapGrid.length; i++) {
      const tile = mapGrid[i];
      const {x, y} = tile.toPoint();
      const newInstance = hexTile.createInstance(`hexTile-${i}`);
      newInstance.position = new Vector3(x+tile.width()/2, y+tile.height()/2, 0);
      newInstance.instancedBuffers["color"] = tileColours[tile.tileType];

      if(scene) {
        newInstance.actionManager = new ActionManager(scene);
        newInstance.actionManager.registerAction(
          new ExecuteCodeAction(
            ActionManager.OnPickTrigger,
            (event) => {
              if(event.sourceEvent.button === 0) {
                handleTileSelected(tile);
              }
            }
          ))
      }
      instances.push(newInstance);
    }

    return () => {
      // dispose of the hexTile and all of the instances
      hexTile.dispose();
    }
  }, [mapGrid, scene, handleTileSelected])

  console.log({scene, mapGrid});

  return null;
}

And then I add it like so:

<div className="App">
      <Engine antialias={true} adaptToDeviceRatio={true} canvasId="sample-canvas">
        <Scene onPointerDown={handleSceneClick}>
          <arcRotateCamera name="arc" target={ new Vector3(0, 1, 0) }
            alpha={-Math.PI / 2} beta={(0.5 + (Math.PI / 2))}
            radius={4} minZ={0.001} wheelPrecision={50} 
            lowerRadiusLimit={8} upperRadiusLimit={20} />
          <hemisphericLight name='hemi' direction={new Vector3(0, 1, 0)} intensity={0.8} />
          <directionalLight name="shadow-light" setDirectionToTarget={[Vector3.Zero()]} direction={Vector3.Zero()} position = {new Vector3(-40, 30, -40)}
            intensity={0.4} shadowMinZ={1} shadowMaxZ={2500}>
            <shadowGenerator mapSize={1024} useBlurExponentialShadowMap={true} blurKernel={32} darkness={0.8}
              shadowCasters={["sphere1", "dialog"]} forceBackFacesOnly={true} depthScale={100} />
          </directionalLight>

          // add GameMap
          <GameMap mapGrid={mapGrid} onTileClick={handleSelectTile} />

        </Scene>
      </Engine>
      <GameOverlay selectedTile={selectedTile} />
    </div>

I like how your example encapsulates it and allows any mesh to be able to be re-used to create instances of any mesh. And I believe since you have put the original mesh into the render tree the instances will automatically get disposed if it is unmounted?
I may end up merging the two approaches together, assuming the way I'm registering actions using the action manager on the instances doesn't hurt performance. I did find an interesting issue with the OnLeftPickTrigger action where it doesn't prevent the action when you rotate the camera, but the OnPickTrigger does so I used it and check which mouse button was clicked. That seems to work fine.

Yeah, I watched the latest video to that series earlier today; I think I'm going to eventually try and modify the shader created in it/will make in the next one, to make my "mountain" tiles have actual mountains (and to generate textures for the rest of the tiles)

[AaronGaddes]

I ended up creating an InstanceFromMesh component similar to the example you provided, however I added some the ability to pass in some additional properties for it, as well as an onClick handler which registers an Action with the ActionManager and the ability to add multiple InstanceBuffers:

import { useEffect } from 'react'
import {
  Mesh,
  Vector3,
  ActionManager,
  ExecuteCodeAction
} from '@babylonjs/core'

export interface InstancedBuffer<T> {
  name: string;
  dataSize: number;
  data: T
}

export type instancedMeshClickHandler = () => any;

interface InstanceFromMeshProps {
  id: number;
  mesh: Mesh;
  position?: Vector3;
  rotation?: Vector3;
  instancedBuffers?: InstancedBuffer<any>[];
  onClick?: instancedMeshClickHandler;
}

export const InstanceFromMesh = ({id, mesh, position, rotation, instancedBuffers, onClick}: InstanceFromMeshProps) => {

  useEffect(() => {
    console.log(`Creating instance`);
    const scene = mesh.getScene();
    const instance = mesh.createInstance(mesh.name);

    instance.position = position || mesh.position.clone();
    instance.rotation = rotation || mesh.rotation.clone();

    if (instancedBuffers) {
      instancedBuffers.forEach((buffer) => {
        const {name, data} = buffer;
        instance.instancedBuffers[name] = data;
      })
    }
    
    if (onClick) {
      instance.actionManager = new ActionManager(scene);
      instance.actionManager.registerAction(
        new ExecuteCodeAction(
          ActionManager.OnPickTrigger,
          (event) => {
            if(event.sourceEvent.button === 0) {
              onClick();
            }
          }
        )
      )
    }

  }, [mesh, position, rotation, instancedBuffers, onClick]);
  
  return null;

}

However, I have found something intriguing; I noticed that in the engine object of the BabylonJSContext (when looking at the React Dev Tools) I can see that the current drawCalls are is ~194 just on initial render and whenever I click on a tile it jumps up by a ~600 draw calls. I would have expected, at least the initial draw calls to only be 1, based on what I see in the inspector of my previous Babylon playground (with added action managers (https://playground.babylonjs.com/#XQ7IFU#17). Is this cause by the inherent declarative nature of the react-babylonjs library or would there be something else causing it? Or am I miss-understanding what the draw calls is in the context?

[AaronGaddes]

I managed to get the BabylonJS debug inspector up and running, and while I do see it initially have a large draw count, though I'm not sure if that's just the component initialising or not as the large number is only there for a split second) it looks like it "settles down" to 1-2 draw calls. Thanks for all of your help on this.

Here's a little look at what I've got so far:

[brianzinn]

@AaronGaddes thanks for the inspiration! <instancedMesh .. /> is out on v2.2.7. Here is a demo:
https://brianzinn.github.io/react-babylonjs/?path=/story/babylon-basic--instances

It looks like the dependencies on your useEffect will create a new instance even when ie: the position changes. Also, the instancedMesh doesn't have an onClick, but can be added similar to how you have done it. Thanks again for your help.