Add access to WebXR depth texture + add example #27695
Conversation
📦 Bundle sizeFull ESM build, minified and gzipped.
🌳 Bundle size after tree-shakingMinimal build including a renderer, camera, empty scene, and dependencies.
|
|
@Mugen87 is there a way to change an existing shader? I'd like to reuse MeshLambertMaterial by adding my helper functions and inserting the calculation of the edge pixels. |
You can use material.onBeforeCompile = shader => {
shader.uniforms.depthTex = { value: /* texture here */ };
shader.fragmentShader = shader.fragmentShader
.replace( /#include<color_fragment>/g, match => /* glsl */`
// your code here
${ match }
` );
};This should also address #27664 |
|
If you want it globally included you can also replace things in |
Generally I'd discourage modifying the built in shaders in the examples. It works but it's brittle. And of course having multiple libraries or components modifying built in materials can easily lead to overwritten changes and breakage.
Global uniforms aren't supported so you'll have to set it per material. UBOs are now in core but the current implementation doesn't make this case much more ergonomic. See #16922. Encapsulating the above in a class can make this easier to manage at an application level but I don't think it's necessary for the example: class CustomMaterial extends MeshStandardMaterial {
set depthTex( v ) {
this._uniforms.depthTex.value = v;
}
constructor( ...args ) {
super( ...args );
this.onBeforeCompile = shader => {
// ...
this._uniforms = shader.uniforms;
};
}
} |
The same is true for I'm just not sure how someone could take that sample and extend it; a common flow would be "this is a cool sample, now I want to load a glTF and keep the materials in that and apply that fancy occlusion." – how would someone do that? Need to be super careful then to not "miss" any material to modify it, can't use other onBeforeCompile callbacks, etc. |
You can design for this case into your app or design your wrapper class to take an onBeforeCompile function and call it if it's set (ie adding a getter / setter for
This is always the case if you want to use a different material on a loaded model. My understanding is that |
|
@Mugen87 it doesn't look like I have access to the internal shaders to extend them. What do you think of this change? |
|
I would leave the PR in its current state then 👍 . |
|
Is it possible to query the depth texture per frame from the render-loop, to for instance bounce objects off of walls sensed through the depth.api? (the webxr cubes example currently has the cubes getting occluded behind walls, would it be possible to query the depth data to see if they should bounce off of them instead?) I have looked at the code from the webxr cubes example and WebXRDepthSensing.js and WebXRManager.js but could not find where it was being defined if this depth data was GPU optimized texture or CPU optimized or understand where one could interject real time interaction possibilities with the depth data. Any example that could show interaction with the Depth api texture data would be most helpful! Also, a CPU optimized depth texture example would be great! |
If you look at this PR, the |
|
It seems the E2E screenshot of the new example requires an update. Can you quickly regenerate the screenshot and see if it helps? If regenerating on your side does not fix the failure, I'll try it myself after the merge. |
Would you mind regenerating it? I developer on a headless server and it doesn't have chrome. |
Np, see #28535. |
Some authors want to use the depth texture to add an effect using a custom shader.
This diff enhances the WebXR code to allow access to the texture and add a sample.
Thanks to @hybridherbst !
This contribution is funded by Meta