Nodes: MaterialX Noise Functions Lib

[sunag]

Related issue: #20541

Description

This is an early part of the implementation.

NodeMaterial

• Add functional interface to GLSLNodeFunction.
• Add support to array for function call
• Add includes for constructor argument
• Convert to nodeObjects function call parameters and includes

ShaderNode

• add fn method to create function using backend shader language. commit

Example

• add webgl_nodes_materialx_noise commit

https://raw.githack.com/sunag/three.js/dev-materialx-functions-lib/examples/webgl_nodes_materialx_noise.html

This contribution is funded by Google via Igalia

[mrdoob]

This is looking great!

[mrdoob]

Thanks!

[mrdoob]

/fyi @jstone-lucasfilm

[jstone-lucasfilm]

This looks really promising, thanks @sunag and @mrdoob.

Over time, I wonder if we might be able to increase the degree of code sharing between the MaterialX and three.js codebases, so that future refinements to core shading code (e.g. noise functions such as mx_worley_noise_vec3) can be made in a central location and inherited by the other project when needed.

Based on work that was started at Autodesk, we have early JavaScript bindings for MaterialX libraries such as MaterialXGenGlsl, so one option would be to determine what's missing from those bindings that would be needed for three.js to call into this library directly, generating the required GLSL code when a MaterialX node (e.g. worleynoise3d) is referenced in an asset.

None of these considerations should block the great work that I'm seeing in progress, but it seems worthwhile to consider them for the future.

[mrdoob]

Thanks for the pointers!

Over time, I wonder if we might be able to increase the degree of code sharing between the MaterialX and three.js codebases, so that future refinements to core shading code (e.g. noise functions such as mx_worley_noise_vec3) can be made in a central location and inherited by the other project when needed.

That would be great indeed! Maybe a npm package with all the snippets?

[jstone-lucasfilm]

@mrdoob That's a really good suggestion. We currently package MaterialX-generated shader snippets for OSL in this way, allowing them to be individually accessed in tools that perform their own final shader generation. It should be straightforward to extend this to GLSL in the future, and the results could indeed be shared directly through npm and other package managers.

For the formatting of this shader snippet library, would it be reasonable to have a separate GLSL file for each MaterialX node, including the definition of that node plus file references to any shared GLSL libraries upon which it depends? That's the pattern we currently use for OSL, so it would likely be the most straightforward starting point for GLSL and other languages.

[mrdoob]

@sunag what do you think?

[sunag]

Thanksss @jstone-lucasfilm :) . Soon after reading the comment I thought of making a live example using MaterialX, and it worked perfectly with the GLSL code already in the MaterialX repository.

Live
https://raw.githack.com/sunag/three.js/dev-materialx-live/examples/webgl_nodes_materialx_live.html

Code

import { MeshPhysicalNodeMaterial, code, fn } from 'three-nodes/Nodes.js';

const mxNoiseURL = "https://raw.githubusercontent.com/AcademySoftwareFoundation/MaterialX/main/libraries/stdlib/genglsl/lib/mx_noise.glsl";
const mxNoiseLive = await fetch( mxNoiseURL );
const mxNoise = code( await mxNoiseLive.text() );

const mxIncludes = [ mxNoise ];

const mx_perlin_noise_float = fn( 'float mx_perlin_noise_float( vec3 p )', mxIncludes );

//..

const material = new MeshPhysicalNodeMaterial();
material.colorNode = mx_perlin_noise_float( someCoordNode );

Using interfaces to identify the signature of the functions we can refine the code at will, including the dependencies.

In a way this is already possible without much effort, of course we can do something more organized creating a npm package if needed or create only one build in the MaterialX repo itself for this purpose, but I think the idea of @jstone-lucasfilm is very good.

[donmccurdy]

For the formatting of this shader snippet library, would it be reasonable to have a separate GLSL file for each MaterialX node, including the definition of that node plus file references to any shared GLSL libraries upon which it depends? ...

In the npm ecosystem, the path an end-user imports is abtracted from the files on disk. Separate files for each node would be completely fine, there just needs to be something (usually a src/index.js entry) that defines a list of all entrypoints to the module:

// index.js
export * from './nodes/a.js';
export * from './nodes/b.js';
export * from './nodes/c.js';
// ...

Those files may in turn import from any shared GLSL libraries they require. The end-user only sees something like this:

import { b } from '@materialx/glsl';

Happy to help with the scaffolding if MaterialX wants to go in that direction!

[sunag]

@donmccurdy I made a draft here a few days ago but I think about doing a PR after the name of the nodes becomes more mature in the context of three.js. Anyway, I also think that would be the better way too.

[sunag]

@jstone-lucasfilm have you ever thought of putting an attribute of canvas position XY in mtlx this is just for the purpose of editing in a node editor?

For example:

<texcoord name="texcoord1" type="vector2" posx="100" posy="50" />

This would represent a node in the editor at position { x:100, y:50 }. I think it would allow exporting not only the material but the entire editing canvas, so it can be used in other node editors with the same arrangement.

[jstone-lucasfilm]

@sunag Although they're not yet leveraged in production tools, there are actually xpos and ypos attributes in MaterialX that address this use case:

https://materialx.org/assets/MaterialX.v1.38.Spec.pdf#page=42

We're planning to support this feature in the upcoming Open MaterialX Graph Editor, which is scheduled to appear on GitHub later this year:

https://materialx.org/assets/ASWF_OSD2022_MaterialX_OSL_Final.pdf#page=26

One caveat is that the xpos and ypos attributes are expressed as multiples of the default width and height of a node in the application, rather than being literal pixel positions. Our thinking is that this makes them more transferrable between applications with radically different node layouts, though we're open to new ideas from the community on improving this convention.

[sunag]

Ohh... Amazing!! Certainly Open MaterialX Graph Editor and Three.js Node Editor will have a great exchange soon.

[mrdoob]

@jstone-lucasfilm Any plans on generating wgsl files too? https://www.w3.org/TR/WGSL/

[jstone-lucasfilm]

@mrdoob Not yet, though if there's a good case to be made for direct generation of WGSL, I believe that would be a straightforward feature to add. What are your thoughts on the benefits this would bring?

[sunag]

@jstone-lucasfilm I don't know if MrDoob sees the same thing, but as we are developing WebGPURenderer we could study a way to port MaterialX and share the same algorithms for both WebGL and WebGPU, and I believe we can move forward in implementing a MeshStandardSurfaceMaterial from Autodesk for both backends.