WebGPURenderer: `StorageBufferNode` Support reading external elements in the WebGL Backend #27661

RenaudRohlinger · 2024-02-01T10:38:00Z

Related issue: #27642

Description
Compute shaders in the WebGL backend can now read other elements just like the storage_buffer in WebGPU automatically.
Using storageObject will convert the dual-buffer transform feedback system into a transform feedback system with a single buffer as output and a Pixel Buffer Object (PBO) texture as input.

Here's a basic demo that alternates the order of the array buffer (element(100) becomes element (0) and so on):
https://raw.githack.com/renaudrohlinger/three.js/utsubo/feat/storage_pbo/examples/webgpu_storage_buffer.html

What's cool in this example is that the left side is using the WebGPU Backend and the right side is a WebGL Backend. 😁

This contribution is funded by Utsubo

examples/webgpu_storage_pbo.html

sunag · 2024-02-01T17:30:44Z

examples/jsm/nodes/utils/ArrayElementNode.js

+					const width = Math.floor( square );
+					const height = Math.ceil( square );
+
+					const pboTexture = new DataTexture( attribute.array, width, height, RGBAFormat, FloatType );


The circular-reference problem is because of UniformNode in ArrayElementNode, UniformNode will use ShaderNode and ShaderNode will include try ArrayElementNode again.

I think it would be better to move this uniform creation block to GLSLNodeBuilder something like builder.getPBOUniform() for example, this could fix the circular reference problem too.

I think we have to consider a new Node, similar to what was done in BufferAttributeNode and buffer.toAttribute() as usage. It could be buffer.toTexture()

I agree, this feels like something that should better be placed in GLSLNodeBuilder.

Also consider add/move it to StorageBufferNode.element() to avoid checks a little, and return and new Node to keep operations more organized, it can even be a StorageArrayElementNode.

Thanks for the feedbacks!

The downside of buffer.toTexture() would be that this implies that somehow the developer would expect that something happens in the WebGPU Backend too.
This feature is more like a patch to fix the fact WebGL cannot access other element of array buffers rather than a new feature such as toAttribute().

I think I can try to move all this part to GLSLNodeBuilder but I'm not sure on how to create a Node here. Maybe once everything works we can think about creating a new Node specific to Pixel Buffer Object, but once again it's only a WebGL thing.

I'll be happy to help you with this, let me know when the PR is ready for review so I can analyze it again.

About the example what do you think about renaming it to webgpu_storage_sorting?

Sounds good to me! Thanks a lot for this 😊

sunag · 2024-02-01T17:32:12Z

examples/jsm/nodes/utils/ArrayElementNode.js

+				const propertyNameTexture = builder.getPropertyName( nodeUniform );
+
+				const snippet = /* glsl */`
+					texelFetch(


I think we can also move to GLSLNodeBuilder similar to what happens in builder.generateTextureLoad().

RenaudRohlinger · 2024-02-03T06:33:43Z

I finished the implementation. The main purpose of this PR is basically to improve the compute shading support by allowing reading data from external elements in the WebGL Backend.

The example is very dev oriented but I believe this will be useful for E2E tests in the future and prevent breaking things in regards to storage buffers. Also it should help some very specific use case such as simulations for advanced developers in the future.

It does support different formats even though that's the initial PR (float, vec4). The lack of vec2 and vec3 support is due to the fact I don't how to cast a texelFetch into these formats dynamically with the current node builder. Maybe you could advise me there @sunag.

Also my detection of the type of the target node is very weak in ArrayElementNode. The left node is often replaced by the texelFetch value (because of my output !== 'assign' condition in ArrayElementNode) which breaks. I'm not sure how to determine if the left value (a.mulAssign(b)) <-- is a the target of an AssignNode?

I added 2 TODO comments in the PR to point to the current limitations:

I believe this can be improved even more in the future but that's already a nice additional to compute shading fallback in my opinion.

It's now ready for review as I lack of knowledge on the node part. 👍

LeviPesin · 2024-02-03T15:57:05Z

examples/jsm/nodes/utils/ArrayElementNode.js

-			return nodeSnippet;
+			snippet = nodeSnippet;
+
+			// TODO: How to properly detect if the node will be used as an assign target?


I think you should just check if output is 'void' (i.e. just change 'assign' to 'void' here and in AssignNode).

LeviPesin · 2024-02-04T22:29:41Z

Now the main issue remaining is how to properly detect if the node will be used as an assign target I think

Is my suggestion above also not working? 🤔

sunag · 2024-02-05T02:36:38Z

examples/jsm/renderers/webgl/WebGLBackend.js

-			} else {
-
-				transformBuffers[ i ].switchBuffers();
+				//this.textureUtils.copyBufferToTexture( transformBuffers[ i ].transformBuffer, transformBuffers[ i ].pbo );


@RenaudRohlinger I have a performance issue, could you open the example webgpu_compute_particles and uncomment the PBO line here? Perhaps you have any suggestions as to what it could be?

webgpu_compute_particles shouldn't need to use PBO so it's probably some sort of infinite loop. It's not related to the PBO line, if you comment transformBuffers[ i ].switchBuffers(); the buffers will not be swapped anymore as they should and all compute examples will have these performances issues @sunag

sunag · 2024-02-05T02:42:25Z

Is my suggestion above also not working?

@LeviPesin It's not clear to me that this would resolve the issue. There seems to be more related things.

RenaudRohlinger · 2024-02-05T04:20:31Z

@sunag I just pushed a patch to handle cases where we would use basic dual-buffer compute shaders and then later on, on the same storage buffer, read from other elements and so switch to PBO.

The remaining issue in webgpu_compute_particles and webgpu_compute_particles_rain seems to be shader/node related. Also webgpu_compute_particles_rain isn't working on the WebGPU backend too.

RenaudRohlinger · 2024-02-05T04:53:40Z

By the way, I just realized that the performance issue in webgpu_compute_particles and webgpu_compute_particles_rain is due to overdraw. This happens because all the particles are positioned at 0,0,0 and are not being properly updated, as the dual-buffer isn't being called. /cc @sunag

sunag · 2024-02-05T05:03:11Z

webgpu_compute_particles shouldn't need to use PBO so it's probably some sort of infinite loop

Would it be correct to say that the objective of the PR is to use PBO only in webgpu_compute_audio and webgpu_storage_buffer?

RenaudRohlinger · 2024-02-05T05:28:18Z

Yes, that is correct. That's why I initially thought it would be safer and easier to introduce a third parameter to the storage buffer.

For instance, we could have accessExternal with the following options: 'none' as the default, 'read' that would use PBO, and in the future, 'write' for full GPGPU fallback.

This approach would makes sense in my opinion because accessing external elements is already quite advanced. /cc @sunag

sunag · 2024-02-05T15:15:05Z

Yes, that is correct. That's why I initially thought it would be safer and easier to introduce a third parameter to the storage buffer.

This makes sense, and will be necessary to implement something since we need to have this information to manipulate the PBO, I'm thinking about implementing a toTextureBuffer() / toBufferObject() or similar, this would certainly only have effects in the context of WebGLBackend for enable PBO, and would work as a "bypass" in WebGPUBackend.

sunag · 2024-02-05T23:25:59Z

@RenaudRohlinger I added new TSL function storageObject , something similar to PBO, but SBO could say we have a Storage Buffer Object.

Now the examples is working, including webgpu_compute_audio for this I needs to call switchBuffer() to work...

RenaudRohlinger · 2024-02-06T01:47:00Z

Nice! I love this pattern.
switchBuffer() shouldn't be needed, I fixed the issue of the webgpu_compute_audio example by making sure attributeData gets always the pbo object attached in generatePBO 👍

examples/webgpu_compute_texture.html

RenaudRohlinger · 2024-02-06T05:56:46Z

Thanks for all the help @sunag! Let's give it a try 😊

I will look into adding writing access via full GPGPU setup and dual-dataTexture in another PR in the future.

LeviPesin · 2024-02-06T08:05:12Z

Names storageObject and bufferObject doesn't really seem to be easily understandable... Maybe storageBufferObject and accessMode?

RenaudRohlinger added 4 commits February 1, 2024 19:17

init pbo

04c41e8

remove flag

b4b82eb

regenerate live example

3a0b66a

single buffer and alternate for example

62238bc

github-advanced-security bot found potential problems Feb 1, 2024

View reviewed changes

examples/webgpu_storage_pbo.html Fixed Show fixed Hide fixed

cleanup, no idea about circ dep

64dee72

github-advanced-security bot found potential problems Feb 1, 2024

View reviewed changes

examples/webgpu_storage_pbo.html Fixed Show fixed Hide fixed

RenaudRohlinger added 2 commits February 2, 2024 00:42

test fix circular

9d32e79

cleanup

e089954

sunag reviewed Feb 1, 2024

View reviewed changes

RenaudRohlinger added 9 commits February 2, 2024 16:23

moved pbo management to GLSLNodeBuilder

9738dd4

fix screenshots

9cbafeb

Merge remote-tracking branch 'upstream/dev' into utsubo/feat/storage_pbo

8072cc5

support more ranges

90bf43a

format dynamically in arrayelementnode

0f5d614

init vertex buffer allocation to prevent issues with some backends

b1c595b

support different type size and update example

71a5311

fix files.json

a6c5b64

puppeteer need webgpu support

0b6b1a8

RenaudRohlinger marked this pull request as ready for review February 3, 2024 06:33

RenaudRohlinger added this to the r162 milestone Feb 3, 2024

RenaudRohlinger added 2 commits February 3, 2024 20:21

unbind post pbo

fbb4d63

add TODO

15033d7

LeviPesin reviewed Feb 3, 2024

View reviewed changes

This was referenced Feb 4, 2024

Node: Add increaseUsage() #27672

Merged

Nodes: Add StorageArrayElementNode #27673

Merged

sunag added 2 commits February 4, 2024 01:44

Merge branch 'dev' into pr/27661

affd3bc

cleanup

cbb95e2

WIP: Test (1)

1a22b0b

sunag reviewed Feb 5, 2024

View reviewed changes

handle compute with non-PBO -> PBO case

de9b203

revert WIP

b47480e

sunag added 2 commits February 5, 2024 16:59

TSL: add storageObject

c9a21ea

revision

00f6a5f

make sure attributeData gets pbo attached in GLSLNodeBuilder

7687fcc

no need transfer anymore

39691a5

sunag reviewed Feb 6, 2024

View reviewed changes

examples/webgpu_compute_texture.html Outdated Show resolved Hide resolved

RenaudRohlinger and others added 2 commits February 6, 2024 11:09

revert compute_texture

8936a37

remove copyBufferToSubBuffer

474c0f7

sunag approved these changes Feb 6, 2024

View reviewed changes

RenaudRohlinger changed the title ~~WebGPURenderer: Extend Compute Shading support in the WebGL Backend~~ WebGPURenderer: StorageBufferNode Support reading external elements in the WebGL Backend Feb 6, 2024

RenaudRohlinger merged commit a2bf250 into mrdoob:dev Feb 6, 2024
11 checks passed

RenaudRohlinger mentioned this pull request Feb 6, 2024

WebGLBackend: Support for Accessing External Elements in StorageBuffer #27642

Closed

sunag mentioned this pull request Feb 6, 2024

TSL: Rename storageObject -> storageBuffer #27694

Closed

This was referenced Feb 17, 2024

r162 three-types/three-ts-types#811

Closed

WebGPURenderer: StorageBufferNode Support reading external elements in the WebGL Backend three-types/three-ts-types#818

Merged

RenaudRohlinger mentioned this pull request Jun 10, 2024

WebGPURenderer: Int/Uint Support to StorageBufferNode in WebGLBackend #28606

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WebGPURenderer: `StorageBufferNode` Support reading external elements in the WebGL Backend #27661

WebGPURenderer: `StorageBufferNode` Support reading external elements in the WebGL Backend #27661

RenaudRohlinger commented Feb 1, 2024 •

edited

Loading

sunag Feb 1, 2024

sunag Feb 1, 2024

LeviPesin Feb 1, 2024 •

edited

Loading

sunag Feb 2, 2024 •

edited

Loading

RenaudRohlinger Feb 2, 2024

sunag Feb 2, 2024

RenaudRohlinger Feb 2, 2024

sunag Feb 1, 2024

RenaudRohlinger commented Feb 3, 2024 •

edited

Loading

LeviPesin Feb 3, 2024

LeviPesin commented Feb 4, 2024

sunag Feb 5, 2024

RenaudRohlinger Feb 5, 2024

sunag commented Feb 5, 2024 •

edited

Loading

RenaudRohlinger commented Feb 5, 2024 •

edited

Loading

RenaudRohlinger commented Feb 5, 2024

sunag commented Feb 5, 2024

RenaudRohlinger commented Feb 5, 2024 •

edited

Loading

sunag commented Feb 5, 2024 •

edited

Loading

sunag commented Feb 5, 2024 •

edited

Loading

RenaudRohlinger commented Feb 6, 2024

RenaudRohlinger commented Feb 6, 2024

LeviPesin commented Feb 6, 2024

WebGPURenderer: StorageBufferNode Support reading external elements in the WebGL Backend #27661

WebGPURenderer: StorageBufferNode Support reading external elements in the WebGL Backend #27661

Conversation

RenaudRohlinger commented Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeviPesin Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

sunag Feb 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RenaudRohlinger commented Feb 3, 2024 • edited Loading

Choose a reason for hiding this comment

LeviPesin commented Feb 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sunag commented Feb 5, 2024 • edited Loading

RenaudRohlinger commented Feb 5, 2024 • edited Loading

RenaudRohlinger commented Feb 5, 2024

sunag commented Feb 5, 2024

RenaudRohlinger commented Feb 5, 2024 • edited Loading

sunag commented Feb 5, 2024 • edited Loading

sunag commented Feb 5, 2024 • edited Loading

RenaudRohlinger commented Feb 6, 2024

RenaudRohlinger commented Feb 6, 2024

LeviPesin commented Feb 6, 2024

WebGPURenderer: `StorageBufferNode` Support reading external elements in the WebGL Backend #27661

WebGPURenderer: `StorageBufferNode` Support reading external elements in the WebGL Backend #27661

RenaudRohlinger commented Feb 1, 2024 •

edited

Loading

LeviPesin Feb 1, 2024 •

edited

Loading

sunag Feb 2, 2024 •

edited

Loading

RenaudRohlinger commented Feb 3, 2024 •

edited

Loading

sunag commented Feb 5, 2024 •

edited

Loading

RenaudRohlinger commented Feb 5, 2024 •

edited

Loading

RenaudRohlinger commented Feb 5, 2024 •

edited

Loading

sunag commented Feb 5, 2024 •

edited

Loading

sunag commented Feb 5, 2024 •

edited

Loading