Add possibility to bind 3D textures and 2D textures array as color attachment to framebuffer

[DavidPeicho]

Hi,

This PR includes the changes:

• WebGLLayeredRenderTarget ⟶ adds the possibility to render to 3d texture and 2d texture arrays.
• Updates WebGLRenderer.setRenderTarget to choose the layer to which the rendering should occur
• Adds a complete example of using WebGLLayeredRenderTarget. The example is simply based on the Texture 2D array example, and is mostly useful for debugging. In the future, I can replace it with something fancier.

Why?

Having the possibility to render to volume allows for instance to:

• Generates volume on the GPU (think about Perlin noise, etc...)
• Compute irrandiance cache
• etc...

Improvements

There are a couple of things that aren't nice and look pretty dirty for now:

• This is a complete hack. I think it would make more sense for a user to directly give the texture to the render target, but that would be an API break change. Alternatively, I guess we could do the texture in a method and the WebGLLayeredRenderTarget could call this function.
• Same here, this feels like a hack. The issue is still hat WebGLRenderTarget is made to only accept Texture.
• In the renderer, I used activeCubeFace as the layer selector. I believe it could makes sense maybe to have a separate function for cube / layered render target?

Let me know what you think. I know this is delicate because it touches to some of the internals.

[DavidPeicho]

It would be better if I could fit the render target in the already existing WebGLRenderTarget instead of creating a new class though....

[mrdoob]

It would be better if I could fit the render target in the already existing WebGLRenderTarget instead of creating a new class though....

Yes, I was thinking that.

I wonder if it would be possible to just have a single WebGLRenderTarget and delegate the logic to the texture type. So instead of WebGLCubeRenderTarget it would be a WebGLRenderTarget with a CubeTexture. Same with DataTexture3D, DataTexture2DArray, etc

Not sure if that would work for MultisampleTexture though.

[Mugen87]

As a first step in this process, how about removing this hack:

three.js/src/renderers/WebGLCubeRenderTarget.js

Line 21 in c3c03a7

this.texture.isWebGLCubeRenderTargetTexture = true; // HACK Why is texture not a CubeTexture?

Make WebGLCubeRenderTarget using a CubeTexture seems like a change the engine definitely wants^^.

[mrdoob]

@Mugen87 Yep! I guess if we give that a go we'll have a better picture of whether the idea is viable or not.

[Mugen87]

@DavidPeicho Do you want to investigate this or should I have a look? 😇

[DavidPeicho]

I wonder if it would be possible to just have a single WebGLRenderTarget and delegate the logic to the texture type

100% what I was thinking. Doing that makes it much easier to have any texture attached as a render target.

Not sure if that would work for MultisampleTexture though.

Not sure yet, I have to check! I am not sure you can multisample a 3D texture.

@DavidPeicho Do you want to investigate this or should I have a look? 😇

I can give it a look. I will need your opinions on some stuff obviously especially regarding the API.

So let me try to use the original WebGLRenderTarget with any supported texture as an attachment, and without breaking the API 😄

[Mugen87]

I can give it a look. I will need your opinions on some stuff obviously especially regarding the API.

I think I would start with a fresh PR which only introduces CubeTexture in WebGLCubeRenderTarget, first. No API changes should be required here. It's be a pure internal change.

[DavidPeicho]

Wouldn't it be better to try to also migrate WebGLCubeRenderTarget in WebGLRenderTarget?
That makes sense, and only the internal renderer cares about what it renders to.

I feel the concept of render target should be dissociated from the texture type it has. It's obviously hard now to make that without a breaking change, but we can keep the current behavior and maybe add console.warn as usual.

WebGLRenderTarget does nothing except exposing the fromEquirectangularTexture that could also be somewhere else?

[Mugen87]

Wouldn't it be better to try to also migrate WebGLCubeRenderTarget in WebGLRenderTarget?

Not sure if we can do this change for backwards compatibility reasons. Keep in mind that users create instances of WebGLCubeRenderTarget, e.g. when using fromEquirectangularTexture() or when creating CubeCameras. This topic needs more investigation.

I would focus on integrating CubeTexture in WebGLCubeRenderTarget first.

[DavidPeicho]

Yes, that's why I was thinking about maybe letting the user set its own texture for now in WebGLRenderTarget. That will fix most issues and allow to render to any texture.
We can make the change in parallel to WebGLCubeRenderTarget so that it's still backward compatible.
Doing that basically doesn't break anything, and this PR can go on.

[Mugen87]

First try: #20182

[DavidPeicho]

Okay great, thanks!

I tried to re-work the more general case, allowing WebGLRenderTarget to accept 3D texture and 2D texture array.

Basically, now you can set your texture via:

const renderTargetTexture = new THREE.DataTexture2DArray();
renderTargetTexture.format = ...;
renderTargetTexture.type = ...;

const rt = new THREE.WebGLRenderTarget();
rt.setTexture( renderTargetTexture );
rt.setSize( 100, 100, 50 );

This sounds pretty reasonable to me. It doesn't introduce any API break change. The few things I am not 100% fan are:

• It's not really possible to pass the depth as an argument to THREE.WebGLRenderTarget, because it's suppose to have an options object. To be fair it could be enough to check whether options is a number.

function WebGLRenderTarget( width, height, options ) {
        
        if ( !isNaN( options ) ) {
        
                this.depth = options;

        }
        ...
}

• A Texture may be created for nothing by default in the render target. I don't think that's too bad because anyway, creating render target isn't supposed to be a hot code path

Let me know what you think @mrdoob and @Mugen87

[DavidPeicho]

I actually found today that using ANGLE, rendering to 3D textures will fail on a layer != 0. I tried on a Windows machine and it failed.

Links:

• https://bugs.chromium.org/p/angleproject/issues/detail?id=4417
• Platform and browser inconsistencies rendering to 3D textures KhronosGroup/WebGL#2558

[DavidPeicho]

Any news on how you want me to proceed API-wise?
I have to look whether this is still an issue in Angle

[mrdoob]

Instead of adding is3D to WebGLRenderTarget I think I would rather add the code here:

isRenderTarget3D = renderTarget.texture.isDataTexture3D || renderTarget.texture.isDataTexture2DArray;

[mrdoob]

I'll see if I can resolve the change and fix this on my end.

[mrdoob]

Aaaand merged. Thanks!

[mrdoob]

@DavidPeicho Seems like it's black though...

https://raw.githack.com/mrdoob/three.js/dev/examples/webgl2_rendertarget_texture2darray.html

[DavidPeicho]

Hi! I will have a look in the next fews days. I let that on the side because of the bug in ANGLE. I will fix the black screen no worries

[mrdoob]

Thanks!