support all texture formats and types in DataTextureArray.addLayerUpdate and CompressedTextureArray.addLayerUpdate

[HunterLarco]

Description

In #27972 I added logic which allows three.js clients to update specific layers within a texture array and in doing so authored this code which determines the number of bytes required to represent a single layer within a texture array.

This code is accurate for many format-type pairs, but not all. In this PR I updated the logic to include all format-type pairs currently supported by THREE. The impact is that setLayerUpdate will now be accessible to more projects.

Design Notes

I've written this PR so that TextureUtils.getByteLength is exposed as a helper to clients. My company often works with texture arrays and would get a lot of value by having this method available. It's frequently helpful when managing memory to know how large a layer is. Knowing this, I feel it's a good helper to expose.

This contribution is funded by SOOT

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

Filesize dev	Filesize PR	Diff
679.4 kB (168.3 kB)	681.3 kB (168.7 kB)	+1.83 kB

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

Filesize dev	Filesize PR	Diff
457.4 kB (110.4 kB)	459.4 kB (110.9 kB)	+1.99 kB

[HunterLarco]

It seems like the provided example is failing just on windows CI=3. I'm not familiar with the difference between CI 0, 1, 2, and 3. I'd appreciate some guidance on what next steps to take and why CI=3 would fail while [0-2] succeeded.

[RenaudRohlinger]

I tried to simplify the example as much as possible to fix the CI, but it seems that Windows might behave differently than macOS on this.

The CI running under Windows generates a diff of 2.3%:

Diff wrong in 2.3% of pixels in file: webgl_texture2darray_layerupdate

Did you also generate the screenshot under macOS? If so, could you try to run it on a Windows see if anything behave differently, and if not maybe try to generate the example on it.

[Mugen87]

If you have troubles with this particular example, let's put on the exception list for now:

three.js/test/e2e/puppeteer.js

Line 43 in abbf28a

const exceptionList = [

[HunterLarco]

Fixed! I realized that the use of setInterval in the example was causing flakiness in CI. To resolve this, the example has been modified so that the controls GUI can be used to manually move textures from a source array into a dest array. In fact this is nice because it allows for even more control/experimentation with the feature.

Screen.Recording.2024-06-18.at.12.03.11.PM.mov

[HunterLarco]

Would it be possible to get this in for r166? Until landed, ETC2 formats cannot be used with addLayerUpdate which is unfortunate because it means that iOS and Safari are both forced into less performant texture array updates. It would be nice to catch the next release so that my company can improve our rendering for Safari + mobile users.

[Mugen87]

Can this function be internal or is it required for the user to have access to it?

If possible, I would prefer to not expose it and move the logic to WebGLUtils instead. In a different PR, there is the plan to add TextureUtils as an addon in examples/jsm (see #28512 (comment)). It is not ideal to have the same module in the core and as an addon.

[Mugen87]

Oh wait, I see getByteLength() is used in the new example...

[Mugen87]

Since this is an advanced feature, I would probably not expose the helper for now and inline the size computation in the example.

[HunterLarco]

Hmm, I think addLayerUpdate has limited value if three doesn't provide clients a way to know the byte bounds of a single layer. You need TextureUtils.getByteLength to know which subarray within the CompressedArrayTexture's data to update. We could make this method internal, but then it would force clients to either implement this logic themselves or copy/paste it from the three repo.

[HunterLarco]

And inlining would fail in the example because the KTX2Loader dynamically selects a format + type depending on the underlying WebGL context. There's no guarantee that the hard-coded byte length will be consistent on different devices.

[HunterLarco]

(also thanks for the quick review)

[Mugen87]

Let's see how we can organize the code regarding #28512.

[Mugen87]

The code in #28512 is so compact so I think we can move it in src as well. That means there is no need to update this PR.

It's definitely good that the layer size computation in uploadTexture() is moved into a helper function.

[Mugen87]

The example is unfortunately not in an ideal state since a black screen by default isn't good. Instead of using a lil-gui, what do you think about cycling through the layers (1, 2, 3) in a fixed interval?

If the E2E test fails, just add it to the exception list.

[HunterLarco]

The example is unfortunately not in an ideal state since a black screen by default isn't good. Instead of using a lil-gui, what do you think about cycling through the layers (1, 2, 3) in a fixed interval?

Sure! That was the original example earlier in this PR, I can put up another change to revert to that version of the example (with an exception in E2E testing).

Alternatively, I could just initialize the example by writing the first 3 layers to the screen and then letting users navigate the dat.gui. Do you have a preference?

[Mugen87]

Alternatively, I could just initialize the example by writing the first 3 layers to the screen and then letting users navigate the dat.gui. Do you have a preference?

Let's use this approach since webgl_texture2darray_compressed is already animated.