Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sized PMREM #23322

Merged
merged 9 commits into from
Feb 2, 2022
Merged

Sized PMREM #23322

merged 9 commits into from
Feb 2, 2022

Conversation

elalish
Copy link
Contributor

@elalish elalish commented Jan 24, 2022

At long last, I've finally refactored PMREM to have variable size, based on the size of the input environment map. This helps in several ways:

  1. First, if an equirect environment image was smaller than 1024x512, it used to get blurred too much (Examples: Use PBR material in webxr_ar_lighting again. #23171 (comment)).
  2. If the image was larger, then its reflection detail was simply lost.
  3. If a small environment image was used, it was handled internally as though it was 1024x512, which meant there was no cache efficiency or generation time advantage to using a smaller envMap.

These are all now fixed. I also changed the internal PMREM format a bit to make it scale more easily. @mrdoob @Mugen87 @WestLangley

It should be noted that there is a direct relationship between the minimum roughness value you want to resolve on your materials and the required environment size. For instance, the old default of 1024x512 (or a cubemap of 256x256) handled down to roughness 0.05. For 0.1, only a 64x64 cubemap is required (which can save a lot of bandwidth on your HDR). The WebXR lighting cubemap is 16x16, so it can only resolve roughness down to 0.21. Now, you may also be able to get away with less environment map resolution if you are rendering to a small canvas or your shiny parts are more curved. However, keep in mind that in the limit of roughness => 0, you may need infinite environment resolution to avoid seeing pixels (this is just the physics of optics).

As an example, here is the old code with a 1024x512 envMap (and zero roughness on the visor):
image

And here is the new way with the HDR downsampled to 256x128 (130kb vs 450kb).
image

@@ -205,8 +222,18 @@ class PMREMGenerator {

_fromTexture( texture, renderTarget ) {

if ( texture.mapping === CubeReflectionMapping || texture.mapping === CubeRefractionMapping ) {

this._setSize( texture.image.length === 0 ? 16 : texture.image[ 0 ].width ?? texture.image[ 0 ].image.width );
Copy link
Contributor Author

@elalish elalish Jan 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mugen87 This is hideous and hacky, and I'd love for there to be a better way. I really wish textures had their own width and height members, because as it stands, finding that info is totally different for textures, cubemaps, data textures, textures created by renderTargets, etc. I'm not sure what the best path forward for three.js is, but this was the best I could manage for now to get this working. Also, I'm falling back to 16 because that's what the WebXR cubemap is. I couldn't find its size anywhere at all; am I missing something?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find its size anywhere at all; am I missing something?

I did not find a note about the size in the documentation.

I'm not sure what the best path forward for three.js is, but this was the best I could manage for now to get this working.

Yeah, it's inconvenient right now to cover all possible texture permutations. The main problem is that cube textures can hold an array of images or an array of (data) textures. I'm not sure yet but one solution could be to wrap the images of a cube texture in instances of Texture to unify the access.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, but I was thinking just add width and height members to the base texture class and fill them in appropriately so that we don't have to dig into the .image at all.

And my point with the WebXR environment cube map is that the three.js code knows the size (it gets an array back from the API), but I don't think it saves that size anywhere that we as users of three.js can retrieve. That feels like a bug, but a bug in the API shape that a texture width and height could solve.

@elalish
Copy link
Contributor Author

elalish commented Jan 25, 2022

@mikkoh This is the PMREM update I told you about to make small environment maps work even better.

@Mugen87
Copy link
Collaborator

Mugen87 commented Jan 25, 2022

It seems the "old" node material needs an update so it works with this PR.

/cc @sunag

@mrdoob mrdoob merged commit 555c65d into mrdoob:dev Feb 2, 2022
@mrdoob
Copy link
Owner

mrdoob commented Feb 2, 2022

Thanks!

@mrdoob
Copy link
Owner

mrdoob commented Feb 2, 2022

@sunag could you take a look at the nodes examples that broke (webgl_materials_envmaps_hdr_nodes and webgl_materials_envmaps_pmrem_nodes) when you have a chance?

Screen Shot 2022-02-02 at 11 37 13 AM

@mrdoob
Copy link
Owner

mrdoob commented Feb 2, 2022

@elalish seems like webgl_materials_cubemap_dynamic broke too...

Screen Shot 2022-02-02 at 11 32 13 AM

@elalish elalish deleted the resizePMREM branch February 2, 2022 16:44
@Mugen87
Copy link
Collaborator

Mugen87 commented Feb 2, 2022

@sunag could you take a look at the nodes examples that broke (webgl_materials_envmaps_hdr_nodes and webgl_materials_envmaps_pmrem_nodes) when you have a chance?

How long do we plan to support the "old" node material? Maybe this is a good opportunity to remove it so we can focus on the new system?

@sunag
Copy link
Collaborator

sunag commented Feb 3, 2022

Maybe this is a good opportunity to remove it so we can focus on the new system?

Yes, I agree too.

donmccurdy pushed a commit to donmccurdy/three.js that referenced this pull request Mar 10, 2022
* new PMREM format works; #defines functional

* regenerate internals when envMap size changes

* hooked up variable size

* added disposal, cleanup

* fixed size fallback

* fix linting

* mrdoob approves?

* removed envMapCubeUV

* mrdoob approves?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants