-
-
Notifications
You must be signed in to change notification settings - Fork 35.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Examples: Use PBR material in webxr_ar_lighting again. #23171
Conversation
Like I said above I'm not sure if it is correct to process an irradiance environment map with |
No. |
Good! Then I don't think the following line should be part of the example: scene.environment = xrLight.environment; It's probably sufficient to just add the instance of |
@Mugen87 I am not inclined to think I think If so, that size is not compatible with PMREM. |
Unfortunately, the type of the map is not clear to me from the spec: https://www.w3.org/TR/2021/WD-webxr-lighting-estimation-1-20210909/ |
Unfortunately the underlying APIs that we use to expose those values to the web aren't very clear about exactly what those values represent either, aside from being used for "reflections". https://developers.google.com/ar/develop/c/lighting-estimation/introduction?hl=cs#hdr_cubemap https://developer.apple.com/documentation/arkit/camera_lighting_and_effects/adding_realistic_reflections_to_an_ar_experience That said, despite the low resolution the intention of the textures is to give the impression of a plausible, if blurry, reflection of the user's environment in shiny surfaces. I would expect rendering using it to visually resemble the "Phong" screenshot that @optimus007 posted, whereas the "Standard" screenshot feels like it's not making use of it at all. And yes, as far as I am aware they're not compatible with |
I believe this PR is about how PMREM can finally be applied to the reflection map. I think our only concerns were pixel format and performance, which might both be solved now. @WestLangley As for size, agreed that it's not optimal, but As for radiance vs irradiance; I personally hate those words because I can never remember which one is which. But yes, |
On another note; that reflection map really shouldn't be 16x16; that's part of why ARCore's lighting estimation is so bad. ARKit can return a much higher-res version, and if they ever ship WebXR, this will likely not be so small. |
So according to the guides and the feedback in this thread the |
@optimus007 Should be fixed now. |
That is way over-bright. Too much light and/or improper tone mapping. Also, you should not be applying a light probe and an environment map with The directional light should also be unnecessary. |
InShot_20220111_211033082.mp4The environment is not reflecting again. No errors in the console |
This is not an error but the result you get with
It is part of I'm just trying to find a way how to use The example is now in its original state (#20876), just with PMREM support. |
@WestLangley regarding the combination of light sources, generally I would agree you shouldn't use them all together, but it depends on how they are being estimated in XR. This passage makes me think that in fact applying all of them is correct:
|
InShot_20220111_223258356.mp4Testing this pr in my webXR page. Looks great 👍, metallic objects don't appear black like before and the webgl warnings are gone. 🙌 The lack of blurry reflection can be solved in another update Google's scene viewer uses the blurry reflection maybe someone who developed that can help |
@elalish This is important. Do you understand that PMREM accounts for both the radiance and the irradiance implied by the environment map? See #22178 (comment) and #22178 (comment). That means adding a radiance probe and an irradiance probe is double-counting irradiance. If you want to accommodate the modeling approach of ARCore, then you have to change how PMREM is implemented. |
Regarding the fact that the reflection is (too) blurred... |
Right. PMREM is not designed for a 16px input. |
Yeah, that may be true; it's a tad unclear from ARCore's documentation. Basically the question is, are their returned spherical harmonics simply an integration of their returned env map (representing its irradiance)? Or are they separate light sources intended to be added together? I should probably ask the ARCore team; I've noticed WebXR scenes often being too dark, so I was thinking that might be because we weren't use all the light sources they gave us, but I may well be wrong. And yes, I really need to update PMREM to properly handle different input env map sizes. |
@elalish Should I merge this in the meantime? |
Seems fine to me; it's certainly an improvement, though there's more work to do. |
Thanks! |
@Mugen87 @mrdoob I'm looking at making PMREM's internal CubeUV format have variable size, depending on the input texture dimensions. I think I've got it worked out, but the part I'm struggling with is where to communicate that size to the shaders. Would you like it as a |
Is it a valid use case to change the size of a single env map over time? I would expect no. I mean you might have maps of different sizes but once a size is defined it is static. If so, I think we should use a define. |
I'm working on re-adding lighting estimation to AFrame, unfortunately it seems something odd is happening. I'm using It seems that the way the texture is being used by the environment is peculiar. It seems to have most of the colour but none of the detail and an artifact that looks like a small thin vertical rectangle. I took a screen shot below, the left shows what the actual texture looks like, the right is how it looks on a material with 0 roughness. I took a look through the lighting estimation example and I don't think I am missing anything. |
@AdaRoseCannon #23322 should improve the reflections of |
Do you think the mismatched size of the env map is what's causing this issue? |
At least the current |
Related issue: -
Description
This PR restores the PBR materials support of
XREstimatedLight
. I have some concerns though:The WebXR API returns an environment map which seems to be an irradiance environment map. I'm not sure but is this type of map the intended input of
PMREMGenerator
?Wouldn't it be more correct to transform the irradiance environment map to an instance of
LightProbe
(since we can't directly sample irradiance environment maps right now) and add it to the scene?Besides,
XREstimatedLight
uses an instance ofLightProbe
andDirectionalLight
for representing the estimated lighting. Does it make sense to add an instance ofXREstimatedLight
and the irradiance environment map to the scene? Isn't the irradiance added twice now?/cc @toji @elalish