PositionalAudio: Stuttering with WebXR when close to the audio's origin.

[shiukaheng]

WebXR does not seem to be compatible with positional audio, specifically when put in VR mode. The audio would start stuttering once you are near the audio source and I suspect it is because the audio listener is switching between the left eye position and right eye position rapidly.

However, positional audio seems to work fine in A-Frame, which is based on three.js, so I wonder if there is an easy fix to this, or even if its expected behavior?

To Reproduce

1. Set up a XR enabled renderer
2. Add a PositionalAudio object to the scene with anything in the audioBuffer
3. Enter XR mode and approach the object. Audio stutters.

Code

The code is a bit too long to be put here so I created a repo.

Live example

Hosted with github pages here.

Expected behavior

AudioListener would follow the position of the XR camera.

Platform:

• Device: [Desktop, Mobile, VR]
• OS: [Windows, MacOS, Linux, Android, iOS]
• Browser: [Chrome, Firefox, Safari, Edge]
• Three.js version: [r123]

[Mugen87]

suspect it is because the audio listener is switching between the left eye position and right eye position rapidly.

I don't think this is the root cause. The audio listener's transformation depends on the scene's camera. When using XR, this camera's transformation is updated only once per frame. The number of sub cameras does not matter.

I've tested your live example with Oculus Quest and Oculus Browser but without hearing any stuttering. What device/browser have you tested with?

[shiukaheng]

Chrome on Windows via WebXR API Emulator and Oculus Browser on the Quest and Quest 2 (I'll have to double check. Encountered before but not using this example). Here's a screen recording: https://youtu.be/RrEBt7rIv8k

[shiukaheng]

Same behaviour in the Quest 2, and the audio seems to have some noticeable time lag as well. The stuttering is less apparent but you'll notice it if you go really near the sound source. Screen recording

[Mugen87]

Um, WebXRManager updates the scene's camera only once BUT camera.updateMatrixWorld() is called at the beginning of WebGLRenderer.render(), see:

three.js/src/renderers/WebGLRenderer.js

Line 1017 in 5199433

if ( camera.parent === null ) camera.updateMatrixWorld();

That means AudioListener.updateMatrixWorld() is also called twice which could be the potential root cause. I wonder if #19085 could fix this issue since calling camera.updateMatrixWorld() would not produce a different world matrix anymore.

@shiukaheng Do you think you could apply the patch and make a test?

[shiukaheng]

Can confirm it has been fixed by #19085! Both on Chrome and the Quest. The latency must be a seperate issue then. It's very noticable. Do you think there might be a possible fix?

[Mugen87]

Um, what you you mean with latency? Are you referring to the stuttering or is this something different?

[shiukaheng]

The audio has around 0.2-0.5 seconds of delay in VR which makes it quite disorienting. Might not be a bug but related to audio buffer size. Just mentioning if it is in fact unexpected.

[Mugen87]

Do you experience this latency with other (non-three.js) WebXR samples, too? If so, this should be reported to the browser vendors.

[mrdoob]

It worked last year... https://twitter.com/mrdoob/status/1181272240932810757 🤔

[Mugen87]

It seems the stuttering only appears if you are really close the the positional audio's origin. Maybe it was not noticed at that time...:thinking:.

[johntee]

Confirming that this seems to be solved! I've checked with both a VR and AR project where previously audio stuttered as you moved away from the audio source.

Seems like camera.getWorldPosition() was returning multiple inconsistent locations per frame so distance from camera to audio source was fluctuating with frame rate.

I had problems with this 6 months ago, but worked around the problem by making positional audio effectively only directional by setting refDistance to a high value. I was planning to submit a bug report at some time but happily it is fixed !!!

[bennlich]

What does "stuttering" here mean? Is it hearing pops and clicks?