fit_textured_mesh.ipynb - RuntimeError: replace inplace operation by an out-of-place one.

[pablodemiguel1]

🐛 Bugs / Unexpected behaviors

After running all the code of the project fit_textured_mesh.ipynb step by step in google colab up until (including) the last bit of code before saving the final predicted mesh...:

visualize_prediction(new_src_mesh, renderer=renderer_textured, silhouette=False)
plot_losses(losses)

... I got the following RuntimeError:

It points me to 8 frames which are involved in the problem (see image below - as many as I could fit in my screen capture):

I've searched on this forum to look for similar questions but couldnt find anyone that has encountered this problem. Stack Overflow has a couple of entries with a similar problem but I couldnt understand the terminology (fairly new to programming). I'll leave both Stack Overflow posts below just in case:

https://discuss.pytorch.org/t/issue-with-using-dataparallel-runtimeerror-output-0-of-broadcastbackward-is-a-view-and-its-base-or-another-view-of-its-base-has-been-modified-inplace/84677

pytorch/pytorch#36608

A bit of help to come out of this pickle would be greatly appreciated!

Thank you,
Pablo

[nikhilaravi]

@pablodemiguel1 were you able to run the optimization loop and visualize predictions during this? Did you only see the error after running the optimization?

[bottler]

I can reproduce this in Colab today using the stable version (0.4.0) of both PyTorch3D and fit_textured_mesh.ipynb. I have modified the notebook to set perspective_correct=False in every RasterizationSettings object in the notebook (because of #561, which seems unrelated to this issue). The error is in the final visualization, not during an optimization loop.

Note that Colab has recently updated to the very recent PyTorch 1.8.1 release which means that PyTorch3D has to be installed from source (which is done automatically by the notebook) and this takes ages. I think this issue might be caused by something which has changed in PyTorch 1.8.0 or 1.8.1.

I can see from the call stack that the error is something to do with the autograd graph inside the rendering inside visualize_prediction, which we calculate but don't need. And therefore a fix / the fix is to replace the line

predicted_images = renderer(predicted_mesh)

in visualize_prediction with

with torch.no_grad():
    predicted_images = renderer(predicted_mesh)

. This makes everything work fine for me. I don't yet know what is really going on so I can't explain exactly why the original used to work and doesn't any more.

[pablodemiguel1]

Hi Bottler,

A massive thank you for the detailed explanation!!! This solved the problem - was getting quite frustrating almost getting to the end to no avail. Now I can put it to good use.

And congrats to the Pytorch3D team, its a really cool technology.

Regards,
Pablo

[bottler]

I've made this change in the main branch now so it will be fixed in the next release.