Suboptimal result with SP

[chen-yy20]

Hi OpenSora Team,

I want to express my appreciation for your outstanding work!

I have reconstructed your code and implemented the OpenSora Pipeline specifically for T2V inference, maintaining most of the original code and model weights intact. However, I've encountered an issue while attempting to optimize the T2V inference process using Sequence Parallel (SP). The results have been notably suboptimal.

Using the prompt "Sunset over the sea," I've observed a significant difference in output quality:

Normal execution produces expected high-quality results:

OpSora_output_ori.mp4

With Sequence Parallel enabled, the output shows degraded quality:

OpSora_output.mp4

I'm curious whether you encountered similar issues during OpenSora's development. Could you provide insights into which pipeline components might be responsible for this quality degradation? Any guidance would be greatly appreciated.

Thank you for your time and consideration.

Best regards