Questions on Ground Truth Optical Flow Computation and Interpolation in the DSEC Dataset

[usamic]

Hello author,
First of all, thank you for providing such an amazing dataset.
Since my English skills may not be perfect, I apologize if my questions aren’t fully clear, but I’ll do my best.
I have a few general questions about the optical flow in the DSEC dataset. If any of this information is already publicly available, I apologize in advance:

1. How are valid pixels (valid status = 1) and non-valid pixels (valid status = 0) defined? From what I can tell, it doesn’t seem to be directly related to pixels where events occur.

2. How is the optical flow computed? For instance, is an optical flow algorithm applied to an accumulated event frame?

3. If I need the optical flow between two arbitrary timestamps, would simple bilinear interpolation yield a reasonable approximation?
   For example, if the dataset provides the optical flow between timestamps (37089500000, 37089600000), and I want the flow between (37089500000, 37089550000), would scaling the flow vector by 0.5 yield a good approximation?

Usami

[magehrig]

Hi @usamic

1. This is described here: https://dsec.ifi.uzh.ch/data-format/

2. If you want the details, I suggest having a look at the paper. Essentially, we use LIdar + IMU odometry for locally accurate ego-motion estimation. Then from the depth values we from the Lidar we can compute the scene flow for static parts of the scene which is used to compute depth and optical flow.

3. It's an approximation but not very accurate. Nowadays, you could actually create your own approximate optical flow ground truth at higher rate using a SOTA non-causal point tracker on the frames at 20 Hz. If you then need intermediate flow approximations you could approximate the flow with cubic splines.

[usamic]

Thank you, @magehrig , for your clear and detailed answers to all of my questions. I appreciate your efforts in addressing my concerns comprehensively. I also apologize for inadvertently including information that has already been made public in my queries.

I have now understood everything perfectly and am truly grateful for the outstanding dataset that you’ve shared.