Subtitle time causes them to show up nearly 10 seconds before character starts speaking

[dillfrescott]

Is there any way to fix this without having to manually adjust every time marker? The subtitles will pop up for around 10 or so seconds before the character says the line.

[dillfrescott]

Here is a good example.

[00:00.000 --> 00:23.600] What's the matter?

It takes the man in the video 1-2 seconds to say that line, yet its on the screen for over 23 seconds!

[Spaarpott]

@dillfrescott How did you get 23.6 seconds?
I only get whole seconds in my output using the medium.en.

I have the same problem, but some extra info.
It starts with 0 seconds and creates every line directly after the line before.
It should start the line, when something is spoken, and not in the silences in between.

[00:00.000 --> 00:03.000] | Line 1
[00:03.000 --> 00:05.000] | Line 2
[00:05.000 --> 00:06.000] | Line 3
[00:06.000 --> 00:09.000] | Line 4
[00:09.000 --> 00:11.000] | Line 5
[00:11.000 --> 00:14.000] | Line 6
[00:14.000 --> 00:16.000] | Line 7
[00:16.000 --> 00:17.000] | Line 8
[00:17.000 --> 00:20.000] | Line 9
[00:20.000 --> 00:23.000] | Line 10
[00:23.000 --> 00:26.000] | Line 11
[00:26.000 --> 00:27.000] | Line 12
[00:27.000 --> 00:32.000] | Line 13
[00:32.000 --> 00:36.000] | Line 14
[00:36.000 --> 00:37.000] | Line 15
[00:37.000 --> 00:39.000] | Line 16
[00:39.000 --> 00:41.000] | Line 17
[00:41.000 --> 00:43.000] | Line 18
[00:43.000 --> 00:45.000] | Line 19
[00:45.000 --> 00:51.000] | Line 20
[00:51.000 --> 00:55.000] | Line 21

[jongwook]

It's not a perfect solution, but you could try adjusting the max_initial_timestamp option in the code which is currently hard-coded to 1.0. The timestamp prediction is quite a lot biased towards the integer timestamps, so with a high probability Whisper produces integer-timestamped captions. These could be improved by better timestamp decoding strategies or prompting with non-integer timestamp tokens, and combining a VAD model (as discussed in #29) could also be a quick fix.

[dillfrescott]

@Spaarpott It was consistently around that amount of time lol. 23.6 seconds feels like an hour when you are watching the video and waiting for the person to say that line haha

[Spaarpott]

I can try this next week, but shouldn’t better timings become standard behaviour?

What is the use case to show the subtitle a few seconds before someone speaks?

[dillfrescott]

I agree

[jianfch]

try #435