Internal engine sometimes produces bad mkv files

[southrop]

Hi, I used the latest versions.json to convert all the new cutscenes, but I've been having an issue with the resultant videos. For some reason, these cutscenes always convert with problems in the video, always at the same positions. These are skips and pixelation(?) so it seems like parts of the video just aren't converting correctly? I ran a repair from the launcher in case it was my local files being the issue, but it didn't modify any of the video files. Has anyone else experienced this issue? Are these possibly issues in the original USM files or something?

[ToaHartor]

No issue in the original USM file, and if the decryption fails it won't look like this. I would've said it looks more like a playback issue, like if your device/player doesn't support the video codec (VP9 in our case). You might want to check the playback statistics in the player, to see if frames are correctly decoded and aren't corrupted.

[phuchptty]

try to use another engine like FFmpeg. In some cases, the internal engine works wrong

[southrop]

Thanks for the resposne. After checking the unmerged files, I realised that the unmerged videos don't have the issues I mentioned, so I tried converting it with ffmpeg using -e ffmpeg and my resultant videos don't seem to have any issues now. Not sure why this happens.

[phuchptty]

Yeah. The internal engine will break 1-2s randomly

[southrop]

By the way is it just me or is GICutscenes no longer outputs the help text with the -h command? Wanted to check help to see if there was a verbose/debug mode or something but was unable to.

Unhandled exception. System.AggregateException: One or more errors occurred. (Method 'get_Encoding' in type 'TextWriterThatWritesToStandardStreamWriter' from assembly 'System.CommandLine, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' does not have an implementation.)
 ---> System.TypeLoadException: Method 'get_Encoding' in type 'TextWriterThatWritesToStandardStreamWriter' from assembly 'System.CommandLine, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' does not have an implementation.
   at System.CommandLine.IO.StandardStreamWriter.CreateTextWriter(IStandardStreamWriter )
   at System.CommandLine.Help.HelpResult.Apply(InvocationContext )
   at System.CommandLine.Invocation.InvocationPipeline.GetExitCode(InvocationContext )
   at System.CommandLine.Invocation.InvocationPipeline.InvokeAsync(IConsole )
   at System.CommandLine.CommandExtensions.InvokeAsync(Command , String[] , IConsole )
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at GICutscenes.Program.Main(String[] args)
   at GICutscenes.Program.<Main>(String[] args)```

[ToaHartor]

Thanks for the resposne. After checking the unmerged files, I realised that the unmerged videos don't have the issues I mentioned, so I tried converting it with ffmpeg using -e ffmpeg and my resultant videos don't seem to have any issues now. Not sure why this happens.

May I have a look at the merged file with the internal engine ? (with some file transfer platform).
I'm quite curious of the kind of tweaking the internal engine would require, as I didn't get any issue on my side.

[southrop]

Sure thing. I've uploaded the videos here and included a log of my console output. There was also a 3.0 cutscene that I noticed had a glitch so I included that too. Please let me know when you've downloaded it so I can remove them.

edit: added some cutscenes from 2.8 event too.

[ToaHartor]

By the way is it just me or is GICutscenes no longer outputs the help text with the -h command? Wanted to check help to see if there was a verbose/debug mode or something but was unable to.

Unhandled exception. System.AggregateException: One or more errors occurred. (Method 'get_Encoding' in type 'TextWriterThatWritesToStandardStreamWriter' from assembly 'System.CommandLine, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' does not have an implementation.)
 ---> System.TypeLoadException: Method 'get_Encoding' in type 'TextWriterThatWritesToStandardStreamWriter' from assembly 'System.CommandLine, Version=2.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' does not have an implementation.
   at System.CommandLine.IO.StandardStreamWriter.CreateTextWriter(IStandardStreamWriter )
   at System.CommandLine.Help.HelpResult.Apply(InvocationContext )
   at System.CommandLine.Invocation.InvocationPipeline.GetExitCode(InvocationContext )
   at System.CommandLine.Invocation.InvocationPipeline.InvokeAsync(IConsole )
   at System.CommandLine.CommandExtensions.InvokeAsync(Command , String[] , IConsole )
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at GICutscenes.Program.Main(String[] args)
   at GICutscenes.Program.<Main>(String[] args)```

This error is really suspect, but it might be related to the CommandLine library.
About a verbose mode, that's something not already implemented, but I thought about doing one with the CLI update (probably when I'll have some time for it).

[phuchptty]

That error happened a long time ago

This video crashed in 1:37 Cs_Inazuma_LQ1203105_YaeMiko_Girl
https://drive.google.com/file/d/1Y_EJTyUAh9njAhfQOu9rqmbIXwZfa97u/view?usp=sharing

[ToaHartor]

Alright thank you guys, I downloaded all the videos. I'll try to look into the mkv structure.

[ToaHartor]

Okay, so analyzing some of these files with mkvinfo -a, I notices that some blocks weren't correctly made. For instance in the cutscene Cs_Sumeru_LQ130111201_EnODeGa_Boy, it prints this :

| + Simple block: track number 1, 1 frame(s), timestamp 00:00:31.833000000
|  + Frame with size 159073
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:31.867000000
|  + Frame with size 39482
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:31.900000000
|  + Frame with size 29858
| + Simple block: track number 1, 1 frame(s), timestamp 00:00:31.933000000
|  + Frame with size 30435
| + (Unknown element: DummyElement; ID: 0xff size: 3)
| + (Unknown element: DummyElement; ID: 0x4f00 size: 9)
| + (Unknown element: DummyElement; ID: 0xc0 size: 512515)
| + (Known element, but invalid at this position: Reference block; ID: 0xfb size: 14335)
| + (Unknown element: DummyElement; ID: 0x72f3 size: 14334)
| + (Known element, but invalid at this position: Chapter display; ID: 0x80 size: 121)
| + (Unknown element: DummyElement; ID: 0xfc size: 40)
| + (Unknown element: DummyElement; ID: 0xfe size: 8)
| + (Unknown element: DummyElement; ID: 0xfe size: 97)
| + (Known element, but invalid at this position: Reference virtual; ID: 0xfd size: 16130)
| + (Unknown element: DummyElement; ID: 0xff size: 27)
| + (Unknown element: DummyElement; ID: 0xff size: 3077)
| + (Known element, but invalid at this position: Codec state; ID: 0xa4 size: 128)
| + (Unknown element: DummyElement; ID: 0x6cfd size: 65197)
| + (Known element, but invalid at this position: Block addition ID name; ID: 0x41a4 size: 73)
| + (Unknown element: DummyElement; ID: 0x7f82 size: 13436)
| + Block group
|  + (Unknown element: DummyElement; ID: 0x365b23 size: 70)
| + (Unknown element: DummyElement; ID: 0x21f1d7 size: 4216)

[ToaHartor]

Okay, with more analysis, I found that blocks with a size of 0xFF were considered invalid, as apparently the matroska format considers that it indicates an unknown size. Since block size is calculated as varint, I have to tweak this function a bit I think.

[ToaHartor]

Looks like some good news (up is the defective internal engine, down is using the fix)

I'm trying to see if the merging performance is correct, then I'll consider making a new bugfix release.

[ToaHartor]

Closing the issue, as it should be resolved in the release 0.4.1.
Feel free to tell me if it happens again.