Bug with merging PGS subtitles if timestamp files are used

Help
1

Dear all,

at first, I apologize that I can’t report this bug on Gitlab. I tried to register there, but they force me to give them my phone number for that purpose. This definitely won’t happen.

I am using the most recent version 84.0 of MKVToolnix / mkvmerge under Windows 10 x64 Enterprise. The following bug report is related to this post.

Bug description / wrong behaviour

Using mkvmerge on the command line, I am converting M2TS files to MKV files, choosing only certain tracks, and using external timestamp files. The source files contain PGS subtitle tracks that are also merged into the result file using external timestamp files. The timestamp files are format 2.

When I play the resulting file in MKV players (tested MPC-HC and VLC), everything works except the PGS subtitles. Some subtitles remain on the screen way too long. That is, a subtitle may remain on the screen for minutes where it actually should remain only for seconds.

This problem does not appear when I let mkvmerge create the result file without external timestamp files for the subtitle tracks.

I have analyzed the cause of problem as far as it is possible for me (being a complete newbie in that area). The problem does nothing have to do with players, but seems to be due to a bug in mkvmerge. I’ll first describe it on a high level and then show some details from a test I did today.

Usually, displaying a PGS subtitle comprises two actions. First, the subtitle appears, and then disappears again. Disappearance of a subtitle is effected by letting the next subtitle appear. This next subtitle can be (and in most cases, is) an “invisible” (transparent) subtitle. The duration of a subtitle then is just the difference between the PTS of that subtitle and the PTS of the next subtitle.

According to a statement from the developer of mkvmerge, that’s also the way that mkvmerge follows to compute a PGS subtitle’s duration.

Now let’s consider some test results. I have a M2TS input file from which I would like to merge tracks 0 (video), 1 (DTS-HD MA audio) and 7 (PGS subtitles) into a MKV file. The input file is named input.m2ts, the MKV output file is named output.mkv

Let us focus on a PGS subtitle that appears roughly at second 85 and disappears roughly at second 92. To analyze what happens, I use the mkvinfo tool and look at the respective clusters. In the following screenshots, track 1 is the video track, track 2 is the audio track and track 3 is the PGS subtitle track.

Test 1: no external timestamp files

The command line:

mkvmerge -o output.mkv -d 0 -a 1 -s 7 input.m2ts

After loading output.mkv into the mkvinfo tool, we can find the following two clusters that relate to the subtitle in question:

Clipboard01

Clipboard02

A few things to note here:

  • The subtitle is encoded in simple blocks.
  • The first screenshot shows the cluster where the subtitle appears (PTS 85.586).
  • The second screenshot shows the cluster where the subtitle disappears (PTS 92.551).
  • Note that there are no more subtitles in the source file. That is, the second screenshot does not show the PTS of the next visible subtitle, but the PTS of a “subtitle” that makes the current subtitle disappear.

That is how things are expected to work: One subtitle frame that makes the subtitle appear with PTS 1 (85.586), one corresponding subtitle frame that makes it disappear with PTS 2 (92.551), where PTS 1 minus PTS 2 is the subtitle duration.

[ As a side note, mkvmerge calculates the subtitle PTS as 85.586, but a few other well-respected tools say that it is 85.669. This may be due to a general offset, which would be OK. However, the second PTS that mkvmerge calculates (that is, the time when the subtitle disappears, 92.551) is exactly the same that those other tools report, too. Therefore, the duration of the subtitle probably is not as intended. But let’s focus on the actual problem and ignore this one for the time being. ]

Test 2: External timestamp file for the PGS subtitle track

For the subtitle in question, there are the following timestamps in the external timestamp file:

85669
92551

The first one is the PTS where the subtitle should appear, the second one is the PTS where it should disappear. The command line to create the MKV file, using the external subtitle timestamp file with name 7.timestamp, is:

mkvmerge -o output.mkv --timestamps 7:7.timestamp -d 0 -a 1 -s 7 input.m2ts

After loading output.mkv into mkvinfo, we can find the following:

Clipboard03

There is no second screenshot in this case, because this is now the only place that refers to the subtitle in question. In other words, there is no later cluster where this subtitle disappears by replacing it with a transparent subtitle, or where track 3 is referenced in any way at all. This may or may not be a problem - I really don’t know.

The first real problem is that mkvmerge does not respect the PTS that is in the timestamp file. The PTS for appearance of the subtitle in the timestamp file is 85.669, but the screenshot shows that mkvmerge keeps using 85.586 (which is the value from the previous test without timestamp file). There is definitely no timestamp with this value in the timestamp file.

The second real problem is that mkvmerge calculates the duration in a wrong way. From the screenshot, we see that it is very small (84 ms). I needed a while until I understood where that value comes from, but then it became clear: It is the PTS from the timestamp file (85.669) minus the PTS that mkvmerge uses instead (85.586) plus 1. The plus 1 may be due to rounding.

Test 3: Same as test 2, but with other PTS value in the timestamp file

I tried to verify my findings and therefore in the timestamp file replaced 85.669 by 85.586 (the value mkvmerge insisted on anyway). The result was as expected:

Clipboard04

Obviously, the duration again has been calculated according to the formula mentioned above: (PTS from the timestamp file (85.586)) minus (PTS mkvmerge uses (85.856)) plus 1 = 1.

Summary

When merging a MKV file, mkvmerge does not behave as intended and described when using external timestamp files for PGS subtitle tracks. Notably, timestamps from the timestamp file may be ignored (in that they do not become the subtitle PTS in the MKV file), and the duration of a subtitle in the MKV file may be not equal to the PTS of the next subtitle minus the PTS of the subtitle. I believe that both problems are due to a bug.

I’ll now try to upload the input file, the two variants of the timestamp file and the three output files to your sftp server. I’ll put the files into the directory Unico.

Please note that everything I have written above pertains only to the subtitle I have investigated. I’ll conduct further tests to find out whether the other (previous) subtitles work correctly.

Thank you very much for your great software, your time and your support!

2

I now have investigated the problem further.

In the respective subtitle track in the source file, there are 13 other subtitles before the one I have analyzed above. All of those previous subtitles work as expected in the player, and mkvinfo shows that mkvmerge has assigned them exactly the PTS that is in the timestamp file, and has calculated the correct duration in each case.

So things start to go wrong with the 14th subtitle, which has not been assigned the PTS from the timestamp file, but instead has been assigned a PTS the source of which is not known, and has been assigned the wrong duration, as described above.

Since the 14th subtitle is the last one in that input file subtitle track, perhaps that’s where the problem lies. Eventually something goes wrong with the last subtitle in each track when using external timestamp files?

Thank you very much again!

3

In the meantime, I have conducted another test:

I first have converted the M2TS input file to a MKV file, including all tracks, using mkvmerge and not using an external timestamp file.

Then I have repeated the tests I have described in the first post, with the only difference that I now have used that new MKV file as input file.

The test results were the same as with the M2TS input file. This means that the bug shows independently of the input file format; at least, we probably can conclude that it is not due to a bug in the M2TS demultiplexer.

Thank you very much again!

4

In my second post in this thread, I have written that the source of the PTS that mkvmerge takes as the start of the 14th subtitle is not known. It seems that I have been blind:

After looking again to the timestamp file, it becomes clear that it is the “out time” of the 13th subtitle. Sorry that I didn’t see this earlier.

To summarize again, the first 13 subtitles are encoded correctly, but for the 14th subtitle wrong data is used. Explained by means of the timestamp file (formatting and omissions for clarity):

# timestamp format v2

22105.4111111111 
26901.8666666667 <- 1st subtitle, correct in the MKV, PTS 22105, duration 4797 ms

40373.6666666667
47297.2444444444 <- 2nd subtitle, correct, like first one

48590.2
52302.2444444444 <- 3rd subtitle, correct like first one

... further subtitles, each of them correct in the MKV, like first one ...

81039.2888888889
85585.5          <- 13th subtitle, correct in the MKV, like fist subtitle

85669            <- 14th subtitle, not correct: In the MKV, the PTS ("in time") is not 85669, but is 85586 (which is the "out time" of the 13th subtitle),
92551               and the duration is 84 ms, which is the *"in time"* from the timestamp file (85669) minus the "out time" of the 13th subtitle (85585).

To summarize, the first 13 subtitles are encoded correctly in the MKV file. That is, the PTS in the MKV is exactly the respective subtitle “in time” from the timestamp file, and the duration in the MKV is exactly the respective “out time” from the timestamp file minus the respective “in time” from the timestamp file.

However, the 14th subtitle goes wrong. Here, the PTS in the MKV file is the subtitle “out time” of the 13th subtitle from the timestamp file, and the duration in the MKV is the “in time” for the 14th subtitle from the timestamp file minus the “out time” of the 13th subtitle from the timestamp file.

That the first 14 subtitle are encoded correctly, but the 14th is not encoded correctly may be due to the fact the 14th subtitle is the last subtitle in the respective PGS subtitle track in the input file.

Hope this helps :slight_smile:

Thank you very much again!

5

I really appreciate the analysis. That being said, can you please open a proper issue for it over on Gitlab? That’s where I track bugs. Otherwise chances are I’ll forget about it again.

Thanks.

6

Thank you very much for reading my long posts.

In my first post above, I have apologized that I couldn’t report the issue via Gitlab, because Gitlab asked my for my phone number (or alternatively, credit card number) during the registration process; of course, I won’t give that kind of personal data to Gitlab, so I had to abort the registration.

Motivated by your comment, I just have tried the registration again, and guess what: the registration now was possible without giving them my phone number. I don’t have any clue why they didn’t let me register 7 days ago, but now did.

Well, I’ll then try to copy this report to Gitlab.

Thank you very much again!

7

OK, issue has been created:

Thank you very much for all your hard work!