Problem with subtitles that are merged using external timestamp files

Help
1

Dear all,

I am currently trying to convert a series of M2TS files to one MKV file using external timestamp files. In the resulting file, there are no problems with the audio and video tracks, but a issue with the subtitle track:

The first subtitle is not removed (“undisplayed”) at the expected time; it should last about two seconds, but in fact is displayed for minutes until the next subtitle replaces it. I have double checked that the second timestamp in the external subtitle timestamp file designates the time when the first subtitle should disappear. I also have verified that the value of this second timestamp is correct (I’ve used BDSup2Sub++ and SubtitleEdit for this verification).

The other subtitles (there are 489 of them) are working correctly; the issue is only with the first subtitle. Furthermore, if I let mkvmerge create the MKV file from the M2TS files without external timestamp files, the issue does not exist.

Could the problem be due to a bug (I guess that it’s only a few people who use external timestamp files with subtitles)? Or are there special requirements for the first subtitle in a subtitle track? Does the PTS of subtitles (display or “undisplay”) need to coincide with the PTS of a video frame?

Thank you very much in advance, and best regards!

P.S. If necessary, I could upload the first part of the M2TS file in question somewhere (along with the timestamp file) and send the link via PM to interested parties. The M2TS file is about 15 GB, but the problematic (first) subtitle is at second 42, so I guess it would be sufficient to upload the first 200 MB.

2

Unfortunately all currently supported timestamp file formats are really unsuited for subtitles. The problem is how the duration of each frame is calculated: it’s always calculated from the timestamp files, not from the bitstream information available in the subtitles themselves.

For timestamp formats 1 & 3 the duration is derived from the currently active number of frames per second; for formats 2 & 4 it’s simply the difference between the current & the following timestamps.

From your description it sounds like you use v2. That matches what I’ve just described: the subtitles being visible until the next one starts means that their duration is exactly the next frame’s timestamp - the current frame’s timestamp.

What would help is a potential extension to the v2 format so that each line contains an optional second number that, if present, would be used as the frame’s duration. Not that hard to implement, but simply not implemented yet.

You might be able to finagle the v1 format into something you can use:

  • one line per subtitle frame
  • start & end frame are the current subtitle number (end frame mitght be + 1, not sure)
  • the third number (the FPS field) should then be the inverse of the duration (1 / duration_in_seconds)

Haven’t tried this, though.

3

Oh wait, what I proposed for v1 won’t work as the timestamps of the frames are calculated based on the duration. Bummer.

So… no currently supported timestamp format works well (enough) for subtitles.

4

As always, thank you very much for your time and for posting a reply that even newbies can understand. Of course, you are right: I am using timestamp file format v2.

I have understood that a subtitle disappears either if has been shown for the time determined by the duration value or if the next subtitle appears and “overwrites” it. Given that, there is one thing that I don’t understand:

In the file in question, no subtitle seems to have a duration attribute. Instead, all subtitles are shown for a few seconds until they disappear because the next “subtitle” appears, which simply is an “empty” image.

Consequently, in my test file, there are 980 subtitle frames for the 490 subtitles. When the frame numbering starts with 1, every frame with an odd number makes a subtitle appear and the next frame (with an even number) makes that subtitle disappear. The PTS of each frame with an even number is always only a few seconds behind the PTS of the corresponding frame with an odd number, while the time gap between frames with odd number (that is, the time gap between the visible subtitles) vastly varies.

The PTS timestamps of the 980 frames are in the timestamp file, and this produces the expected, correct result for all subtitles except the first one.

Therefore, I have looked into the first subtitle’s data quite thoroughly and could not find anything special about it. It is the same principle as with the other subtitles. There is a frame at 42.xxx which makes the subtitle appear (this is the first subtitle frame in the input file) and a corresponding frame at 44.xxx which should make it disappear (this is the second subtitle frame in the input file). But the first subtitle remains visible for about 62 seconds, while it should be visible for about 2 seconds.

These two timestamps mentioned above (42.xxx and 44.xxx) are the first two timestamps in the timestamp file. Nevertheless, when playing the MKV file, the first subtitle does not disappear when the second timestamp is passed.

Perhaps I’ll try whether it’s the same with one or two other input files. If yes, this would be somehow suspicious.

Best regards, and thanks again.

5

Different subtitle formats work different. Most formats (including all text formats) only have a single frame per piece of content to display. This frame has an associated duration. It’s the player’s job to display this one frame’s content only for the duration given.

Certain other formats, mostly HDMV Presentation Stream Graphis (also called “PGS”) as found on Blu-rays use a different approach: here each “show this piece of content” frame is accompanied by a very small “clear all displayed content” frame that happens later. The player can therefore hide the content either when the duration associated with the “show content” frame is exceeded or whenever it encounters a “clear content” frame.

From Matroska’s point of view both count as individual frames, each with their own timestamp (and therefore with their own entry in an external timestamps file).

6

Thank you very much again!

Yes, the subtitle stream in question is a HDMV_PGS stream. I believe that I have understood how it works in general, but actually that is my problem. For the first subtitle in that stream (and all others), there is a “show this content” frame and a “clear content” frame. But for the first subtitle (only), the “clear content” frame effects nothing, and I can’t find out why. All other subtitles work as expected.

I really can’t understand this since every time I was using the external timestamp files I had verified that the “show this content” frame timestamp and the “clear content” frame timestamp for the first subtitle were in the external timestamp file for the subtitle.

Plus, the problem with the first subtitle appears only if there is an external timestamp file for the subtitle track. If I mux the MKV file from the M2TS file without external timestamp files, there is no problem with the first subtitle.

To my regrets, I could not investigate yet whether the same issue arises with other M2TS input files, too. As soon as I have done that, I’ll report back.

Are you eventually interested in investigating the issue? In this case, I would provide you the first 200 MB of the M2TS input file, the timestamp files I have created and the command lines I have used.

Best regards, and thanks a lot.

7

Sorry, but I’m not really interested. I don’t have any particular insight into how players handle PGS subtitles in particular and why it plays out the way it does.