I have been trying to consolidate a bunch of servers recently, and one of the ones that I am looking to sunset is a box that has been used as a BD ripping and media encoding box. The reason it HASN’T been sunset yet is that, well, there are a bunch of things on it that have been ripped but not yet encoded for playback on an AppleTV and I needed to get on that.
Aug 18, 2020 FFmpeg can easily extract embedded subtitles from videos. This command will grab the default subtitle track and export it as a srt file: ffmpeg -i inputfile out.srt What if we want to get a different subtitle? Subtitles are pretty much the same. The subtitles will be printed in the info and then you can extract them, similar to: ffmpeg -threads 4 -i VIDEO.mkv -vn -an -codec:s:0.2 srt myLangSubtitle.srt. 0.2 is the identifier that you have to read from the info. SOLUTION 3: The following script extracts all audio streams from files in current directory.
Anyway, that’s in process, and I was deleting a bunch of source files that had already been encoded and that’s when I ran across Maken-Ki.
If you’ve never seen it, it’s a really forgettable show whose chief draw is a violent pettanko who may or may not be a dragon. I kind of regret watching both seasons of it, and I am very unlikely to ever watch it again, but finding the source files made me remember something about the series that had REALLY annoyed me; which was a frustrating typo in the subtitles in one of the episodes.
So I thought to myself, how hard could this be to fix? I’ve got ffmpeg, which is like a Swiss army knife on steroids when it comes to video file manipulation, I’m sure I can sort it out.
What could possibly go wrong?
My first step was to look at how the original file was laid out, which you can do just by throwing it into ffmpeg, like so:
And this gives a lot of output:
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv), 1920×1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 23.98 tbc (default)
Metadata:
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_WRITING_APP-eng: mkvmerge v30.1.0 (‘Forever And More’) 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-02-10 22:33:00
BPS-eng : 2583237
DURATION-eng: 00:23:42.046000000
NUMBER_OF_BYTES-eng: 459185284
NUMBER_OF_FRAMES-eng: 34095
Stream #0:1(eng): Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
Metadata:
title : English 5.1 channel AAC
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_WRITING_APP-eng: mkvmerge v30.1.0 (‘Forever And More’) 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-02-10 22:33:00
BPS-eng : 440524
DURATION-eng: 00:23:42.036000000
NUMBER_OF_BYTES-eng: 78305206
NUMBER_OF_FRAMES-eng: 66658
Stream #0:2(jpn): Audio: aac (LC), 48000 Hz, stereo, fltp
Metadata:
title : Japanese 2.0 channel AAC
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_WRITING_APP-eng: mkvmerge v30.1.0 (‘Forever And More’) 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-02-10 22:33:00
BPS-eng : 179997
DURATION-eng: 00:23:42.087000000
NUMBER_OF_BYTES-eng: 31996579
NUMBER_OF_FRAMES-eng: 66661
Stream #0:3(zxx): Subtitle: ass (default)
Metadata:
title : Signs/Karaoke [Hatsuyuki] – [WHW]
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_WRITING_APP-eng: mkvmerge v30.1.0 (‘Forever And More’) 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-02-10 22:33:00
BPS-eng : 470
DURATION-eng: 00:23:04.820000000
NUMBER_OF_BYTES-eng: 81502
NUMBER_OF_FRAMES-eng: 601
Stream #0:4(eng): Subtitle: ass
Metadata:
title : [Hatsuyuki] – [WHW]
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
_STATISTICS_WRITING_APP-eng: mkvmerge v30.1.0 (‘Forever And More’) 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-02-10 22:33:00
BPS-eng : 562
DURATION-eng: 00:23:37.210000000
NUMBER_OF_BYTES-eng: 99571
NUMBER_OF_FRAMES-eng: 897
Stream #0:5: Attachment: ttf
Metadata:
filename: HeyGorgeous.ttf
mimetype: application/x-truetype-font
Stream #0:6: Attachment: ttf
Metadata:
filename: INCOLHUA_R.ttf
mimetype: application/x-truetype-font
I’ve cut a lot out of this because what I want to know about is the Streams, which are all of the different components that make up the video file. This file has one video stream, two audio streams, two subtitle streams and a bunch of embedded fonts. I’ve cut off all the fonts after the first couple because the output was already long enough.
The English audio track and the “Signs/Karaoke” subtitle track are unnecessary, so let’s get rid of those first:
Breaking this down:
just tells ffmpeg which file to read.
tells it to copy all streams from file 0 to the output file. ffmpeg starts counting everything from 0. It’s annoying but you get used to it. Except, we don’t want the English audio or signs and karaoke subtitles, so we use two more map commands to exclude those.
is telling ffmpeg to drop streams 1 and 3 from file 0.
Ffmpeg Remove Subtitle
tells ffmpeg to copy the remaining audio, video, and subtitle streams without transcoding them. This is very fast and doesn’t affect quality.
Finally, the last thing on your ffmpeg command line is the filename you want ffmpeg to write to. “tt.mkv” is just my regular shorthand for temporary output files.
This cheerfully made a new mkv file, and I threw it into VLC to confirm that the subtitles and audio I wanted were present. It was also 80 megabytes smaller without the English audio so you could use this to reduce the size of mkv files where you don’t care about some of the languages.
Then I needed to separate the subtitle file from the mkv file so I could make changes.
Again I’m using the -map -0:2 command to exclude a stream, in this case the subtitle stream. This gives me a file with just video, audio, and fonts.
I also extract the subtitle track to a text file with a .ass extension, like so:
ffmpeg is smart enough to know that you can’t put video or audio into a .ass file so it drops those and we’re left with just the script.
After that, I made my changes to the script and needed to mash it all back into one file.
Two -i commands to tell ffmpeg that it needs to read from two input files, two -map commands to tell it to take all streams from file 0 (the mkv file) and all streams from file 1 (the subtitles file), copy them without transcoding and put everything into another file called ttt.mkv.
I am not particularly inventive with my file names.
I was then able to throw ttt.mkv through Handbrake and it gave me a lovely AppleTV-compatible .m4v file that I will probably never actually watch because Maken-Ki was not really worth a second watch, but damnit I fixed the typo and this was very important to me.
Though Himegami – that’s the violent pettanko’s name, had to look it up – WAS a hilarious character and it might be worth watching a couple episodes. Someday. When I get through all of the other shows in my queue.
Extract subtitles with ffmpeg from a .ts video file
This post chronicles my ultimately failed attempt to extract subtitles with ffmpeg / avconv from a .ts DVB-S video-file recorded from live television.
This is what ffmpeg has to say about the file I wanted to extract subs from
I started with a search for anyone else having tried that and found many pointers on stackoverflow and such, but all were referring to .srt subtitling. As it seems a lot of people are transcoding, remuxing and repackaging their anime, but only very few people try to extract subs from broadcasted video streams. For example, the ffmpeg docs provide this example.
My ffmpeg can decode and encode all we'd need, right?
Ffmpeg Extract Srt
Wrong! When you do a naive conversion with ffmpeg, extracting the dvb_sub program-stream from the .ts file and sending it to, let's say, an .srt file, you'll get this dreaded ffmpeg / avconv error: 'Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height'
The explanation behind this is: there are basically two formats in subtitling - image and text based - and most subs discussed on the web are text based, like .srt, .stl, .webvtt; but we here are facing an image-based sub format! (This user here (Is it possible to extract SubRip (SRT) subtitles from an MP4 video with ffmpeg?) had the same learning curve.)
The MPEG legacy has brought us here, I think, because on DVDs and obviously in .ts broadcast streams subs are in image-based formats, VobSub, dvb_sub, dvd_sub.
To see what I'm talking about, refer to vlc's comparison of subtitle formats, as Wikipedia doesn't have one.
For example, try something like this with a similar file:
ffmpeg -i 000.ts -map 0:0 -vcodec copy -acodec copy -map 0:s:1 -scodec dvdsub test.mkv
Just exchange dvdsub with srt for example, and you'll get the above mentioned error. What the command does, on the other hand, is transcoding the subsitles found in the second subtitle stream (streams are zero based, so '1' is the second; which is stream 0:6 in this file, and 0:5, the first sub-stream has teletext). It's beyond me what the actual difference between dvb_subtitles and dvd_subtitles is. But when you watch the result in vlc (mplayer has problems displaying them..), the quality of the subs has degraded. (But this detail only as a side-note, I think it's from DVB Subs being in 16bit BMP / PGM format, while the transcode wrote DVD Subs in 4 bit bitmaps, losing quality in alpha or similar).
With that learned and at least some working command for ffmpeg, I stumbled over this mailing-list post (Can extract DVB-Sub, cannot extract DVB-T), where someone had issues with dumping subtitles from a .ts file. And it gave me this command, which actually worked in dumping the raw subtitles-only stream to a file: ffmpeg -i file1.ts -vn -an -scodec copy -f rawvideo dvbsub.datThis post here discusses something similar. My variation of it was
ffmpeg -i 000.ts -map 0:0 -vcodec copy -acodec copy -map 0:s:1 -scodec copy -f rawvideo sub.data
But looking at the file with a hex editor, with me being a hex noob, brought nothing resembling a BMP or PGM file, any headers or structures I'd recognise.
So how can I extract image-based subtitles with ffmpeg? First I tried if piping a substream to an image format would work:
ffmpeg -i 000.ts -map 0:0 -vn -an -map 0:s:1 -scodec copy -f image2 sub_%03d.bmp
and this actually wrote many images but all were unusable. I don't know how ffmpeg actually chopped the data into files, based on timecode-subtitle-triggers?? I don't know. Finally giving up on ffmpeg, I asked the search engine if any other tools were able to extract image-based subtitles as rendered images/pictures from the video. Some claimed mencoder could do that and I actually found example commands, but none worked for me and all examples centered around DVD and VobSub format type of work, like writing .idx and .sub files from DVD etc.
This post then, although discussing a VOB workflow has pointers into the only feasible way of converting an image-based sub-stream into something text-based or into raw text. There the author used mencoder and a tool called vobsub2pgm and finally sends the resulting character images into an OCR solution. This post does something similar and uses tesseract for OCR. Ffmpeg can't do that, and so far only the other way round, encoding/rendering textual chars as images has been mentioned for ffmpeg in this ticket to add rasterization for sub transcoding.
Just before giving up, as I wasn't inclined to go a painful console-based path of trial and error with multiple tools just to extract some subtitles, I found that AviDemux offers a GUI tool to do just that, OCR'ing of image-based subs! Found in the '> Tools' menu, the older routine is called 'OCR (VOBSub -> srt)' and more recent avidemux builds have 'OCR (TS->srt)'.
Ffmpeg Subtitle Stream
Running this on my .ts file didn't work. If you're interested, this page has screenshots of the workflow, which is a bit cumbersome, as OCR is not perfect and you eventually have to edit what's being recognised.
All I got was a weird error 'backdoor) >> 16' and something which brought me to this thread, which once more mentioned a DVB tool called ProjectX. Despite the generic name, it's a very dedicated tool, focusing on inspecting and decoding DVB style .ts files as streamed by European broadcasters. And users on the forums say it's able to extract subtitles from the video mux.
Ffmpeg Subtitle Srt
And although it processed my .ts file, and printed all sorts of very involved looking things about packets, streams and elements found in the stream, I was not successful in properly targeting and extracting the 'sub-picture' teletext subtitles stream found in my MPEG transport stream file. And that's the end of it. Post a comment when you have tried something similar and can provide pointers.
Ffmpeg Extract Subtitle From Mkv
For keywords:
How to extract subs with avconv
Dump subtitles with ffmpeg
Extract subs with ffmpeg and write images / bitmaps / ocr
How to write subtitles from a video into a separate srt stl subtitles file