Thanks WebCaptioner. Hope you can let us know when you get the upgrades added to Web Captioner.

Thank you GeorgeR.

I support a church that uses ProPresenter for lyrics which is sent to two 16x9 ft screens via Fullscreen in vMix. Yesterday a thought popped into my head (can be scary :-)) - closed captioning for the sermon, announcements etc. would be a great addition. After testing https://webcaptioner.com/ at home I decided to go to the church to test this using the their audio system input set up for vMix live stream broadcasting. I used an screen input selecting screen 2 to capture the browser which I had sized to show only two lines of text. Then I cropped to fit and moved it down for lower thirds. Since the background is black I was able to use luma key to make the background transparent. Then all I had to do was select an overlay. BTW - this same set up used for ProPresenter but I don't need to position the input because the screen is full 16 x 9 black and ProPresnter sends the lyrics lower thirds already.

The Associate Pastor, who gave the sermon, is from Kenya and has a heavy accent. For the most part the program was right on. I compared this to closed captioning on a YouTube video with someone who has a heavy accent at it was about the same. For people who's first language is English this seems to work 99% of the time. There were a few times when it skipped phrases.

WOW - this could provide some great added value benefits including the fact that a live recording could already have closed captioning embedded.

Wish list - suggestions

- add lower thirds formatting (16 x 9 aspect ratio) with no more than two lines at the bottom of a black screen and allow font size adjustment.

- OR parse the ASCII text to a vMix title input (via data sources). Then you could do all the formatting via Title Editor. If the data source could be saved then it could also be used for transcription edits, a major time saver.

- stand alone software (like ProPresenter) that sends the data to an NDI feed (eliminates a screen capture input).

Today I added some new options for changing the text layout that might make doing lower thirds easier: https://webcaptioner.com...captioner-a-custom-look/

WebCaptioner is fascinating. Far from perfect (who is that by the way), but at many times pretty good. I've done some tests, which together with my thoughts/findings are posted in the WebCaptioner User group on Facebook.

I guess it's up to the developer of WebCaptioner to provide direct integration into vMix (via the API). In any case as others have pointed out here already, it's relatively easy to add WebCaptioner's output into a live production.

Side note, WebCaptioner now supports up to 40 languages (ie understands, not translates).

Maybe, but the updates must be made many times per second, and I don't know how often vMix currently polls the source.


I would rather say that most OVP (CDNs) does not support closed captions. Although certainly many of the dominant ones do offer support for *on demand video*. Very few does it for live streams. YouTube is one that does. The offer two ways. One is by "Sending Embedded 608/708 Closed Captions", which to my best knowledge requires expensive extra hardware for the caption ingestion. The second way is by "Enabling Closed Captions". As far as I know this works via their proprietary protocol that is supported by a few, and with one exception rather expensive, cloud services. The exception is StreamText. But so far we have not been able to use them with a decent sync between speech and text. Over a year ago we tried to add live captions this way via YouTube's API, but we never got it working. And YouTube never answered our questions.


Agree indeed, but it's as often a budget issue. Our customers typically regards it is sufficient to send only one stream with open captions (added as a lower third on the video). Sometimes we send two strewams, one without captions, and one with. In the case of YouTube we use their multi-camera support to allow the viewer easy selection of which stream to watch. Or we use javascripts to switch between different live web video players.

Of course I would love to see live video production tools like vMix being able to use the 608/708 method. But we would still be dependent on OVP/CDN support for that.

Another thing to keep in mind is the (time coded) recording of the captions generated live. Besides the need to at least afterwards edit major errors, the captioning also benifits from manual resync afterwards. At least when the output from human Speech To Text Recorders tend to be slighly delayed relative to what is being said. (Well, sometimes they finish the sentence before the speaker does).

On a side note, here are a couple of cases we have published:
- Captioning with the help of UN Web TV
- Captioning and sign language for a Public Service event

Another update, vMix integration still in the works (some errenous behavior in the Title's text scroll after a while).

Meanwhile, we used Web Captioner for real at a breakfast seminar a week ago. Which I have documented at westreamu.se/kunder/case-automatic-live-subtitles-with-web-captioner/

I know this is very old thread, but thought I'd add some information since I stumbled across it.
Sadly, WebCaptioner has been sunsetted by the publisher; but he's released the code to the public domain.
At least one company (Maestra) has picked it up and has basically the same system available:
https://maestra.ai/tools/web-captioner/app/