Subtitle Generation and Translation

Case 1: Generating subtitles for an existing file

1. Zero code generation in the console

1. Log in to the MPS console and go to Schemes. Add a content discovery action (which can be used for speech recognition and translation). Click

and select a template.

MPS offers the following preset content discovery templates:
Template ID
Chinese speech recognition
English speech recognition
Chinese speech to Chinese and English text
English speech to English and Chinese text
If the preset templates do not meet your requirements, please submit a ticket. We can customize a template for your needs.
2. Enable the scheme. Wait about five minutes for the configuration to take effect, and upload your file to the trigger bucket.

3. After the task is executed, a VTT file will be generated and saved to the output directory.

Chinese subtitle sample:

Chinese and English subtitle sample:

2. Generation via an API

1. Call ProcessMedia, specifying the input file information and the scheme ID.
    "InputInfo": {
        "Type": "COS",
        "CosInputInfo": {
            "Bucket": "facedetectioncos-1251132611",
            "Region": "ap-guangzhou",
            "Object": "/video/123.mp4"
    "ScheduleId": 20073,
    "Action": "ProcessMedia",
    "Version": "2019-06-12"
2. If you have configured a callback address, for details about the data returned, see ParseNotification.

3. Embedding the subtitles into the video

Create a transcoding task, setting SubtitleTemplate to the VTT file generated in step 1 or step 2. For details, see Data Types.
    "MediaProcessTask": {
        "TranscodeTaskSet": [
                "Definition": 206390,
                "OverrideParameter": {
                    "Container": "mp4",
                    "RemoveVideo": 0,
                    "RemoveAudio": 0,
                    "VideoTemplate": {
                        "Codec": "libx264",
                        "Fps": 30,
                        "Bitrate": 2346,
                        "ResolutionAdaptive": "close",
                        "Width": 1920,
                        "Height": 0,
                        "Gop": 0,
                        "FillType": "black"
                    "AudioTemplate": {
                        "Codec": "libmp3lame",
                        "Bitrate": 0,
                        "SampleRate": 32000,
                        "AudioChannel": 2
                    "SubtitleTemplate": {
                        "Path": "https://lily-1256342427.cos.ap-nanjing.myqcloud.com/mps_autotest/subtitle/1.vtt",
                        "StreamIndex": 2,
                        "FontType": "simkai.ttf",
                        "FontSize": "10px",
                        "FontColor": "0xFFFFFF",
                        "FontAlpha": 0.9
    "InputInfo": {
        "Type": "URL",
        "UrlInputInfo": {
            "Url": "https://lily-1256342427.cos.ap-nanjing.myqcloud.com/mps_autotest/subtitle/123.mkv"
    "OutputStorage": {
        "Type": "COS",
        "CosOutputStorage": {
            "Bucket": "lily-1256342427",
            "Region": "ap-nanjing"
    "OutputDir": "/mps_autotest/output2/",
    "Action": "ProcessMedia",
    "Version": "2019-06-12"

Case 2: Generating subtitles for a live stream

1. Configuring a callback that returns the text in real time.

1. Call ProcessLiveStream to start a speech recognition task using a preset subtitle generation template.
    "Url": "http://5000-wenzhen.liveplay.myqcloud.com/live/123.flv",
    "AiRecognitionTask": {
        "Definition": 10101
    "OutputStorage": {
        "CosOutputStorage": {
            "Bucket": "6c0f30dfvodgzp251000800-10022853",
            "Region": "ap-guangzhou-2"
        "Type": "COS"
    "OutputDir": "/6c0f30dfvodgzp251000800/0d1409d34565514956657373652/",
    "TaskNotifyConfig": {
        "NotifyType": "URL",
        "NotifyUrl": "http://xxxx.qq.com/callback/qtatest/?token=xxxxxx"
    "Action": "ProcessLiveStream",
    "Version": "2019-06-12"
2. Receive the callback. For details about the data returned, see ParseLiveStreamProcessNotification.

2. Embedding the subtitles into the live stream

1. Make sure you have activated Cloud Streaming Services and MPS.
2. Submit a ticket to enable the live subtitling feature for your account.
3. The publishing and playback logic remains unchanged. You can get subtitled streams simply by adding a suffix that specifies the subtitle language to the playback URL.
For example, suppose the stream ID is 5000_test.
Original stream (with no subtitles)
Chinese speech to Chinese text subtitles
Chinese speech to Chinese and English text subtitles
Chinese speech to English text subtitles
English speech to English text subtitles
English speech to English and Chinese text subtitles
If you want to use subtitles in other languages, please submit a ticket.