This support forum is no longer in use. Please use the Cloudinary Community at https://community.cloudinary.com/ to receive assistance from other members of the community and from Cloudinary's support team. For account-specific questions or if you believe that you've encountered a bug, please contact the Cloudinary team directly via the "submit a request" option on this support site.

Adaptive streaming (hls) with subtitles linked in the m3u8 playlist, is it achievable?

Follow
Avatar
DevOps Triggerz
  • Edited

Hello, I have created a upload preset that uses Eager Transformations: sp_4k/m3u8 to convert simple mp4 files to adaptive m3u8 playlist files. I'm then feeding a capable web video player (not the cloudinary player) the m3u8 file and it nicely streams adaptively to the network bandwidth and so on.

However, I am having troubles with subtitles. The video player is based on hls.js which should be capable of handling captions when included in the playlist file (the m3u8) but I can't find blog post, forum post or documentation about how to add captions to cloudinarys adaptive streaming playlists. Is it even possible? I know an obvious answer would be: "just use the cloudinary player and feed the subtitles separately" but it's simply not a option so that wont solve my current need unfortunately.

I hope that someone can shine some light to what needs to be done or whether this is even possible?

Thanks!

0

Comments

7 comments

Sort by Date Votes
  • Avatar
    Stephen Doyle

    Hi,

    Currently, the m3u8 playlists we create when using adaptive bitrate streaming do not include the option of adding a subtitle file to the playlist automatically.

    Depending on what you need, a possible workaround is to create your own .m3u8 playlist, referencing the .ts files already created by us, and then upload that playlist and use it when delivering the video to players that can't have the subtitles sent as separate configuration.

    For example, you could:

    • Download the m3u8 file for the video, or for one representation of the video, depending on what you need
    • Edit that m3u8 to add a reference to the subtitle file
    • Also edit the URL for each of the EXT-X-STREAM-INF entries to be absolute instead of relative (see note below)
    • Re-upload that m3u8 file back to the account, with `resource_type: raw`
    • Use that new m3u8 URL to deliver the video to the player

    (The URLs need to be updated because the delivery path of a raw file will begin with /raw/upload... but the videos need to be accessed via /video/upload, so it won't match if you're using the relative paths we add to the playlist by default)

    We do also have a feature request to add direct support for us creating an m3u8 with subtitles , and it may be added as a new feature in the future but I'm afraid I don't know when that may be.

    Another option may be to use our 'subtitle' transformation which takes an SRT or VTT file and overlays it onto the video, so it will be present in the actual video frames, and will be seen by all viewers: https://cloudinary.com/documentation/video_manipulation_and_delivery#adding_subtitles

    Regards,

    Stephen

    1
    Comment actions Permalink
  • Avatar
    DevOps Triggerz
    • Edited

    Thank you very much for a very thorough rely Stephen! Really well described 🥇

    0
    Comment actions Permalink
  • Avatar
    Stephen Doyle

    You're very welcome; please let me know if there's anything else we can help with, and you can also contact our support team directly via the "Submit a request" link on the top of this support site

    0
    Comment actions Permalink
  • Avatar
    DevOps Triggerz

    Thanks again, and yes there might be.

    I have tried manually constructing the m3u8 file like you described, and by following this article: https://mux.com/blog/subtitles-captions-webvtt-hls-and-those-magic-flags/

    When I run it on my local machine with VLC it actually streams the video nicely, so I think the absolute path part is done correctly. I'm not too sure if I am referencing the subtitle correctly though, because I can't choose subtitles in VLC still, but that could also be because VLC might not support it. Or maybe it's not possible at all referencing vtt subtiltles in hls manifest and need to be a different subtitle format, that's not 100% clear to me.

    Anyway. I'm not allowed to upload the file to cloudinary (at least from the dashboard), it responds with "invalid file" I am not sure if that is because the dasoboard upload is always expecting an actual media file? In which case I need to find a way to upload it using the API so I can specify that it should be resource_type: raw, as you mention? or, it might be because my files is constructed wrong, and cloudinary detects it.

    Would you mind taking a look at this manifest and see if you can spot something obviously wrong?

    #EXTM3U
    #EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="sub1",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog",NAME="English",AUTOSELECT=YES,DEFAULT=NO,FORCED=NO,LANGUAGE="en-US",URI="https://res.cloudinary.com/triggerz-eu-cld/raw/upload/v1605598514/HabitDrivers/Videos/HabitDrivers_for_Teams.mp4.vtt"
    #EXT-X-STREAM-INF:BANDWIDTH=678000,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1152x720,SUBTITLES="sub1"
    https://res.cloudinary.com/triggerz-eu-cld/video/upload/c_limit,w_1920,h_1080,vc_h264:high:4.0,br_8500k/v1605597336/HabitDrivers/Videos/HabitDrivers_for_Teams.m3u8
    #EXT-X-STREAM-INF:BANDWIDTH=659000,CODECS="avc1.4D401F,mp4a.40.2",RESOLUTION=1152x720,SUBTITLES="sub1"
    https://res.cloudinary.com/triggerz-eu-cld/video/upload/c_limit,w_1280,h_720,vc_h264:main:3.1,br_5500k/v1605597336/HabitDrivers/Videos/HabitDrivers_for_Teams.m3u8
    #EXT-X-STREAM-INF:BANDWIDTH=413000,CODECS="avc1.4D401F,mp4a.40.2",RESOLUTION=864x540,SUBTITLES="sub1"
    https://res.cloudinary.com/triggerz-eu-cld/video/upload/c_limit,w_960,h_540,vc_h264:main:3.1,br_3500k/v1605597336/HabitDrivers/Videos/HabitDrivers_for_Teams.m3u8
    #EXT-X-STREAM-INF:BANDWIDTH=349000,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION=576x360,SUBTITLES="sub1"
    https://res.cloudinary.com/triggerz-eu-cld/video/upload/c_limit,w_640,h_360,vc_h264:baseline:3.0,br_2m/v1605597336/HabitDrivers/Videos/HabitDrivers_for_Teams.m3u8
    #EXT-X-STREAM-INF:BANDWIDTH=254000,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION=432x270,SUBTITLES="sub1"
    https://res.cloudinary.com/triggerz-eu-cld/video/upload/c_limit,w_480,h_270,vc_h264:baseline:3.0,br_800k/v1605597336/HabitDrivers/Videos/HabitDrivers_for_Teams.m3u8
    #EXT-X-STREAM-INF:BANDWIDTH=196000,CODECS="avc1.42C01E,mp4a.40.2",RESOLUTION=320x200,SUBTITLES="sub1"
    https://res.cloudinary.com/triggerz-eu-cld/video/upload/c_limit,w_320,h_240,vc_h264:baseline:3.0,br_192k/v1605597336/HabitDrivers/Videos/HabitDrivers_for_Teams.m3u8
    0
    Comment actions Permalink
  • Avatar
    Victor Li

    Hi there,

    Just to clarify, your issue is two-fold?

    1) The new .m3u8 file you constructed plays in VLC locally but not with subtitles, and

    2) You are receiving an "invalid file" error when uploading the new .m3u8 file through our UI to your account?

    Could you try uploading the files via our Upload API with `resource_type: raw` or alternatively, open a ticket with us and attach them so that we can take a look at both issues on our end? 

    Regards,
    Victor

    0
    Comment actions Permalink
  • Avatar
    DevOps Triggerz
    • Edited

    Hi Victor.

    I have now written an application and uploaded the m3u8 file successfully via the API.

    File: https://res.cloudinary.com/triggerz-eu-cld/raw/upload/v1613557055/HabitDrivers_for_Teams-subtitile-test.m3u8

    However, I think my handcrafted m3u8 playlist (pasted above) is constructed with fault, as I do not see subtitles when I play the stream. If you prefer me to open a ticket I will do that.

    Regards, Rasmus


    0
    Comment actions Permalink
  • Avatar
    Stephen Doyle

    Hi Rasmus,

    The format of your m3u8 file looks OK to me, and the video plays for me from this m3u8, but I also don't see the subtitles, as you say. I checked this in VLC, though, and I'm not sure 100% sure if VLC certainly supports this; it did appear to load the .vtt file from Cloudinary, but I didn't see any errors or warnings, it just didn't render anything

    In which player were you checking, and did you see the player suggest that subtitles were available? were any errors thrown?

    Regarding the other error that you mentioned, it's not currently possible to force the resource type to 'raw' when uploading via Media Library: you'll need to use the API to upload the .m3u8 if you don't want us to try and auto-detect the format (which will fail in this case because we treat .m3u8 as a video format)

    Thanks,
    Stephen

    0
    Comment actions Permalink

Post is closed for comments.