Articles in this section

See more

Guidelines for self implementing chunked upload to Cloudinary

Avatar
Nadav Ofir
  • Updated
Follow

When implementing chunked upload you must comply with the following restrictions:

  1. You must include the X-Unique-Upload-Id header and make sure it's identical (and unique) for all parts.
  2. Every part must contain a Content-Range header which is applicable to the location of the chunk in the overall file (Format: 'bytes #start-#end/#total', e.g. 'bytes 0-5999999/22744222').
    • If the total size is unknown, set it to -1, except for the last one.
  3. Each chunk must be larger than 5MB except for the last one.

Here's how it's implemented within our Ruby libraries: 
https://github.com/cloudinary/cloudinary_gem/blob/a738f8c57ef452ea75c924f806fb05e9682749c5/lib/cloudinary/uploader.rb#L96

 

Attached is a code example uploading a file using chunked upload. Before running the code, please enter your cloud_name, upload_preset and the public_id in the index.js file. 

 

 

 

 

Related articles

Comments

10 comments

Sort by Date Votes
  • Avatar
    Han BaHwan

    How can I issue `X-Unique-Upload-Id`?
    What's Content-Range format?
    If I understand correctly, this is below:
    `Content-Range: 0-99999/2000000`
    `Content-Range: 100000-199999/2000000`
    `Content-Range: 200000-299999/2000000`
    Right?

    And how can I handle errors? It looks exception can happen.

    0
    Comment actions Permalink
  • Avatar
    Nadav Ofir
    • Edited

    For `X-Unique-Upload-Id` you can use any random/unique string generator, for example Unix timestamp.
    `Content-Range` - must contain a preceding "bytes".
    Error messages (bad responses) may arise in case of upload failure. Handling it depends on your use case and the HTTP client you're using.

    0
    Comment actions Permalink
  • Avatar
    Nadav Ofir

    Han, I've just noticed that the content-range header directions were missing the necessary "bytes" prefix. This is now fixed in the above explanation.

    0
    Comment actions Permalink
  • Avatar
    Mohamed Habashy
    • Edited

    Hi Everyone,

    I think there is a additional restriction to the mentioned above by Nadav.

    "First chunk must be first. meaning it must arrive the server before any other chunks."

    Then you can do bulk upload the remaining chunk except the last as mentioned above.

    Kind regards,
    Mohamed

    0
    Comment actions Permalink
  • Avatar
    Geoff Plitt

    We are using react-native-fetch-blob:

    RNFetchBlob.fetch(

    'POST',
    upload_url,
    {
    'Content-Type':'multipart/form-data',
    'Transfer-Encoding':'Chunked' // <-- CHUNKED header
    },
    [{
    name:'file',
    filename:'upload.mp4',
    type:'video/mp4',
    data:RNFetchBlob.wrap(uri),
    },
    {name: 'timestamp', data: timestamp },
    {name: 'signature', data: signature},
    {name: 'eager', data: 'sp_full_hd_wifi/m3u8'},
    {name: 'eager_async', data: 'true'},
    {name: 'api_key', data: cloudinary_config.api_key}
    ]
    )
     
     
    We got a successful response from Cloudinary.  How do we know whether the upload was actually chunked or just a normal upload?  (I'm using the header marked above as per the documentation but I'm don't know whether it is actually chunking correctly for Cloudinary)
     
    0
    Comment actions Permalink
  • Avatar
    Raya Straus

    Hi Geoff,

    If you open a ticket at support@cloudinary.com with more information on your request, we can take a look at our logs. Another option would be to examine the network tab while the upload is being performed. If the upload is chunked you'll be able to see multiple requests going through. 

    0
    Comment actions Permalink
  • Avatar
    leon

    I have trouble implementing this from the browser in javascript. I got the error : 

    { message: "Chunk size doesn't match upload size: 6000000 - 8000037" }

    I create my chunks using

    initialFileAsBlob.slice(0, 6000000) 

    to create chunk of 6MB, then I read the chunk with

    chunkToSend = readAsDataUrl(chunk) 

    and send it (with signature) and the following headers : 


    'X-Unique-Upload-Id': 'uniqueid.mp4',
    'Content-Range': 'bytes 0-6000000/17500142

    0
    Comment actions Permalink
  • Avatar
    Shirly Manor

    It might be because the chunk that you specify is off by 1. when specifying 0-6000000 its mean 6000001.

    If that doesn't help, can you please open a support ticket at support@cloudinary.com for further investigations.

     

    0
    Comment actions Permalink
  • Avatar
    Roni Yosofov

    Thank you for the guide!
    does Cloudinary also save them in chunks? e.g. do they get served in chunks from the CDN?

    0
    Comment actions Permalink
  • Avatar
    Eric Pasos

    Hi Roni,

    The file will be spliced into chunks while uploading to your Media Library account and once the last chunk is uploaded, all the chunked files will be put together and will result back into a single file. Hence, you will only see one file in your account and a single file will also be delivered to CDN when accessed.

    Hope this helps.

    0
    Comment actions Permalink

Please sign in to leave a comment.