URL length limit when using download_zip_url

Snapt Soft

July 07, 2021 15:51

We have the ability for users to bulk download their media assets in our application. Everything works exactly as expected, until they attempt to download more than roughly 50 assets at a time. 

We are using the download_zip_url helper function from the cloudinary library on Node JS to return a URL to the user which is then automatically downloaded. When the user selects too many assets, the request fails with the following error: 

Cross-Origin Resource Sharing error: MissingAllowOriginHeader

When trying to manually fetch the URL, I get:

414 Request-URI Too Large

Our public IDs are quite length due to the hierarchy structure. An example public ID

52b7357c-0edd-4c95-85c3-dfdbcdfad9ac/bc23a478-e859-481d-a7a3-d1e6ebb73daf/b0ddfc0f-b199-4656-80d7-d850c06013a1-1625671643109.jpg

We are locked into the folder/key structure that has been created. Are there any other alternatives to download a large number of images that do not involve storing an additional zip file on cloudinary? I suspect that this may still have issues as the URL to create those would also be quite lengthy. 

0

• Official comment

  

  Akshay Joshi

  July 08, 2021 19:00

  Hi there,

  By default, our public_id limit is 255 characters.

  If you want to download many or all images from your Cloudinary account there are several ways to do this, depending on your requirements, the size of your account, and whether you are able to use our SDK or API to perform the operations:

  1. If you're already using our SDK or API or if the account is very large, you can use our Admin API or Search API to list all resources in your account. The list will include the URL for accessing or downloading the original assets, which you can do using any HTTP library or downloading tool, or can use to upload the file to another service or storage bucket.
  2. Our command line interface tool can be used to perform the same operation with the 'sync' command. The sync command can use the search API to list all files, then download them to your computer or server: https://cloudinary.com/documentation/cloudinary_cli#sync
  3. On any paid Cloudinary account you can choose to store backups of the files in your Cloudinary account in an Amazon S3 or Google Cloud Storage bucket - the naming structure of the backup is different from how you see the files in Media Library, but this is a fast way to get a copy of all files if you can't use one of the above methods: https://cloudinary.com/documentation/backups_and_version_management#using_your_own_backup_storage
  4. You can use our API to create a ZIP file containing up to 1000 assets, with a maximum size of 100 MB or your account's 'raw' file size limit (whichever is larger): https://cloudinary.com/documentation/image_upload_api_reference#generate_archive_method
  5. If none of those other options are suitable, you can select the files or folders that you want to download using our Media Library and click "Download" which will add them to a ZIP file and download it to your computer. This is subject to the same restrictions as above, a maximum of 1000 files in each ZIP and each ZIP must be below (the larger of) 100 MB or the 'raw' file size limit of your account: https://cloudinary.com/documentation/dam_folders_collections_sharing#download_folders

  Please let me know if you're able to list or download all of your files, and if there's anything else I can help with

  Regards,

  Akshay

Post is closed for comments.