There are a variety of ways to migrate your existing images to Cloudinary, both using the Media Library UI and the API, as well as using Cloudinary's SDKs and other features.
Using Cloudinary's REST API
Cloudinary's upload API supports a single file upload per API call, but can be called multiple times in parallel. As such, your application can use multiple threads to upload many files in a short term.
We recommend that such multi-threaded operations start by using approximately 10 simultaneous requests, and that you slowly increase the rate if necessary. If you reach account concurrency limits and receive an HTTP 420 error, you should reduce the number of simultaneous requests, and retry the failed request(s).
To speed up the upload process, you can provide a file's existing HTTP/HTTPS, S3, or Google Cloud Storage URL as the "file" parameter to the upload API method instead of sending the actual data. This allows for a much faster migration process because we can retrieve the images from the specified location instead of your code needing to download the file from the existing storage and then upload it to our API: https://cloudinary.com/documentation/upload_images#file_source_options
To specify an S3 or Google Cloud Storage URL, please ensure that the bucket can be accessed by Cloudinary and that you've specified which Cloudinary accounts should be allowed to copy images from the bucket. More info can be found in the Upload from a private storage URL (Amazon S3 or Google Cloud) section of the upload API documentation.
SDK Methods (Bulk Upload with Contextual Metadata)
Cloudinary offers a variety of SDKs that can be used to bulk upload assets.
In the example below the Ruby SDK is used to bulk upload and add contextual metadata to the corresponding asset. See the process below:
- Ruby Environment setup
- Fork from the Cloudinary Ruby-SDK-Quickstart documentation
- Set up Cloudinary configuration in the config.rb file. Credentials can be found on the Cloudinary account dashboard.
- Use the listfilenames.rb file to list the file names and their given descriptions in the test.csv file. A hash was used as an example to map the descriptions.
- Finally, use the bulkupload.rb file to implement multiple threads to bulk upload the assets with their respective contextual metadata.
Command line interface
The Cloudinary CLI provides an interface to call the API for common use-cases, and for bulk uploads there are several File Management commands, such as 'migrate' to upload a list of external media URLs, 'upload_dir' to upload a local folder to Cloudinary, or 'sync' to synchronize a local folder with your account
Using the Media Library UI to upload multiple files
The Media Library upload widget allows selecting multiple files at once, including dragging and dropping folders onto the Media Library window:
The Media Library uses the same upload API methods mentioned above so it is not rate limited as well.
Automatic migration of files from existing storage
Auto-upload remote resources - This feature allows you to link a folder in your Cloudinary account with a corresponding folder on your existing server or in an existing Amazon S3 bucket or Google Cloud Storage bucket. This would upload the images lazily (i.e., every time they are specifically called by the end-user).