How to sample representative frames from a video?

Ryan Berger

August 04, 2021 22:44

Hi Support Forum, 

I'm interested in using computer vision services for my videos (for classification, object detection, etc). One way to start is by pulling out frames, and sending the images to a service. I saw that I can transform a video into a gif, effectively sampling video frames every x seconds here: https://cloudinary.com/documentation/videos_to_animated_images. But I'm cheap! is there a smarter way to sample frames that represent different shots in the video? 

0

• 

  Eric Cauan Pasos

  August 05, 2021 06:38

  Hi Ryan,

  In order to be able to identify which frame is to be sampled for your gif (i.e. instead of using video_sampling), you could splice or concatenate the target frames of your video before converting it to gif. This can be achieved in a single delivery URL. For example, the following transformed delivery URL is the spliced/concatenated portions for the dog.mp4 from the demo account: 

  https://res.cloudinary.com/demo/video/upload/eo_3,so_2/c_fill,h_400,q_auto:best,w_600/eo_6,fl_splice,l_video:dog,so_5/c_fill,h_400,w_600/fl_layer_apply/eo_9,fl_splice,l_video:dog,so_8/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_12,fl_splice,l_video:dog,so_11/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_14,fl_splice,l_video:dog,so_13/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/dog.mp4

  As for the URL details, there are 5 splices of 1s duration being concatenated together:

  1st frame (2s - 3s): https://res.cloudinary.com/demo/video/upload/eo_3,so_2/c_fill,h_400,q_auto:best,w_600/<other_transfomrations_here>/dog.mp4

  2nd frame (5s - 6s): ../eo_6,fl_splice,l_video:dog,so_5/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/..

  3rd frame (8s - 9s): ../eo_9,fl_splice,l_video:dog,so_8/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/..

  4th frame (11s - 12s): ../eo_12,fl_splice,l_video:dog,so_11/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/..

  5th frame (13s - 14s): ../eo_14,fl_splice,l_video:dog,so_13/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/..

   

  The URL above can be converted to gif by changing the filename extension (i.e. dog.mp4 to dog.gif):

  https://res.cloudinary.com/demo/video/upload/eo_3,so_2/c_fill,h_400,q_auto:best,w_600/eo_6,fl_splice,l_video:dog,so_5/c_fill,h_400,w_600/fl_layer_apply/eo_9,fl_splice,l_video:dog,so_8/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_12,fl_splice,l_video:dog,so_11/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_14,fl_splice,l_video:dog,so_13/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/dog.gif

  Finally, you could add the transformation parameter e_loop:

  https://res.cloudinary.com/demo/video/upload/eo_3,so_2/c_fill,h_400,q_auto:best,w_600/e_loop/eo_6,fl_splice,l_video:dog,so_5/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_9,fl_splice,l_video:dog,so_8/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_12,fl_splice,l_video:dog,so_11/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/eo_14,fl_splice,l_video:dog,so_13/c_fill,h_400,q_auto:best,w_600/fl_layer_apply/dog.gif

  For more information, kindly visit the following links:
  https://cloudinary.com/documentation/video_manipulation_and_delivery#concatenating_videos
  https://cloudinary.com/documentation/transformation_reference#fl_splice
  https://cloudinary.com/documentation/transformation_reference#so_start_offset
  https://cloudinary.com/documentation/transformation_reference#eo_end_offset
  https://cloudinary.com/documentation/transformation_reference#e_loop

  Hope this helps, please let me know if you have any further questions.

  Best regards,
  Eric

  0

Post is closed for comments.