You can delete an empty folder in the Media library.
This can be done by choosing the empty folder, and clicking on the garbage can next to it
To make sure the operation runs successfully, you must make sure the folder is completely empty. Navigate through the different sections (Images, Videos & Audio, Files) and filter by "All Images/Videos/Files" on the top-right pulldown menu.
Its now possible to delete folders with resources within them. deleting the folder would delete all resources associated with that folder.