AWS CLI S3: Copying File Locally Using Terminal: Fatal Error: An error occurred (404) while calling the HeadObject operation
I am trying to copy files locally from s3 bucket. I can get a list of files in my bucket:
aws s3 ls s3://myBucket/myDirectory/todaysFiles/
But when I try to copy files locally:
aws s3 cp s3://myBucket/myDirectory/todaysFiles/ .
I am getting this error:
fatal error: An error occurred (404) when calling the HeadObject operation: Key "myDirectory/todaysFiles/" does not exist
But I am trying to copy only one file locally:
aws s3 cp s3://myBucket/myDirectory/todaysFiles/somefile .
I am getting this error:
warning: Skipping file s3://myBucket/myDirectory/todaysFiles/somefile. Object is of storage class GLACIER. Unable to perform download operations on GLACIER objects. You must restore the object to be able to the perform operation. See aws s3 download help for additional parameter options to ignore or force these transfers.
Does any of you know why I am getting this error or a workaround for these errors?
I really appreciate your help
source to share
For the first error, add a recursive flag:
aws s3 cp s3://myBucket/myDirectory/todaysFiles/ . --recursive
This will copy all files in the "todaysFiles" directory to the current directory.
However, the second error indicates that your files are in Glacier. This complicates things a little, as the glacier is not real-time - depending on what you are willing to pay, it could be hours before the data is restored. For more information, see the docs Restoring Objects . You cannot copy from S3 until the object is restored from the glacier to S3.
Note that if you do this, you will have costs from both the glacier and S3.
As an aside, if these files are indeed files from today, there should be a lot more time between storing on S3 and clicking on Glacier. But I am assuming that there is a date related component in the parent directories as well.
source to share