Access denied to S3 bucket from Athena although permissions = allow all
I followed the tutorial http://docs.aws.amazon.com/athena/latest/ug/connect-with-jdbc.html . I configured an IAM user (with weird permissions) and an S3 bucket and was able to query for sample Athena tables and the outputs were written to my S3 bucket. I now have credentials from a client to access his Athena table. It doesn't work and I get the following error:
Access denied while writing output to URL: s3: //my-test-bucket/b36-f3c0-482-a225-34d63d355.txt. Please make sure you are allowed access to the S3 bucket. If you encrypt the request results with a KMS key, make sure you have access to your KMS key
My S3-bucket is as popular as it gets. Permissions for "Any Authenticated AWS User": "Read, Write". Permissions for "everyone": read, write. Permissions for "Log Delivery": Read, Write. "Access permissions" for everything: Read, Write
Bucket policy allows everyone to do everything.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::my-test-bucket"
},
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::my-test-bucket/*"
},
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::my-test-bucket/*"
}
] }
CORS configuration:
<?xml version="1.0" encoding="UTF-8"?> <CORSConfiguration
xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>DELETE</AllowedMethod>
<AllowedHeader>*</AllowedHeader> </CORSRule> </CORSConfiguration>
However, I get the error: Access denied while writing output to url ...
Edit: From time to time I get another error: "Failed to test / create output bucket my-test-bucket". Not sure why I am getting different errors.
What can I do?
source to share
Has almost the same problem with a specific file on s3. I was unable to read the file. Solving the problem by changing the file permissions using the mv command and argument --acl
. An attempt to access a file named data.jsonlines
gave an error ACCESS DENIED
. Solved it by running the following commands:
NOTE. You need to install AWS CLI :$ pip install --upgrade --user awscli
aws s3 cp s3://<s3 bucket name>/path/to/file/data.jsonlines s3://cfa-opengazettes-ke/gazettes/data_copy.jsonlines
aws s3 mv --acl public-read s3://<s3 bucket name>/path/to/file/data_copy.jsonlines s3://cfa-opengazettes-ke/gazettes/data.jsonlines
Or you can combine them by doing:
aws s3 cp s3://<s3 bucket name>/path/to/file/data_out.jsonlines s3://cfa-opengazettes-ke/gazettes/data_out2.jsonlines && aws s3 mv --acl public-read s3://cfa-opengazettes-ke/gazettes/data_out2.jsonlines s3://<s3 bucket name>/path/to/file/data_out.jsonlines
These commands perform the following steps:
-
Copy:
s3://<s3 bucket name>/path/to/file/data.jsonlines
tos3://c<s3 bucket name>/path/to/file/data_copy.jsonlines
-
move:
s3://<s3 bucket name>/path/to/file/data_copy.jsonlines
tos3://cfa-opengazettes-ke/path/to/file/data.jsonlines
Basically, it makes a copy of the file and then deletes it during the move when the file's permissions change.
Pay attention to the parameter --acl
and argument public-read
. Perhaps one of the following resolutions might work for you. You can replace it public-read
with a different resolution. From the doc:
- acl (string) Sets the ACL for the object when the command is executed. If you use this parameter, you must have "s3: PutObjectAcl" permission included in the list of actions for your IAM. Only accepts the values private , public read , public-read-write , authenticated-read , aws-exec-read , slave owner-owner , bucket-owner-full-control, and write-write-write .
More useful information on this AWS page
source to share