Run jobs using EMR cluster and s3 files from another account

I would like to know if there is a way to access the jar files and I / O location from s3 of another account. I have an EMR cluster running on account 1. I want to access files and jars from s3 account 2. I am using AWS SDK to run AWS Simple workflow. Thank.


source to share

1 answer

You need to create a cross account access role:

You can set up cross-account access using IAM roles. You define a role in account 2 that the user can use (IAM user or federated user) in account 1. Using roles for cross-account access allows you to grant access to any resource on account 2 (in your case it is S3)


You need to first create a role with readwrite access to S3 in account 2 (let's call it "S3-ReadWrite-role") and give permission for users from account 1 to use the role "S3-ReadWrite-role"

Check this link, it will explain you how to do it: matches-Roles-span

After you finish the first step, you can use this code (not tested): Using your credentials, you will get temporary security credentials to use "S3-ReadWrite-role", then you will use temporary security credentials to access to S3;)

import java.util.HashMap;

import com.amazonaws.auth.*;

public class AssumeRoleDemo {
    private static final String ROLE_ARN =

    private static AWSCredentials longTermCredentials_;

    private static void init() throws Exception {
    // acquire long term credentials from the properties file ( you should use this method)
    //longTermCredentials_ = new PropertiesCredentials(AssumeRoleDemo.class.getResourceAsStream(""));

    // or you can use this one
    longTermCredentials = new BasicAWSCredentials(access_key_id, secret_access_key);

    public static void main(String[] args) throws Exception {

       // Step 1. Use Joe.s long-term credentials to call the
       // AWS Security Token Service (STS) AssumeRole API, specifying 
       // the ARN for the role S3-RW-role in account2.

        AWSSecurityTokenServiceClient stsClient = new

        AssumeRoleRequest assumeRequest = new AssumeRoleRequest()

       AssumeRoleResult assumeResult =

      // Step 2. AssumeRole returns temporary security credentials for 
      // the IAM role.

      BasicSessionCredentials temporaryCredentials =
         new BasicSessionCredentials(

     // Step 3. Make S3 service calls to read data from a 
     // S3, stored in account2, using the 
     // temporary security credentials from the S3-ReadWrite-role 
     // that were returned in the previous step.

     AmazonS3 s3Client = new AmazonS3Client(temporaryCredentials);
     S3Object object = s3Client.getObject(
              new GetObjectRequest(bucketName, key));
     InputStream objectData = object.getObjectContent();
     // Process the objectData stream.





All Articles