How can I push AWS CodeCommit to S3 using Lambda?
Python is my preferred language, but whatever Lambda supports will do. - All AWS Architecture -
I have Prod, Beta and Gamma branches and corresponding folders in S3. I'm looking for a way to have Lambda respond to a CodeCommit trigger and based on the Branch that called it, clone the repo, and put the files in the appropriate S3 folder.
- S3: // Example-folder / Application / Prod
- S3: // Sample-folders / Application / Beta
- S3: // Example-folders / Application / Gamma
I tried to use GitPython but it doesn't work because Lambda doesn't have Git installed based on Lambda AMI and GitPython depends on it.
I also looked through the Boto3 docs and only custodian tasks are available; it cannot get the project files back.
Thanks for the help!
source to share
Unfortunately, CodeCommit does not currently have an API to load a repository into an S3 bucket. However, if you are open to testing CodePipeline, you can configure AWS CodePipeline to use a branch in the AWS CodeCommit repository as the source for your code. This way, when you make changes to the selected tracking branch in CodePipeline, the repository archive at the tip of that branch will be delivered to your CodePipelie bucket. For more information on CodePipeline see the following link: http://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html
source to share
The latest version boto3 codecommit
includes methods get_differences
and get_blob
. You can get the entire contents of the codecommit repository using these two methods (at least if you are not interested in preserving the .git history).
Below is the script all the contents of the master branch and added to the tar file. After that, you can upload it to s3 as you like. You can run this as a lambda function that can be called when you click on codecommit.
This works with the current python 3.6 lambda environment .
botocore==1.5.89
boto3==1.4.4
import boto3
import pathlib
import tarfile
import io
import sys
def get_differences(repository_name, branch="master"):
response = codecommit.get_differences(
repositoryName=repository_name,
afterCommitSpecifier=branch,
)
differences = []
while "nextToken" in response:
response = codecommit.get_differences(
repositoryName=repository_name,
afterCommitSpecifier=branch,
nextToken=response["nextToken"]
)
differences += response.get("differences", [])
else:
differences += response["differences"]
return differences
if __name__ == "__main__":
repository_name = sys.argv[1]
codecommit = boto3.client("codecommit")
repository_path = pathlib.Path(repository_name)
buf = io.BytesIO()
with tarfile.open(None, mode="w:gz", fileobj=buf) as tar:
for difference in get_differences(repository_name):
blobid = difference["afterBlob"]["blobId"]
path = difference["afterBlob"]["path"]
mode = difference["afterBlob"]["mode"] # noqa
blob = codecommit.get_blob(
repositoryName=repository_name, blobId=blobid)
tarinfo = tarfile.TarInfo(str(repository_path / path))
tarinfo.size = len(blob["content"])
tar.addfile(tarinfo, io.BytesIO(blob["content"]))
tarobject = buf.getvalue()
# save to s3
source to share