Immutable Cloudwatch Rule Messages
I created a vibrant AWS feed that works well when I test it and when I manually create a cron job using a cloudwatch rule.
It displays metrics as calls (failed) as well as logs with execution details.
Then I decided to remove this manually created Cloudwatch rule to create one with the impossible.
- name: Create lambda service.
lambda:
name: "{{ item.name }}"
state: present
zip_file: "{{ item.zip_file }}"
runtime: 'python2.7'
role: 'arn:aws:iam::12345678901:role/lambda_ecr_delete'
handler: 'main.handler'
region: 'eu-west-2'
environment_variables: "{{ item.env_vars }}"
with_items:
- name: lamda_ecr_cleaner
zip_file: assets/scripts/ecr-cleaner.zip
env_vars:
'DRYRUN': '0'
'IMAGES_TO_KEEP': '20'
'REGION': 'eu-west-2'
register: new_lambda
- name: Schedule a cloudwatch event.
cloudwatchevent_rule:
name: ecr_delete
schedule_expression: "rate(1 day)"
description: Delete old images in ecr repo.
targets:
- id: ecr_delete
arn: "{{ item.configuration.function_arn }}"
with_items: "{{ new_lambda.results }}"
This creates pretty much the same cloudwatch rule. The only difference I can see with manually created is the targets, the lambda version / alias is set to "Default" when created manually, when set to version, with the corresponding version number when created with impossible.
The cloudwatch rule created with ansible only has failed calls.
Any idea why this is so? I don't see any logs. Is there a way I can set for the default version as well with the cloudwatchevent_rule module?
source to share
I too wasted time with the same error and the same confusion (why is there no log for failed calls?), I'm going to share my "solution", this will solve the problem for someone, and help others to debug and find the final solution.
Note. Be careful, this can allow any AWS account to run your lambda functions
Since you called the function by manually creating the target manually, I am assuming you have added permission to make lambda calls from CloudWatch, however, it looks like the original account id is different when the event is generated by the cli / api and when created by the AWS panel / console
If you add the "Source Account" clause in the lambda call resolution from the "events.amazonaws.com" principal to prevent any AWS account from running your lambdas, simply remove it (at your responsibility!).
So if your lambda policy looks like this:
{
"Sid": "<sid>",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "lambda:InvokeFunction",,
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "<account-id>"
}
},
"Resource": "arn:aws:lambda:<region>:<account-id>:function:<lambda-function>"
}
Remove the "Condition" field
{
"Sid": "sid",
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "lambda:InvokeFunction",,
"Resource": "arn:aws:lambda:<region>:<account-id>:function:<lambda-function>"
}
And "maybe" it will work for you.
I think something strange is happening with the data of the event owner / creator of the cloud event when the event is generated by the cli / api ... maybe a bug? Not sure. I will keep working on it
source to share
To expand on the answer here https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CWE_Troubleshooting.html#LAMfunctionNotInvoked . Since you are creating it via the API, you must add the Lambda permission as mentioned earlier. Without compromising security, you can do the following:
Add a rule with an PutRule
api call , it will return you
{
"RuleArn": "string"
}
Use RuleArn
in Lambda AddPermission call
aws lambda add-permission \
--function-name MyFunction \
--statement-id MyId \
--action 'lambda:InvokeFunction' \
--principal events.amazonaws.com \
--source-arn arn-from-PutRule-request
source to share