Clever Geek Handbook

Where can I export AWS Cloudwatch logs (for Loggly)?

I know they store it somewhere. Maybe a DynamoDB bucket or S3?

I would like to use Cloudwatch as a log archive for my EC2 instances and set it up so that they are piped to Loggly to view / sort / etc.

I know that Loggly recently released a way to capture logs from an S3 bucket ( https://www.loggly.com/docs/s3-logs/ ), but I cannot find the source of these logs on the AWS side.

My thoughts so far:

  • Perhaps I can configure Metric / Alarm to pass all incoming logs to Loggly?
  • Could I be able to access where AWS stores these Cloudwatch logs?
  • Set up a cron somewhere that bounces the Cloudwatch logs to the S3 bucket and exits from there?
  • Use Cloudwatch API calls in cron to grab logs directly and send them to Loggly via HTTP message, etc.
+3


source to share


3 answers


This API will allow you to export data to S3, however please note that for non-real-time historical data ... the data will not be available for export until several hours, so most useful for daily statements: http://docs.aws.amazon .com / AmazonCloudWatch / latest / DeveloperGuide / S3Export.html



For live delivery, you need to use the Subscription API: http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/Subscriptions.html

0


source


I did this by setting up a log group subscription that runs the Lambda function. The function converts the log data and writes it as a .log file to S3, or sends the records to Kinesis Firehose for a collection and then to S3 as an archive.



You can check out the complete entry or just a Lambda function if you are trying something like this.

+1


source


Loggly doesn't seem to support a similar service.

According to Loggly Log File Monitoring Options , you have the following options:

Sincerely.

0


source






More articles:


All Articles