Want to process a large application log file
I am running my application on Google App Engine ..... My log files are huge due to high traffic The log file looks like this:
122.167.221.243 - - [06/Mar/2012:01:11:29 -0800] "GET /Bengali/enable HTTP/1.1" 404 0 "http://www.epicsearch.in/" -
---Similar rows ---- below
I want to process them to get meaningful statistics like no. unique IP addresses or not. hits on a given page .... I'm a newbie so I don't know how to do it !!!!!
+3
source to share
1 answer
Assuming you haven't already done so, the first thing you need to do is upload your logs to App Engine using appcfg.py
, for example:
appcfg.py --num_days=5 request_logs appname/ logs.txt
After downloading it, you will need to collapse the script or program to unravel your logs for any information...
This Stack Overflow question might be a good place to start - indeed, any decent scripting language should be able to adequately parse these logs for you.
+1
source to share