Want to process a large application log file

I am running my application on Google App Engine ..... My log files are huge due to high traffic The log file looks like this:

122.167.221.243 - - [06/Mar/2012:01:11:29 -0800] "GET /Bengali/enable HTTP/1.1" 404 0 "http://www.epicsearch.in/" -
---Similar rows ---- below

      

I want to process them to get meaningful statistics like no. unique IP addresses or not. hits on a given page .... I'm a newbie so I don't know how to do it !!!!!

+3


source to share


1 answer


Assuming you haven't already done so, the first thing you need to do is upload your logs to App Engine using appcfg.py

, for example:

appcfg.py --num_days=5 request_logs appname/ logs.txt

      



After downloading it, you will need to collapse the script or program to unravel your logs for any information...

This Stack Overflow question might be a good place to start - indeed, any decent scripting language should be able to adequately parse these logs for you.

+1


source







All Articles