Stop Python Script from writing to file after reaching a certain size on linux
A bit new to Python and new to Linux. I created a script that runs the Twitter Streaming API. The script writes to a CSV file when things in the stream match my parameters.
I would like to know if there is a way to stop my script as soon as the file reaches 1 gigabyte. I know cron can be used during a script and everything, but I'm more worried about the file size than the time it takes.
Thanks for your input and attention.
source to share
In your case, you probably don't need to os.stat
, but os.stat
may give you an erroneous size in some cases (namely, the buffers are not flushed). Why not just use f.tell()
to read the size with something like this
with open('out.txt', 'w', encoding='utf-8') as f:
csvfile = csv.writer(f)
maxsize = 1024 # max file size in bytes
for row in data():
csvfile.writerow(row)
if f.tell() > maxsize: # f.tell() gives byte offset, no need to worry about multiwide chars
break
source to share
Use python os.stat()
to get information about a file, then check the total bytes of the existing file ( fileInfo.st_size
) and the size of the data you are about to write.
import os
fileInfo = os.stat('twitter_stream.csv')
fileSize = fileInfo.st_size
print fileSize
# Now get data from twitter
# determine number of bytes in data
# write data if file size + data bytes < 1GB
source to share