Delete old files on full disk
An application that writes data to a disk in 100MB chunks and increments the filename by adding +1, therefore n1, n2 ... n1000
. This will end up using all the free space on the (Linux host) partition. I'm looking for a way to delete files that were first serialized until disk space is used for a specific use.
If the latter, this stackoverflow question / 579742 / ... would be a good solution?
The following solution has been suggested and looks like a viable solution for this StackOverflow question . How can this be changed to handle the extension of an incremental file as it is currently running the script, it does not remove the filenames filename *, the asterisk being an incremental number starts with the oldest one?
import os
def free_space_up_to(free_bytes_required="161061273600", rootfolder="/data/", ex
tension="filename-*"):
file_list= files_to_delete(rootfolder, extension)
while file_list:
statv= os.statvfs(rootfolder)
if statv.f_bfree*statv.f_bsize >= free_bytes_required:
break
os.remove(file_list.pop())
source to share
Well, if you know that all files are 100MB in size (at least one size) and assume that there is nothing drastic about the disk usage on the machine, you don't need to check the free space on every iteration.
Also, if all files have the same name, besides the counter at the end, you can skip the call to os.stat (which can also be useless for files created in quick succession) and sort by filenames based on the counter
import os
def free_space_up_to(free_bytes_required=161061273600, rootfolder="/data/", filesize=104857600, basename="filename-"):
'''Deletes rootfolder/basename*, oldest first, until there are free_bytes_required available on the partition.
Assumes that all files have file_size, and are all named basename{0,1,2,3,...}
Returns number of deleted files.
'''
statv = os.statvfs(rootfolder)
required_space = free_bytes_required - statv.f_bfree*statv.f_bsize
basepath = os.path.join(rootfolder, basename)
baselen = len(basepath)
if required_space <= 0:
return 0
# "1 +" here for quickly rounding
files_to_delete = 1 + required_space/filesize
# List all matching files. If needed, replace with os.walk for recursively
# searching into subdirectories of rootfolder
file_list = [os.path.join(rootfolder, f) for f in os.listdir(rootfolder)
if f.startswith(basename)]
file_list.sort(key=lambda i: int(i[baselen:]), reverse=True)
# Alternatively, if the filenames can't be trusted, sort based on modification time
#file_list.sort(key=lambda i: os.stat(i).st_mtime)
for f in file_list[:files_to_delete]:
os.remove(f)
return files_to_delete
(Not fully tested, I recommend a test run replacing "os.remove" with "print";))
source to share