Pause a loop for a specific time at a specific time in R
I need to run a long loop that updates some data and stores it on my server. The problem is that the company runs the backup procedure at midnight, and for that they shutdown the server in about 15 minutes.
So, given that I have to write a file for each iteration, when the server goes down it breaks the loop.
I managed to get around the problem by writing a loop like this
for(i in bills.list){
url = paste0("ulalah",i,"/")
# Download the data
bill.result <- try(getURL(url)) # if there is an error try again
while(class(bill.result)=="try-error"){
Sys.sleep(1)
bill.result <- try(getURL(url))
}
# if iteration is between 23:59:00 and 23:59:40 wait 17 min to restart the loop
if(as.numeric(format(Sys.time(), "%H%M%S")) > 235900 &
as.numeric(format(Sys.time(), "%H%M%S")) < 235940){
Sys.sleep(1020)
}
# Write the page to local hard drive
write(bill.result, paste0("bill", i, ".txt"))
# Print progress of download
cat(i, "\n")
}
The problem is that by evaluating the time in all iterations, I am wasting valuable time. Any more effective thoughts?
+3
source to share
1 answer
I think you could just try to keep the date. If that fails, you may be in the backup window
store <- function(data, retry = 10) {
while(retry > 0) {
result <- try(write(data, "/some_broken_place"))
if(class(result) == "try-error") {
# it might be we are in the backup window
cat("I will sleep if it turns out that it backup time")
if(as.numeric(format(Sys.time(), "%H%M%S")) > 235900 &
as.numeric(format(Sys.time(), "%H%M%S")) < 235940){
Sys.sleep(1020)
}
retry <- retry - 1
}
}
if(retry == 0) {
cat("Very, very bad situation - no chance to store data")
}
}
+1
source to share