Python skips for loop
I am writing code to find latitudes and longitudes and calculate the distance within a specific radius of a point and split the two files.
For the first 5 iterations, the program runs fine, but after that the program doesn't go through the inner loop for
. I went through the code, it just steps over the loop for
. It seems to depend on what I set the to variable radius
. If radius
less, it will allow fewer inner loop iterations for
.
I'm afraid it might be a problem with the way I read in the file. I believe it is infile_2
empty after the 5th iteration , but I cannot figure out how to fix it.
def main():
global infile_1, infile_2
## import geocoded college dataset
infile_2 = open("university_FIPS.csv", "r")
## import great_lakes_sample
infile_1 = open("great_lakes_sample.csv", "r")
outfile_within_bound = open("great_lakes_blood_college", "w")
outfile_outside_bound = open("great_lakes_blood_NOcollege", "w")
inside_buffer_count = 0
outside_buffer_count = 0
global lat_1_index, long_1_index, lat_2_index, long_2_index
## set radius to desired length (in miles)
radius = 100
## when it is generalized, use this:
## radius = input_buffer_radius()
# create two subsets of blood drive data, one within
# radius of college, one outside
# skip header
n_1 = 0
for infile_1_line in infile_1:
infile_1_line = infile_1_line.strip().replace("\"", "").split(","),
infile_1_line = infile_1_line[0]
record_stored = False
# find index of lat_2, long_2
if( n_1 == 0 ):
lat_2_index = infile_1_line.index( "lat" )
long_2_index = infile_1_line.index( "long" )
infile_1_header_list = infile_1_line
# assign lat_2, long_2 latitude and longitude values
lat_2 = infile_1_line[ lat_2_index ]
long_2 = infile_1_line[ long_2_index ]
# skip header
if n_1 > 0:
print( "\n\nExamining Record:", n_1 )
try:
lat_2 = float( lat_2 )
long_2 = float( long_2 )
except ValueError:
print( "Value error, skipping record" )
continue
except TypeError:
print("Type error, skipping record" )
continue
print( "Coordinates for record:", lat_2, long_2)
# skip header
n_2 = 0
# WILL NOT ENTER LOOP ON THIRD ITERATION, it dependent on radius, when radius is 100, it stops at n_1 = 6
if ( n_1 > 0):
print("\n\n\nbefore loop")
for infile_2_line in infile_2:
infile_2_line = infile_2_line.strip().split(",")
if ( n_2 == 0):
print( "in" )
# find index of lat_1, long_1, create header list
if( n_2 == 0 and n_1 == 1):
lat_1_index = infile_2_line.index("lat")
long_1_index = infile_2_line.index("long")
infile_2_header_list = infile_2_line
# creat headers for each outfile
write_to_csv( infile_1_header_list, outfile_within_bound )
write_to_csv( infile_1_header_list, outfile_outside_bound )
# assign values for lat_1, long_1, after header
if( n_2 > 0 ):
lat_1 = infile_2_line[ lat_1_index ]
long_1 = infile_2_line[ long_1_index ]
try:
lat_1 = float( lat_1 )
long_1 = float( long_1 )
value_error = False
except ValueError:
continue
except TypeError:
continue
dist = haversine_distance(lat_1, long_1, lat_2, long_2)
if( dist <= radius ):
print( "\nRecord", n_1, "is",
dist, "miles from", lat_1, long_1)
write_to_csv( infile_1_line, outfile_within_bound )
record_stored = True
print( "Record stored in outfile_inside_bound." )
print( "Moving to next record." )
inside_buffer_count += 1
break
n_2 += 1
if( record_stored == False):
print( "\nOutside buffer." )
write_to_csv( infile_1_line, outfile_outside_bound )
outside_buffer_count += 1
print( "Record stored in outfile_outside_bound." )
n_1 += 1
print("\nRecords within buffer:", inside_buffer_count, "\nRecords outside buffer:", outside_buffer_count)
infile_1.close()
infile_1.close()
outfile_within_bound.close()
outfile_outside_bound.close()
source to share
The direct answer is that you iterate over the file in a style loop for x in f
, python actually keeps track of how far you go into the file. So if you iterate 10 times in the inner loop until you reach the breakpoint, the next time you try to iterate through the file with, infile_2
you run 10 lines into the file!
It looks like in your case, on the third iteration, you read the entire file, so the iterator infile_2
will only sit at the end of the file on all subsequent iterations of the outer loop. An easy fix is to do it infile_2.seek(0)
before the inner loop is started. This will change position infile_2
to look at the beginning of the file again. Uff ...
This is all fine and dandy, but I would like to offer you a couple of things:
-
When you open files use
with open("test.txt","r") as f
as shown in this SO post . This allows you to remember to close the file explicitly, since it is closed implicitly at the end of the block. -
It is often best to read a file in a list, perform calculations, and then write the results in just one snapshot. This makes your code more organized (also easier to read) and also avoids bugs like the one you are working with.
To illustrate these strategies, here's how I'll read the files in your example code:
def main():
global infile_1, infile_2
with open("great_lakes_sample.csv", "r") as infile_1:
#List comprehension to format all of the lines correctly
infile1_lines = [line.strip().replace("\"", "").split(",") for line in infile_1]
with open("university_FIPS.csv", "r") as infile_2:
#List comprehension to format all of the lines correctly
infile2_lines = [line.strip().split(",") for line in infile_2]
#Both files are automatically closed when their respected with blocks end.
source to share