Why can't I pass the csv handler to the python stream function?
import threading
import csv
csv_content = []
def load_csv(f_csv):
global csv_content
heading = next(f_csv)
print("heading: {0}".format(heading))
for lin in f_csv:
print(lin)
csv_content.append(lin)
def main():
with open('test.csv', 'rt') as f:
f_csv = csv.reader(f)
threadA = threading.Thread(target=load_csv, args=(f_csv,))
threadA.start()
global csv_content
curr_idx = 0
while curr_idx < len(csv_content):
print("[{0}]: {1}".format(curr_idx, csv_content[curr_idx]))
curr_idx += 1
if __name__ == "__main__":
main()
File "thread_csv.py", line 8, in load_csv
heading = next(f_csv)
StopIteration
I just discovered that the code will fail if I pass the csv handler to the python threading function as above. Instead, if I pass in the filename it works.
Questions> Can someone please share with me a light why can't we pass the csv file handler to the streaming function?
thank
==== Update Working Code ===
1 import threading
2 import csv
3
4 csv_content = []
5
6 def load_csv(file_name):
7 with open(file_name, 'rt') as f:
8 f_csv = csv.reader(f)
9 global csv_content
10 heading = next(f_csv)
11 print("heading: {0}".format(heading))
12 for lin in f_csv:
13 print(lin)
14 csv_content.append(lin)
15
16 def main():
17 threadA = threading.Thread(target=load_csv, args=('test.csv',))
18 threadA.start()
19
20 global csv_content
21 curr_idx = 0
22 while curr_idx < len(csv_content):
23 print("[{0}]: {1}".format(curr_idx, csv_content[curr_idx]))
24 curr_idx += 1
25
26 if __name__ == "__main__":
27 main()
source to share
Found it out. This code is the culprit:
with open('test.csv', 'rt') as f:
f_csv = csv.reader(f)
threadA = threading.Thread(target=load_csv, args=(f_csv,))
threadA.start()
What happens after the call .start()
? You exit the dispatcher. The manager closes the file and the handler now points to the closed file. Consequently, StopIteration
. This is a somewhat race condition.
To fix this, you need to wait for the thread to complete in the context manager, so call threadA.join()
:
with open('test.csv', 'rt') as f:
...
threadA.start()
threadA.join()
Alternatively, don't use a context manager - open the file with f = open(...)
and then start the stream as usual.
source to share