Python: linecache not working as expected?

Question

Python: linecache not working as expected?

Hello, I have this python script that I need to use to move some directories and fetch some information inside some files in those directories.

So I have a lot of directories. Within each of these directories, there are 5 more subdirectories. And there are 3 text files in each of these 5 subdirectories. One is .txt, which I ignore, the other is .out, which I need to read to see if it has one line with the word "Fin". If it has this line, then I should read the remaining .time file. This file has a Unix time command output that looks like this:

real    0m1.185s
user    0m0.027s
sys     0m0.026s

From this file I need to extract the realtime string which is the second line in this file (real 0m1.185s), the first line is "\ n".

So, I extract this line for each of the files in 5 subdirectories (which means 5 shared files) in the current directory and I have to sum the total number of seconds that each line of each file indicates and then divide it by 5 to get the average from 5 subdirectories.

Now for each of these averaged totals, I write an output file with the value. Therefore, if I have two directories

1/
2/

Each of these directories has 5 subdirectories

1/1 1/2 1/3 1/4 1/5
2/2 2/2 2/3 2/4 2/5

In these subdirectories I have text files, this means there is a file something.out at 1/1, hopefully with the word "Fin" inside. If it is that then there is a 1/1 something.time file where I extract the realtime string. Then I sum the values of the time files in 1/1 1/2 1/3 1/4 1/5 and dividing them by 5 to get the average. Then I write this average to the output file.

The problem I am facing is that I am using the linecache.getline command to fetch the second line from something.time file, but that does not work as expected as it oddly fetches the same line in every subdirectory. So, in the 1/1 subdirectory, the second line of the something.time file is "real 0m1.809s". My code does it well, but then it goes into the 1/2 subdirectory and fetches the second line of the something.time file and oddly enough it shows that it is the same "real line 0m1.809s" but if I am a cat in something. the time file at 1/2 shows it "real 0m1.009s".

Now the same thing happens in directory 2 /. It fetches the first line of the file over the first subdirectory it goes into, but then it just repeats that line 5 times.

Here is my code can someone point me where my error is?

def proArch(dirArch):
    dirList = os.listdir(dirArch)
    resultado = 0
    valores=[]
    for f in dirList:
       if("out" in f):
          for linea in open(f):
            if "Fin" in linea:
              for f_v in dirList:
                if("time" in f_v):
                  linea=linecache.getline(f_v, 2)
                  valores=re.split("['\tms']",linea)[1:3]
                  resultado=(float(valores[0])*60)+float(valores[1])
                else:
                  print("El archivo "+dirArch+" no se proceso bien.")

    return resultado


dirList_g = os.listdir(".")
dirOrig = os.getcwd()
res_tot=0.0
for d in dirList_g:
    if os.path.isdir(d) == True:
     os.chdir(dirOrig+"/"+d)
     dirAct = os.getcwd()
     dirList_w = os.listdir(".")
     for d_w in dirList_w:
       os.chdir(dirAct+"/"+d_w)
       dirArch = os.getcwd()
       res_tot=res_tot+proArch(dirArch)

     res_tot=res_tot/5
     os.chdir(dirOrig)
     with open("output.txt", "w") as text_file:
        text_file.write(dirAct+" "+str(res_tot)+"\n")
     res_tot=0.0

+3

python linecache

Atirag 04 Aug 14 at 20:57

source to share

1 answer

stderr · Accepted Answer · 2014-08-04T21:12:43+0000

It is possible linecache

that the caching

line from the file with the same name from the last time is also tinkering with you .

Also, it looks like you are not using the full file path to open a different file than you would expect.

For example, instead, f_v

you want to do something like:

filepath = os.path.join(<dirname>, <filename>)

Try replacing linecache.getline

with something like:

def get_line(filename, n):
    with open(filename, 'r') as f:
        for line_number, line in enumerate(f):
            if line_number == n:
                return line

Unlike linecache, this will actually open the file and read it every time.

Finally, this code will probably be much clearer and easier to deal with if you rewrote it with os.walk

:

https://docs.python.org/2/library/os.html

For example:

import os
for root, dirs, files in os.walk('someplace'):
    for dir in dirs:
        # do something with the dirs
    for file in files:
        # do whatever with the files

Python: linecache not working as expected?

More articles: