slow File Processing in python

I am trying a file operation using python.Aim is to continuously read a file of size(100bytes),pack and send them through socket. These files are read from a directory.

Problem: when i run the program continuously, execution time is increasing. Initially, execution time is less than a second; later it reaches till 8~10seconds. I am not able get exact reason for the delay.If anyone can throw some light on the issue, it will be more helpful.

Here i have attached my code...

def handlefile(filename):
        for sat in range(len(Numfiles)):
                filename = 
                fsize = os.path.getsize(filename)
                if fsize != 100:
                        continue
                rfile = open(filename,'rb')
                text = rfile.read()
                msg = struct.unpack("<100b",text)
                for i in range(len(开发者_开发知识库msg)):
                        packMessage  = packMessage + struct.pack("<b",msg[i])
                print "time:",datetime.datetime.now() - startTime

The file are binary files.

Initial time taken : 671 ms

on executing continuously for more than 10 times,time increases slowly. Last few values, 671ms . . . . 9.879 ms 88.686 ms 135.954 ms

I am using python-2.5.4 version.

If anyone had come across similar problem. Please provide me some inputs.

Thanks Das

---

From what I see, packMessage is growing monotonically:

packMessage  = packMessage + struct.pack("<b",msg[i])

If you repeat it many times, it may grow big, consume a lot of memory, and at some point your application may become much slower. Try looking at top or htop when you run your program (in top, press M to sort by memory allocation, f to add resident memory field).

Also opening and reading the same file every time is not the best solution from the performance point of view. Consider reading it only once before entering the loop.

---

Have you checked the number of file-handles your process has open? You may want yo use the with-statement to make sure they get closed when not needed anymore:

with open(filename, 'rb') as rfile:
    text = rfile.read()
    # etc.

When the with-block is left, the file will closed automatically.