I'm writing a script using two separate thread one doing file reading operation and the other doing appending, both threads run fairly frequently.
My question is, if one thread happens to read the file while the other is just in the middle of appending strings such as "This is a test" into this file, what would happen?
I know if you are appending a smaller-than-buffer stri开发者_运维知识库ng, no matter how frequently you read the file in other threads, there would never be incomplete line such as "This i" appearing in your read file, I mean the os would either do: append "This is a test" -> read info from the file; or: read info from the file -> append "This is a test" to the file; and such would never happen: append "This i" -> read info from the file -> append "s a test".
But if "This is a test" is big enough(assuming it's a bigger-than-buffer string), the os can't do appending job in one operation, so the appending job would be divided into two: first append "This i" to the file, then append "s a test", so in this kind of situation if I happen to read the file in the middle of the whole appending operation, would I get such result: append "This i" -> read info from the file -> append "s a test", which means I might read a file that includes an incomplete string?
If you're worried about this, just have your consumer look for a special character (endline would work) so that it knows that there wasn't an incomplete write. So your producer (the one writing data to the file) can output partial data but the consumer (one reading from the file) will know it only got a partial write.
Is there a reason you're not using a PIPE instead of a file? And is there a reason you're using threading? You don't really gain anything except for maybe simplicity in coding but IMO you may as well have separate processes, then you can get a gain from this model.
Added: Unfortunately this I/O stuff is not just how Python handles things but how the OS handles things. Everything you've said about worrying about the buffer is true.
http://docs.python.org/library/functions.html#open
I would try to figure out what your buffer size is and for that I don't know even how to check. I'm using OSX anyways.
精彩评论