Why does Python's queue.Queue.get() permit returning early from timeouts?

UPDATE: This question is based on a faulty mental model of how Queue.get() was actually behaving, which was caused by some slightly ambiguous documentation but mainly by a buggy, hand-rolled implementation of timedelta.total_seconds(). I discovered this bug when trying to prove that the original answers where incorrect. Now that timedelta.total_seconds() is provided by Python (since 2.7), I will move to use that.

Sorry for the confusion.

---

This isn't a "Why does my code not run?" question, but a "What is the motivation behind this design decision?"

Since 2.3, Python's queue module contains a Queue class with a get method, that takes a timeout parameter. Here's the section from t开发者_JAVA百科he manual:

Queue.get([block[, timeout]])

Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available within that time. [...]

(Emphasis mine)

Note that it may raise an Empty exception even if it hasn't reached the timeout. In fact, I am seeing that behaviour on Ubuntu (but not Windows). It is bailing just a little early and it has had minor consequences on my code - I can code around it though.

Most blocking timeouts take a minimum timeout, which makes sense on a non-real-time OS, including Windows and Linux. There is no guarantee that the OS will context switch to your process or thread by any given deadline.

However, this one takes a maximum timeout. Can anyone explain how this design decision might make sense?

---

I think you are misinterpreting the documentation. It isn't saying it might raise the Empty exception after less than timeout seconds, it is saying it will block for at most timeout seconds. It might block for less than that, if it can satisfy the get.

I realize you are saying you see it raising Empty early, but honestly, that sounds like either a bug, or that you are relying on more accuracy than the system can provide. (It does seem as though, to obey the exact wording of the specification, an implementation should round timeout down to the resolution of its timer rather than up as you seem to desire.)

---

Unless you're doing real-time programming, a correct program must assume any timeout value is a guidance anyway, so it does not matter on which side you error. Note that queue does wait for at least the timeout if the queue stays empty (Confirmed by a look in the code which is basically a loop that exits if remaining (time) is less than 0). It returns earlier if an item is put into the queue in the meantime, though.

You're expecting this behavior:

[0] (Thread) A calls Queue.get(True, 10)
[0] B waits 9 seconds
[9] B does something on X
[11] A destroys X

But a valid schedule would be as well:

[0] (Thread) A calls Queue.get(True, 10)
[0] B waits 9 seconds
[9] Operating system is busy for 2 seconds
[11] A gets picked by the OS, destroys X    
[12] B does something on X

You need to fix the program to work independent of the actual timing of which thread is picked when.

---

There is definitely a bug in Queue.get, at least in python 2.6.6. On posix a queue.get(timeout=1) seems to exit (raising the Empty exception) almost immediately, whereas queue.get(timeout=2) is working fine.

I was using a single queue with concurrent threads *get*ing on it...