What resources/references are available for multithreaded programming in Python? [closed]_问答_开发者

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers. 开发者_Go百科

We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.

Closed 3 years ago.

Improve this question

I'm evaluating the use of Python for a new project and ran through some basic tutorials but am looking for some recommendations and resources for multithreaded development in Python? How does it compare to other languages?

I'd recommend http://herbsutter.wordpress.com/ (scroll down to the efficient concurrency columns) for a really great overview of what multiprocessing is all about. Yes, that guy talks about concurrency in a C++ context, but most of it is applicable for any language.

If you mention concurrency and Python, a lot of people might yell out "global interpreter lock" and say Python can't really do concurrency. That's nonsense from the past. The Multiprocess module allows proper usage of multiple cores, with the nice benefit that it is really easy to split certain kinds of tasks over multiple machines as well as cores.

The multiprocessing module is fairly recent, and a consequence is that Python still has much unexplored potential in doing concurrency. But the bottom line is that Python has all the proper tools, from multiprocessing with its message passing and shared memory to green threads

Just to be sure, you mention multithreading and not multiprocessing. Information on Multithreading is available at that link. Multithreading is functionally different than multiprocessing. As mentioned in the other answers, I recommend Multiprocessing since it does a good job of utilizing multiple processors. I have used the multiprocessing modules successfully in a few projects.

The multiprocessing module is great and relieves from the limitations of the GIL.

Of course you can also have recourse to C based modules with threads etc.

here are 2 good python threading tutorials with code samples and some good background info:

http://linuxgazette.net/107/pai.html
http://www.ibm.com/developerworks/aix/library/au-threadingpython/

To multithread in python, you can use the python built-in threading module. However, be aware that if your threads are compute-bound rather than IO bound, you are likely to have worse performance when multithreading. If your execution threads are indeed compute-bound, take a look at the multiprocessing module or stackless.

Although the comments about multiprocessing being a very useful new package that allows easily taking advantage of multi-core systems, your requirements for multithreaded behaviour may not require that, and may also be entirely unaffected by the "limitations of the GIL".

We've built many applications, all of them multithreaded, which are used for various types of machine control (or "instrumentation", if you wish). The GIL is entirely a non-issue for us, perhaps because we're generally not CPU-bound though in the few areas where we are it's also never been an issue as the external packages we take advantage of release the GIL when required. (The intensity of the negative comments about the GIL always surprises me, probably because we work in somewhat atypical areas and ways.)

Python's threading support is highly effective, very easy to work with, and very robust. Not only is it rare that we have to concern ourselves with the lower-level concurrency issues (in some languages you have to spend half your time thinking about what to lock and when to lock it), but in the few areas where we still need to pay attention to it the available support makes it all relatively easy.

One key thing is to leverage the Queue class, since using it where possible often eliminates any remaining concerns about critical sections, race conditions, etc. Python's primitive objects (integers, dicts, lists, etc) are all about as thread-safe as one would like, though if you are new to threaded applications they won't protect you from your own ignorance. For experienced programmers in this area, I think Python's highly effective and easy to use.