Threads in Python again

guys!

My application is a bot. It simply receives a message, process it and returns result.

But there are a lot of messages and I'm creating s开发者_开发百科eparate thread for processing each, but it makes an application slower (not a bit).

So, Is it any way to reduce CPU usage by replacing threads with something else?

---

You probably want processes rather than threads. Spawn processes at startup, and use Pipes to talk to them.

http://docs.python.org/dev/library/multiprocessing.html

---

Threads and processes have the same speed. Your problem is not which one you use, but how many you use.

The answer is to only have a fixed couple of threads or processes. Say 10. You then create a Queue (use the Queue module) to store all messages from your robot. The 10 threads will constantly be working, and everytime they finish, they wait for a new message in the Queue.

This saves you from the overhead of creating and destroying threads. See http://docs.python.org/library/queue.html for more info.

def worker():
    while True:
        item = q.get()
        do_work(item)
        q.task_done()

q = Queue()
for i in range(num_worker_threads):
     t = Thread(target=worker)
     t.daemon = True
     t.start()

for item in source():
    q.put(item)

q.join()       # block until all tasks are done

---

You could try creating only a limited amount of workers and distribute work between them. Python's multiprocessing.Pool would be the thing to use.

---

You might not even need threads. If your server can handle each request quickly, you can just make it all single-threaded using something like Twisted.