How to split python tasks across a network

I have a python program that performs several independent and time consuming processes. The 开发者_StackOverflow中文版python code is generally an automater, that calls into several batch files via popen.

The program currently takes several hours, so I'd like to split it up across multiple machines. How can I split tasks to process in parallel with python, over an intranet network?

---

There are many Python parallelisation frameworks out there. Just two of the options:

• The parallel computing facilities of IPython

• The parallelisation framework jug

---

For the remote execution you could use execnet. Do you have to distribute the data too?

---

I might suggest STAF. It's advertised as a software testing framework, yet it allows for distribution of activities across multiple PCs (and multiple platforms). You can run scripts, copy data, and easily communicate between your multiple sessions. Best of all, it's fairly easy to integrate with already existing scripts.