What are some good ways to do intermachine locking?

Our server cluster consists of 20 mac开发者_运维百科hines, each with 10 pids of 5 threads. We'd like some way to prevent any two threads, in any pid, on any machine, from modifying the same object at the same time.

Our code's written in Python and runs on Linux, if that helps narrow things down.

Also, it's a pretty rare case that two such threads want to do this, so we'd prefer something that optimizes the "only one thread needs this object" case to be really fast, even if it means that the "one thread has locked this object and another one needs it" case isn't great.

What are some of the best practices?

---

If you want to synchronize across machines you need a Distributed Lock Manager.

I did some quick googling and came up with: Stackoverflow. Unfortunately they only suggest Java version, but it's a start.

If you are trying to synchronize access to files: Your filesystem should already have some wort of locking service in place. If not consider changing it.

---

I assume you came across this blog post http://amix.dk/blog/post/19386 during your googling?

The author demonstrates a simple interface to memcachedb which it uses as a dummy distributed lock manager. It's a great idea, and memcache is probably one of the faster thing's you'll be able to interface with. Note that it does use the more recently added with statement.

Here is an example usage from his blog post:

from __future__ import with_statement
import memcache
from memcached_lock import dist_lock

client = memcache.Client(['127.0.0.1:11211'])
with dist_lock('test', client):
   print 'Is there anybody out there!?'

---

if you can get the complete infrastructure for a distributed lock manager then go ahead and use that. But that infrastructure is not easy to setup! But here is a practical solution:

-designate the node with the lowest ip address as the the master node (that means if the node with lowest ip address hangs, a new node with lowest ip address will become new master)

-let all nodes contact the master node to get the lock on the object.

-let the master node use native lock semantics to get the lock.

this will simplify things unless you need complete clustering infrastructure and DLM to do the job.

---

Write code using immutable objects. Write objects that implement the Singleton Pattern.

Use a stable Distributed messaging technology such as IPC, webservices, or XML-RPC.

I would take a look at Twisted. They got plenty of solutions for such task.

I wouldn't use threads in Python esp with regards to the GIL, I would look at using Processes as working applications and use a comms technology as described above for intercommunications.

Your singleton class could then appear in one of these applications and interfaced via comms technology of choice.

Not a fast solution with all the interfacing, but if done correctly should be stable.

---

There may be a better way of doing this, but i would use the Lock class from the threading module to access the "protected" objects in a with statement, here would be an example:

from __future__ import with_statement    
from threading import Lock

mylock = Lock()
with mylock.acquire():
    [ 'do things with protected data here' ]
[ 'the rest of the code' ]

for more examples about Lock usages, have a look here.

Edit: this solution isn't suitable for this question as threading.Lock is not distributed, sorry