What's a good Java-based Master-Slave communication mechanism?_问答_开发者

I'm creating a Java application that requires master-slave communication between JVMs, possibly residing on the same physical machine. There will be a "master" server running inside a Java EE application server (i.e. JBoss) that will have "slave" clients connect to it and dynamically register itself for communication (that is the master will not know the IP addresses/ports of the slaves so cannot be configured in advance). The master server acts as a controller that will dole work out to the slaves and the slaves will periodically respond with notifications, so there would be bi-directional communication.

I was originally thinking of RPC-based systems where each side would be a server, but it could get complicated, so I'd prefer a mechanism where there's an open socket and they talk 开发者_开发问答back and forth.

I'm looking for a communication mechanism that would be low-latency where the messages would be mostly primitive types, so no serious serialization is necessary. Here's what I've looked at:

RMI
JMS: Built-in to Java, the "slave" clients would connect to the existing ConnectionFactory in the application server.
JAX-WS/RS: Both master and slave would be servers exposing an RPC interface for bi-directional communication.
JGroups/Hazelcast: Use shared distributed data structures to facilitate communication.
Memcached/MongoDB: Use these as "queues" to facilitate communication, though the clients would have to poll so there would be some latency.
Thrift: This does seem to keep a persistent connection, but not sure how to integrate/embed a Thrift server into JBoss
WebSocket/Raw Socket: This would work, but require a lot more custom code than I'd like.

Is there any technology I'm missing?

Edit: Also looked at:

JMX: Have the client connect to JBoss' JMX server and receive JMX notifications for bidirectional comms.

Well, I suggest JMS if you are looking for something which is based on Java. It has all the features that you are looking for plus a strong application server such as JBoss. However, another option which is not completely java based and is not using Queues would be using using HTTP protocol and JAXB (RESTful Web services). This is a very light way of communicating between two sides. Your objects would be transformed to XML by using JAXB, and would be transferred to the other side, and then you cast it back to object once you receive it.

Honestly, I would just stick with JMS. You have one queue your slaves can take messages out of, and one queue that they put them back into. You can set properties about who processed each message (for accounting) right on the envelope. You get persistence with many J2EE providers (glassfish, jboss).

Plus, you can easily move to multiple-server distributed JVM with it with no extra programming.

However, it may not fit the definition of "low-latency" in some cases.

Two more options:

Zookeeper (http://hadoop.apache.org/zookeeper/) Haven't used it, but sounds appropriate here. RabbitMQ (http://www.rabbitmq.com/) Low latency message queueing. Lots of flexibility here.

If you're looking for performance and both of your applications are java, without a firewall in between, then you can immediately exclude all the XML based protocols. If the communication is synchronous (master waits for the slave to complete the job), then use RMI, otherwise JMS. You can also use TCP directly for maximum performance. If there are many slaves that receive the same message, you could look into group communication toolkits like JGroups or Spread (faster but not 100% java). Ultimately, you might find that other people have already build what you're trying to build. Take a look at Gridgain.

You can also check Mule. I feel that It is perfect solution for your problem.

We have a similar application, and we just used Servlet on JBoss with "slaves" hitting it in a timer driven manner. This is okay, but not optimal for low latency.

We are now looking into Netty. You can use whatever protocol you want to use.. We'll probably use HTTP and JAXB. I guess this could be categorized as "WebSocket/Raw Socket", but it's much better than using a raw one..

You might want to have a look at Activatable. If the slaves are responding when the work is complete then rmi and Activatable could be a good solution in this instance. If you use the rmiregistry on the controller, then all of the slaves can register themselves with the controller easily, and these can be activated when required.

Depending on your requirements you might find that Raw Socket has less code than other approaches. It will certainly have the lowest latency. You can get it down to about 20 micro-seconds over loop back.

The more requirements you have the more likely you want a high level solution. However if all you want is something lightweight, raw sockets can be the simplest.

For thrift you can use TServlet as the entry point for your Thrift service. What you described though sounds like more of a problem to be solved with activemq or zookeeper.

WebSockets will help you with communication between Web server and client, not between two JVMs. If that's in fact what you want, http://fermiframework.org provides a java to JavaScript (server to client) RMI.

You can also take a look at Redisson project it has publish/subscribe and other shared distributed data structures for you needs.