We don’t allow questions seeking recommendatio开发者_StackOverflow社区ns for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this questionI'm currently in the market for a new queue system for jobs we have in our system. I've tried beanstalk but it's been unable to keep up with the load. I'm looking for a simple system to get up and running that I can put pieces of data in from producers and have consumers in Java, PHP and Python pull data off and process it.
Ideally I'd like to see features such as:
Job verification -> jobs are removed from the queue only when I've marked them as finished (in case of failures I don't have to put the jobs back in)
Priorities -> ability to prioritize jobs
Multiple channels -> ability to have one queue that can service several apps with separate data streams(or databases)
Disk Persistence -> ability to have jobs written to disk in case of failures
anyone have any good suggestions?
Currently looking at RabbitMQ
How about Apache ActiveMQ.
Accessible from Java, PHP, Python.
Supports all the features you requested.
RabbitMQ is good messaging system and there are bindings for Java, PHP, Python and many other languages.
The Berkeley database can be used to build a priority queue with bindings to most relevant languages. The HA (High Availability) configuration can make it distributed as well. I believe the Sun Grid Engine, for instance, uses just that to synchronize jobs.
精彩评论