java API or framework for queue processing

i need an open-source java API or framework for processing items in a queue. i can开发者_开发百科 develop something myself, but do not want to re-invent the wheel (and i don't have much experience in multi-threading). is there such a thing?

the closest solution that i can think of is a business process management (BPM) solution.

right now, i am using multiple Quartz jobs to process the items in my queue. it is not really working out because of scalability and concurrency issues.

---

Sounds like you'd want to use an Executor

---

A queue of what sort? How many items? Is Quartz not working out because it's too big or too small?

I'd give some serious thought to using message queues in something like OpenMQ.

---

You can use JMS with ActiveMQ and can create optimized queue system as well as ESB. And want to manage workflow based system then tpdi is right. Use JBoss jbpm.

You can process JMS messages with ThreadPool also. In this case, you can use Executors.

---

Would the actor model fit your process? It's based around the idea of asynchronously passing messages between other actors. So you can set up a simple state machine to model your process and have all the transitions handled concurrently.

---

You need to determine if the problem in is the framework you are using or your code. I suggest you measure how fast your application is running and how fast your framework will go if its not doing anything at all. (just passing trivial tasks around) You should be able to perform between 100K to 1 million tasks per second using your in process framework. Even using JMS you should be able to achieve 10K messages per second. If you need to do closer to 10 million tasks per second, I suggest you try grouping your tasks together so each task does more work.

I would be very surprised if your framework was the bottleneck in which case I would suggest using an Executor.

If the framework isn't the cause of your scalability and concurrency issues (which is more likely) you need to restructure your code so it can run for longer periods of time without inter dependencies. i.e. you have to fix your code, a framework won't do that for you.

---

I know it is 5 years late, but this might help someone else that has been driven into this question.

Nowadays, there is http://queues.io and it contains a whole lot of queuing (and messaging) frameworks...