Multi:Threading - Is this the right approach?

Experts -

I need some advice in the following scenario.

I have a configuration file with a list of tasks. Each task can have zero, one or more dependencies. I wanted to execute these tasks in parallel [right now they are being executed sequentially]

The idea is to have a main program to read the configuration file and load al开发者_StackOverflow社区l the tasks. Read individual tasks and give it to an executor [callable] that will perform the task and return results in a Future. When the task is submitted to the executor (thread) it will monitor for its dependencies to finish first and perform its own task.

Is this the right approach? Are there any other better approaches using java 1.5 features?

---

Sounds fine, but beware of Thread starvation deadlock. Basically, don't use a bounded thread pool.

Here is a example that illustrates this problem.
http://www.javaconcurrencyinpractice.com/listings/ThreadDeadlock.java

Also, if you have e.g. a pooled DB connection, you might run into problem, too. 10 threads can block, holding all the pooled connection, waiting for the 11th thread that can't acquire the 11th pooled connection because there isn't anymore available..

---

What you have described is the Java5/6 approach.

Just make sure that your tasks/Callables are threadsafe / don't share state.

---

Consider using ValueFuture (from guava) to communicate between tasks. Add a listener to each ValueFuture which submits a new task to the ExecutorService only when all of its inputs are available. That way you will never have a thread block part-way through a task waiting for another task to finish.

---

Database activity, archiving and rule processing are perfect uses for Ant (or even Maven)

Don't reinvent this. Just use Ant.

Write Ant Tasks for each task. Define the dependencies in XML, using Ant's dependency rules.

Turn Ant loose to run your tasks based on the dependencies. You can start out with "executable" Ant Tasks (http://ant.apache.org/manual/tasksoverview.html#exec) rather than defining your own subclass of Task.

Ant understands parallel tasks. You don't need to do anything but declare the tasks as parallel. http://ant.apache.org/manual/Tasks/parallel.html

Don't invent any new software for this. Use Ant.

---

Actually, you do want multi-threading when the cost of starting the multiple processes is expensive.

Which, frankly, is almost anything that spins up a copy of the JVM.

To answer the OP, ANT is able to run tasks in parallel but NOT targets in parallel. So, even if you do have a build.xml file that you'd like to change to run parallel, you'll have to do a possible major rewrite. (I think that was a rather odd design decision made by the ANT developers since you've already got your dependency graph described by the targets themselves.)