Can I call Hadoop APIs from my Axis2 Web Service?

I intend to develop a web service which can talk to Hadoop master node perform some tasks. These tasks include: 1. Starting and Stoping hadoop Cluster 2. Adding and Deleting slave nodes from hadoop cluster 3. Calling some of the APIs such as metrics one from web service.

I dont expect you guys to tell me everythin开发者_如何学JAVAg ( please dont!) but just tell me how can I call Hadoop APIs from my web service. Is including Hadoop jars in classpath enough? I think I will need some configuration too. Please guide me in setting up a simple call flow between web service and Hadoop Cluster.

---

Since starting and stopping are shellscripts you would not be able to restart the cluster from within the Hadoop jar.

So a quick listing what you have to do for your tasks:

1. Starting and stoping are shellscripts, you need to go for a Runtime.getRuntime().exec("YOUR SHELL SCRIPT") and let the shellscript start and stop the cluster.
2. Adding and deleting nodes is quite unusual, cause Hadoop will manage it itself. For decommissioning you have to edit the exclude file and refresh the nodes. This won't be a task that could be managed by the Hadoop API at all. Here you can read further: I want to make a large cluster smaller by taking out a bunch of nodes simultaneously. How can this be done?
   However you can refresh the nodes with the DFSAdmin class. The file itself must be edited on the host. See the next point how to set this up.
3. This one is exactly what you are talking about, you can simply put the jar and just the call things you need. BUT you have to setup a Configuration object that is containing the fs.default.name and mapred.job.tracker property. The values for these properties are the host names of the namenode and jobtracker. You find it in your config xmls' on the master. Then you can feed this object to a FileSystem.get(YOURCONFIGURATION) for example and work with the API.

I hope this ain't to much text :) Good luck!