How to run a jar file in hadoop
I have created a jar file using the java file
javac -classpath /usr/local/hadoop/hadoop-core-1.0.3.jar -d /home/user1/dir Dictionary.java
/usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/hduser/dir
Now i have tried running this jar with various commands
user1@ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar
Output:
Warning: $HADOOP_HOME is deprecated.
RunJar jarFile [mainClass] args... user1@ubuntu:~$ /usr/local/hadoop/bin/hadoop jar Dictionary.jar DictionaryOutput: Warning: $HADOOP_HOME is deprecated. Exception in thread "main" java.lang.ClassNotFoundException: Dictionary at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355)at java.security.AccessController.doPrivileged(Native Method)at java.net.URLClassLoader.findClass(URLClassLoader.java:354)at java.lang.ClassLoader.loadClass(ClassLoader.java:423)at java.lang.ClassLoader.loadClass(ClassLoader.java:356)at java.lang.Class.forName0(Native Method)at java.lang.Class.forName(Class.java:264)at org.apache.hadoop.util.RunJar.main(RunJar.java:149)How can i run the jar in hadoop?
Basically, the directory that you are packaging into the jar is confusing the jar file in locating the main class file. Instead if trying do the following usr/lib/jvm/jdk1.7.0_07/bin/jar cf Dictionary.jar /home/user1/dir/Dictionary.class i.e. package the class file specifically into hadoop jar command and then run: /usr/local/hadoop/bin/hadoop jar Dictionary.jar Dictionary It just works fine provided that you have a main function in your class called Dictionary. The problem is when you package a full directory inside a jar then the jar also needs to be aware of the directory structure to locate the class file. For this, we need to have a well defined package hierarchy to define the class location. So, when you are packaging /home/hduser/dir/ into the jar, the jar is not aware of the location of the class file which is located deep inside this directory structure. For this you need to add a package name to your .java file according to the directory structure , for example home.hduser.dir and while running the hadoop jar command specify the class name with the package structure, for example home.user1.dir.Dictionary.