Using the hadoop Native library

Hadoop has native implementations of certain components for performance reasons and for non-availability of Java implementations. These components are available in a single, dynamically-linked native library called the native hadoop library. On the *nix platforms the library is named libhadoop.so.
If the native library is not used, you will get warning shown as follows:

WARN util.NativeCodeLoader: Unable to load native-hadoop library ... 

We can compile hadoop from src to utilize the native library. The following steps can help you do so:

1. Download hadoop src and untar the source code.
2. Go to the src code directory and run the following compile command:

mvn package -Pdist,native -DskipTests -Dtar

If the compilation is successful, we can find the native library files under directory ./hadoop-dist/target/hadoop-2.2.0/lib/native

3. Copy the compiled native library files to your working hadoop native library folder.

cp hadoop-dist/target/hadoop-2.2.0/lib/native/* $HADOOP_HOME/lib/native

4. Add the following two lines into your $HADOOP_HOME/etc/hadoop/hadoop-env.sh file

export HADOOP_COMMON_LIB_NATIVE_DIR="$HADOOP_HOME/lib/native"
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native" 

5. No need to restart hadoop daemons and your native libraries will be used and the warning will be gone.