Hadoop MapReduce

                                 HADOOP

To Start and Stop Hadoop

$ start-dfs.sh           --> To start hadoop

$ start-yarn.sh         ---> To start yarn

$ jps                         --> to view processes status

$ stop-yarn.sh         --> To Stop yarn

$ stop-dfs.sh           --> To stop hadoop

To Copy File

$ hadoop fs -mkdir vrush

$ hadoop fs -ls

$ hadoop fs -ls vrush

$ ls -l

$ hadoop fs -copyFromLocal data/*txt vrush

MapReduce for Word Counting

$ cd myc

$  ls -l

$  vi map.c

$  press esc button :q  --> To escape from vi editor

$  cc -o mymapper map.c

$  cc -o myreduce reduce.c

$  ls -l

$  hadoop jar $HADOOP_HOME/share/hadoop/tools/lib/hadoop-streaming-2.9.0.jar  -input  bunty -output  out1 -mapper mymapper  -reducer myreduce -file $HOME/myc/mymapper

  -file $HOME/myc/myreduce

$ hadoop fs -ls

$ hadoop fs -ls out1

$ hadoop fs -copyToLocal out1/part-00000 out1.txt

$ nano out1.txt