Hadoop MapReduce

                                 HADOOP


To Start and Stop Hadoop

$ start-dfs.sh           --> To start hadoop
$ start-yarn.sh         ---> To start yarn
$ jps                         --> to view processes status
$ stop-yarn.sh         --> To Stop yarn
$ stop-dfs.sh           --> To stop hadoop


To Copy File

$ hadoop fs -mkdir vrush
$ hadoop fs -ls
$ hadoop fs -ls vrush
$ ls -l
$ hadoop fs -copyFromLocal data/*txt vrush


MapReduce for Word Counting

$ cd myc
$  ls -l
$  vi map.c
$  press esc button :q  --> To escape from vi editor
$  cc -o mymapper map.c
$  cc -o myreduce reduce.c
$  ls -l
$  hadoop jar $HADOOP_HOME/share/hadoop/tools/lib/hadoop-streaming-2.9.0.jar  -input  bunty -output  out1 -mapper mymapper  -reducer myreduce -file $HOME/myc/mymapper
  -file $HOME/myc/myreduce
$ hadoop fs -ls
$ hadoop fs -ls out1
$ hadoop fs -copyToLocal out1/part-00000 out1.txt
$ nano out1.txt


Comments

Post a Comment

Popular posts from this blog

Remote Method Invokation (RMI) in Java

Image
# Remote Method Invokation (RMI) in Java # What is RMI ?  RMI is an API that provide an mechanism to create distributed application in java. RMI allow an object to invoke the method of an object  running on another JVM(Java virtual Machine). RMI provide Remote Communication to the application using two special objects which are stub and skeleton . what is stub ? stub is an object act as gateway for client side application. All outgoing request are routed by stub. Following are the tasks of stub.     To establish the connection with the remote JVM. To write and trasmit the parameters to the remote object. (Marshalling)   stub wait for the result. To read the return value from remote object.(Unmarshalling) To send the result to its caller.   what is skeleton ? skeleton is an object act as gateway for server side application. All incoming request are routed by stub. Following are the tasks of skeleton.   To read the parameters from caller Invoke...
Read more

Changing the Title of Website on Button Click in React Hook tut 14

Changing the Title  of Website on Button Click in React Hook filename : ChgTitle.jsx import   React , {  useState ,  useEffect  }  from   'react' ; let   ChgTitle  = ()  =>  {      let  [ num ,  setNum ] =  useState ( 0 );      useEffect (() => {          document . title  =  `Clicked me 😎 ${ num }` ;     })      return  (          < >              < center >                  < br   />< br   />                  < button   onClick = { ()  => ...
Read more

useContext() in React tut 12

useContext() in React We can use useContext to receive the data directly from context provider instead of using complex context consumer  file hierchy :  ContextAPI.jsx ==> ComA.jsx ==> comB.jsx    we are sending data directly from ContextAPI.jsx to ComB.jsx. File : ContextAPI.jsx import   React , {  createContext  }  from   'react' ; import   ComA   from   './ComA' ; var   FilmName  =  createContext (); var   Earned  =  createContext (); let   ContextAPI  = ()  =>  {      return  (          < >              < FilmName.Provider   value = { 'GolMaal film' } >                  < Earned.Provider   value = { '25 crore' } > ...
Read more