MULTINODE DEPLOYMENT

STEPS:

1. SSH COPY ID TO ALL NODES : 

2. CONFIGURE MASTERS AND SLAVES

3. CONFIGURE CORE-SITE.XML AND MAPRED-SITE.XML 

4. FORMAT DFS 

5. START-ALL.SH

STEP 1
SSH :

FROM HNNAME (NAMENODE) WE NEED TO GENERATE SSH KEY AND DISTRIBUTE IT TO ALL THE SLAVE NODES AND ALSO SECONDARY NAME NODE

ssh-copy-id -i $HOME/.ssh/id_rsa.pub hadoop@coed159
ssh-copy-id -i $HOME/.ssh/id_rsa.pub hadoop@coed160
ssh-copy-id -i $HOME/.ssh/id_rsa.pub hadoop@coed162
ssh-copy-id -i $HOME/.ssh/id_rsa.pub hadoop@coed163

(USERNAME@SYSTEMNAME)

COPY FINGERPRINT : GIVE YES

DO THE SAME FOR ALL DATA NODES AND FOR SECONDARY NAME NODE

CHECK

ssh coed159

SHOULD NOT ASK FOR PASSWORD

STEP 2:

CONFIGURATION OF MASTERS AND SLAVES
NEED TO DO IT ON NAMENODE ALONE : (NOT ON THE DATA NODES AND SECONDARY NAMENODE)
GO TO NAMENODE

cd /usr/local/hadoop/conf

FIND THE TWO FILES
masters
slaves

MASTERS FOR NAMENODE AND SECONDARY NAMENODE
SLAVES FOR DATANODES

sudo nano /usr/local/hadoop/conf/masters

BY DEFAULT IT WILL CONTAIN LOCALHOST

CHANGE IT TO NAME OF NAME NODE

coed161

Ctrl + o to save
Enter
Ctrl + x to exit

sudo nano /usr/local/hadoop/conf/slaves

BY DEFAULT IT WILL CONTAIN LOCALHOST

CHANGE IT TO CONTAIN ALL DATANODE NAMES ONE PER LINE

coed159
coed160
coed162
coed163

Ctrl + o to save
Enter
Ctrl + x to exit


STEP 3:

GO TO SLAVES/SECONDARY NAMENODE AND WE NEED TO POINT THEM TO THE MASTER

sudo nano /usr/local/hadoop/conf/core-site.xml

CHECK WHETHER IT IS POINTING TO HNNAME OR NAMENODE NAME IN FS.DEFAULT.NAME

hdfs://coed161:10001 should be there 

if it is pointing to localhost:10001 change it to coed161


Ctrl + o to save
Enter
Ctrl + x to exit

SIMILARLY FOR MAPRED-SITE.XML

sudo nano /usr/local/hadoop/conf/mapred-site.xml

CHECK IF IT POINTING TO THE JOB TRACKER IN NAMENODE

coed161:10002

CHANGE IF IT IS LOCALHOST:10002

REMOVE LOCALHOST ENTRIES FROM /ETC/HOSTS FILE

sudo nano /etc/hosts

remove localhost and entries for 127.0.0.1


STEP 4:

IF CONVERTING THE EXISTING SINGLE NODE INSTALLATION THEN 
YOU MUST DELETE THE /USR/LOCAL/HADOOP/TMP
AND THEN CREATE IT AGAIN IN ALL THE NODES AND THEN FORMAT IT FROM NAMENODE ALONE

TO REMOVE DIRECTORY
sudo rm -r /usr/local/hadoop/tmp


CREATE TEMP DIRECTORY
sudo mkdir /usr/local/hadoop/tmp


sudo chown hadoop /usr/local/hadoop/tmp

sudo chown hadoop /usr/local/hadoop

FORMAT NAMENODE

hadoop namenode -format

CHECK FOR FORMAT SUCCESSFULL MESSAGE


STEP 5:

start-all.sh

jps

CHECK EACH SYSTEM SEPARATELY TO FIND SPECIFIC DAEMONS RUNNING ON EACH OF THEM

CHECK NUMBER OF LIVE NODES IN WEB GUI INTERFACE

stop-all.sh