Skip to main content

Posts

Showing posts from August 11, 2019

Hadoop Cluster Setup + Ubuntu + MultiNode + Yarn + Hive + Spark + Cassandra + Scala + Ignite

1 Name Node : 10.0.1.13 2 Data Nodes : 10.0.1.5 & 10.0.1.6 Name Node setup. ############### sudo apt-get update sudo apt-get install default-jre sudo apt-get install default-jdk root@namenode:~# java -version openjdk version "1.8.0_222" cd /opt/packages https://www-eu.apache.org/dist/hadoop/common/hadoop-2.8.5/hadoop-2.8.5.tar.gz tar -xvf hadoop-2.8.5.tar.gz cp -a hadoop-2.8.5 /etc/hadoop cd /etc/hadoop chown -R ubuntu. hadoop Make sure you are now logged in as Ubuntu user Then, add this to your .bashrc or .profile file. ############################################## export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64 export PATH="$PATH:$JAVA_HOME/bin" export HADOOP_HOME=/etc/hadoop export PATH=$PATH:$HADOOP_HOME/bin export PATH=$PATH:$HADOOP_HOME/sbin ############################################## ubuntu@namenode:~$ ssh-keygen -b 4096 ubuntu@namenode:~$ cat /home/ubuntu/.ssh/id_rsa.pub >> /home/ubuntu/.ssh/authorized