注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

Oracle、MySQL资料及经验

.

 
 
 

日志

 
 

安装hadoop  

2016-07-27 13:44:35|  分类: hadoop与nosql |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |



grid用户下主节点上执行如下:

tar -zxvf hadoop-2.5.2.tar.gz 
$ mkdir /home/hadoop/data tmp

vi /home/hadoop/hadoop-2.5.2/etc/hadoop/hadoop-env.sh
#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=/usr/jdk1.8.0_101

$ find . -name hadoop-env.sh
./hadoop-2.5.2/etc/hadoop/hadoop-env.sh
cd /home/hadoop/hadoop-2.5.2/etc/hadoop
vi core-site.xml 
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://server:9000</value>           
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/grid/hadoop/tmp</value>
</property>
</configuration>

vi hdfs-site.xml 
<configuration>
<property>
<name>dfs.data.dir</name>
<value>/home/hadoop/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>

vi mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>server:9001</value>
</property>
</configuration>

vi masters
server

vi slaves 
server1
server2

把/home/hadoop/hadoop-2.5.2目录拷到其他节点

初始化:
$ cd /home/hadoop/hadoop-2.5.2/bin
$ ./hadoop namenode -format
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
16/07/26 16:25:02 INFO namenode.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = server/192.168.238.131
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 2.5.2
。。。。。。
16/07/26 16:25:04 INFO util.GSet: Computing capacity for map NameNodeRetryCache
16/07/26 16:25:04 INFO util.GSet: VM type       = 64-bit
16/07/26 16:25:04 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB
16/07/26 16:25:04 INFO util.GSet: capacity      = 2^15 = 32768 entries
16/07/26 16:25:04 INFO namenode.NNConf: ACLs enabled? false
16/07/26 16:25:04 INFO namenode.NNConf: XAttrs enabled? true
16/07/26 16:25:04 INFO namenode.NNConf: Maximum size of an xattr: 16384
16/07/26 16:25:04 INFO namenode.FSImage: Allocated new BlockPoolId: BP-452228410-192.168.238.131-1469521504695
16/07/26 16:25:04 INFO common.Storage: Storage directory /home/hadoop/tmp/dfs/name has been successfully formatted.
16/07/26 16:25:05 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
16/07/26 16:25:05 INFO util.ExitUtil: Exiting with status 0
16/07/26 16:25:05 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at server/192.168.238.131
************************************************************/


启动:
$cd /home/hadoop/hadoop-2.5.2/sbin
$ ./start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
16/07/26 17:08:08 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [server]
server: starting namenode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-namenode-server.localdomain.out
server1: starting datanode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-datanode-server1.localdomain.out
server2: starting datanode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-datanode-server2.localdomain.out
Starting secondary namenodes [0.0.0.0]
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
RSA key fingerprint is 43:b3:85:e0:61:b8:c9:9c:76:59:7f:13:81:a7:35:70.
Are you sure you want to continue connecting (yes/no)? yes
0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.
0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop-2.5.2/logs/hadoop-hadoop-secondarynamenode-server.localdomain.out
16/07/26 17:09:31 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
starting yarn daemons
resourcemanager running as process 3607. Stop it first.
server2: starting nodemanager, logging to /home/hadoop/hadoop-2.5.2/logs/yarn-hadoop-nodemanager-server2.localdomain.out
server1: starting nodemanager, logging to /home/hadoop/hadoop-2.5.2/logs/yarn-hadoop-nodemanager-server1.localdomain.out


检查启动的进程:
主节点:
$ /usr/jdk1.8.0_101/bin/jps
4021 NameNode
4197 SecondaryNameNode
3607 ResourceManager
4379 Jps

data节点:
$ /usr/jdk1.8.0_101/bin/jps
3234 DataNode
3333 NodeManager
3436 Jps


重启hadoop:
cd /home/hadoop/hadoop-2.5.2/sbin
$ ./stop-all.sh
$ ./start-all.sh 


http://192.168.86.128:50070 查看节点状态 









  评论这张
 
阅读(25)| 评论(0)
推荐 转载

历史上的今天

在LOFTER的更多文章

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017