Hdp安装问题杂解

5.在安装的时候遇到的问题

5.1使用ambari-server start的时候出现ERROR: Exiting with exit code -1.

5.1.1REASON: Ambari Server java process died with exitcode 255. Check /var/log/ambari-server/ambari-server.out for more information

解决:

由于是重新安装,所以在使用/etc/init.d/postgresql  initdb初始化数据库的时候会出现这个错误,所以需要

先用yum –y remove postgresql*命令把postgresql卸载

然后把/var/lib/pgsql/data目录下的文件全部删除

然后再配置postgresql数据库(执行1.6章节内容)

然后再次安装(3章节内容)

5.1.2在日志中有如下错误:ERROR [main] AmbariServer:820 - Failed to run the Ambari Server

com.google.inject.ProvisionException: Guice provision errors:

1) Error injecting method, java.lang.NullPointerException

at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)

at org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)

while locating org.apache.ambari.server.api.services.AmbariMetaInfo

for field at org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)

at org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)

while locating org.apache.ambari.server.controller.AmbariServer

1 error

at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)

at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)

at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)

Caused by: java.lang.NullPointerException

at org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)

at org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)

at org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)

at org.apache.ambari.server.stack.StackManager.(StackManager.java:119)

at org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()

at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)

at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)

at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)

at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

at com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)

at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

at com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)

at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)

at com.sun.proxy.$Proxy26.create(Unknown Source)

at org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)

5.2安装HDFS和HBASE的时候出现/usr/hdp/current/hadoop-client/conf  doesn‘t exist

5.2.1/etc/Hadoop/conf文件链接存在

是由于/etc/hadoop/conf和/usr/hdp/current/hadoop-client/conf目录互相链接,造成死循环,所以要改变一个的链接

cd /etc/hadoop

rm -rf conf

ln -s /etc/hadoop/conf.backup /etc/hadoop/conf

HBASE也会遇到同样的问题,解决方式同上

cd /etc/hbase

rm -rf conf

ln -s /etc/hbase/conf.backup /etc/hbase/conf

ZooKeeper也会遇到同样的问题,解决方式同上

cd /etc/zookeeper

rm -rf conf

ln -s /etc/zookeeper/conf.backup /etc/zookeeper/conf

5.2.2/etc/Hadoop/conf文件链接不存在

查看正确的配置,发现缺少两个目录文件config.backup和2.4.0.0-169,把文件夹拷贝到/etc/hadoop目录下

重新创建/etc/hadoop目录下的conf链接:

cd /etc/hadoop

rm -rf conf

ln -s /usr/hdp/current/hadoop-client/conf conf

问题解决

5.3在认证机器(Confirm Hosts)的时候出现错误Ambari agent machine hostname (localhost) does not match expected ambari server hostname

Ambari配置时在Confirm Hosts的步骤时,中间遇到一个很奇怪的问题:总是报错误:

Ambari agent machine hostname (localhost.localdomain) does not match expected ambari server hostname (xxx).

后来修改的/etc/hosts文件中

修改前:

127.0.0.1   localhost dsj-kj1

::1         localhost dsj-kj1

10.13.39.32     dsj-kj1

10.13.39.33     dsj-kj2

10.13.39.34     dsj-kj3

10.13.39.35     dsj-kj4

10.13.39.36     dsj-kj5

修改后:

127.0.0.1  localhost
localhost.localdomain localhost4 localhost4.localdomain4

::1          localhost localhost.localdomain localhost6
localhost6.localdomain6

10.13.39.32     dsj-kj1

10.13.39.33     dsj-kj2

10.13.39.34     dsj-kj3

10.13.39.35     dsj-kj4

10.13.39.36     dsj-kj5

感觉应该是走的ipv6协议,很奇怪,不过修改后就可以了。

5.4ambary-server重装

删除使用脚本删除

注意删除后要安装两个系统组件

yum -y install ruby*

yum -y install redhat-lsb*

yum -y install snappy*

安装参考3

5.5Ambari连接mysql设置

在主节点把mysql数据库连接包拷贝在/var/lib/ambary-server/resources目录下并改名为mysql-jdbc-driver.jar

cp
/usr/share/java/mysql-connector-java-5.1.17.jar
/var/lib/ambari-server/resources/mysql-jdbc-driver.jar

再在图形界面下启动hive

5.6在注册机器(Confirm Hosts)的时候出现错误Failed
to start ping port listener of: [Errno 98] Address already in use

某个端口或者进程一直陪占用

解决方法:

发现df命令一直执行没有完成,

[[email protected] ~]# netstat -lanp|grep 8670

tcp        0      0
0.0.0.0:8670               
0.0.0.0:*                  
LISTEN      2587/df

[[email protected] ~]# kill -9 2587

kill后,再重启ambari-agent问题解决

[[email protected] ~]# service ambari-agent restart

Verifying Python version compatibility...

Using python  /usr/bin/python2.6

ambari-agent is not running. No PID found at
/var/run/ambari-agent/ambari-agent.pid

Verifying Python version compatibility...

Using python  /usr/bin/python2.6

Checking for previously running Ambari Agent...

Starting ambari-agent

Verifying ambari-agent process status...

Ambari Agent successfully started

Agent PID at: /var/run/ambari-agent/ambari-agent.pid

Agent out at: /var/log/ambari-agent/ambari-agent.out

Agent log at: /var/log/ambari-agent/ambari-agent.log

5.7在注册机器(Confirm Hosts)的时候出现错误The following hosts have Transparent HugePages (THP) enabled。THP should
be disabled to avoid potential Hadoop performance issues

解决方法:

在Linux下执行:

echo never
>/sys/kernel/mm/redhat_transparent_hugepage/defrag

echo never >/sys/kernel/mm/redhat_transparent_hugepage/enabled

echo never >/sys/kernel/mm/transparent_hugepage/enabled

echo never >/sys/kernel/mm/transparent_hugepage/defrag

5.8启动hive的时候出现错误unicodedecodeerror ambari in position 117

查看/etc/sysconfig/i18n文件,发现内容如下:

LANG=”zh_CN.UTF8”

原来系统字符集设置成了中文,改成如下内容,问题解决:

LANG="en_US.UTF-8"

5.9安装Metrics的时候报如下错误,安装包找不到

1.failure: Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm
from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.

在ftp源服务器上执行命令:

cd /var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6

mkdir Updates-ambari-2.2.1.0

cp -r /var/www/html/ambari/Updates-ambari-2.2.1.0/ambari
/var/www/html/ambari/HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0

然后重新生成repodata

cd /var/www/html/ambari

rm -rf repodata

createrepo ./

2.failure: HDP-UTILS-1.1.0.20/repos/centos6/Updates-ambari-2.2.1.0/ambari/ambari-metrics-monitor-2.2.1.0-161.x86_64.rpm
from HDP-UTILS-1.1.0.20: [Errno 256] No more mirrors to try.

在/etc/yum.repos.d目录下删除mnt.repo,并使用yum clean all命令来清空yum的缓存

cd /ec/yum.repos.d

rm -rf mnt.repo

yum clean all

5.11jps 报process information unavailable解决办法

4791 -- process information unavailable

解决办法:

进入tmp目录,

cd /tmp

删除该目录下

名称为hsperfdata_{ username}的文件夹

然后jps,清净了。

脚本:

cd /tmp

ls -l | grep hsperf | xargs rm
-rf

ls -l | grep hsperf

5.12namenode启动报错在日志文件中ERROR
namenode.NameNode (NameNode.java:main(1712)) - Failed to start namenode

日志中还有java.net.BindException: Port in use: gmaster:50070

Caused by: java.net.BindException: Address
already in use

判断原因是50070上一次没有释放,端口占用

netstat下time_wait状态的tcp连接:

1.这是一种处于连接完全关闭状态前的状态;

2.通常要等上4分钟(windows server)的时间才能完全关闭;

3.这种状态下的tcp连接占用句柄与端口等资源,服务器也要为维护这些连接状态消耗资源;

4.解决这种time_wait的tcp连接只有让服务器能够快速回收和重用那些TIME_WAIT的资源:修改注册表[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\Tcpip\Parameters]添加dword值TcpTimedWaitDelay=30(30也为微软建议值;默认为2分钟)和MaxUserPort:65534(可选值5000
- 65534);

5.具体tcpip连接参数配置还可参照这里:http://technet.microsoft.com/zh-tw/library/cc776295%28v=ws.10%29.aspx

6.linux下:

vi /etc/sysctl.conf

新增如下内容:

net.ipv4.tcp_tw_reuse
= 1

net.ipv4.tcp_tw_recycle = 1

net.ipv4.tcp_syncookies=1

net.ipv4.tcp_fin_timeout=30

net.ipv4.tcp_keepalive_time=1800

net.ipv4.tcp_max_syn_backlog=8192

使内核参数生效:

[[email protected] ~]# sysctl -p

readme:

net.ipv4.tcp_syncookies=1 打开TIME-WAIT套接字重用功能,对于存在大量连接的Web服务器非常有效。

net.ipv4.tcp_tw_recyle=1

net.ipv4.tcp_tw_reuse=1 减少处于FIN-WAIT-2连接状态的时间,使系统可以处理更多的连接。

net.ipv4.tcp_fin_timeout=30 减少TCP KeepAlive连接侦测的时间,使系统可以处理更多的连接。

net.ipv4.tcp_keepalive_time=1800 增加TCP SYN队列长度,使系统可以处理更多的并发连接。

net.ipv4.tcp_max_syn_backlog=8192

5.13在启动的时候报错误resource_management.core.exceptions.Fail:
Execution of ‘ambari-sudo.sh  -H -E /usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh

在日志中有如下内容:

2016-03-31 13:55:28,090 INFO 
security.ShellBasedIdMapping
(ShellBasedIdMapping.java:updateStaticMapping(322)) - Not doing static UID/GID
mapping because ‘/etc/nfs.map‘ does not exist.

2016-03-31 13:55:28,096 INFO 
nfs3.WriteManager (WriteManager.java:(92)) - Stream timeout
is 600000ms.

2016-03-31 13:55:28,096 INFO 
nfs3.WriteManager (WriteManager.java:(100)) - Maximum open
streams is 256

2016-03-31 13:55:28,096 INFO 
nfs3.OpenFileCtxCache (OpenFileCtxCache.java:(54)) - Maximum
open streams is 256

2016-03-31 13:55:28,259 INFO 
nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:(205)) - Configured
HDFS superuser is

2016-03-31 13:55:28,261 INFO 
nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:clearDirectory(231)) - Delete
current dump directory /tmp/.hdfs-nfs

2016-03-31
13:55:28,269 WARN  fs.FileUtil
(FileUtil.java:deleteImpl(187)) - Failed to delete file or dir
[/tmp/.hdfs-nfs]: it still exists.

说明hdfs这个用户对/tmp没有权限

赋予权限给hdfs用户:

chown  hdfs:hadoop /tmp

再启动问题解决

5.14在安装ranger组件的时候出现错误连接不上mysql数据库rangeradmin用户和不能赋权的问题

在数据库中先删除所有rangeradmin用户,注意使用drop user命令:

drop user ‘rangeradmin‘@‘%‘;

drop user ‘rangeradmin‘@‘localhost‘;

drop user ‘rangeradmin‘@‘gmaster‘;

drop user ‘rangeradmin‘@‘gslave1‘;

drop user ‘rangeradmin‘@‘gslave2‘;

FLUSH PRIVILEGES;

再创建用户(注意gmaster是ranger安装的服务器机器名)

CREATE USER ‘rangeradmin‘@‘%‘ IDENTIFIED BY
‘rangeradmin‘;

GRANT ALL PRIVILEGES ON *.* TO
‘rangeradmin‘@‘%‘  with grant option;

CREATE USER ‘rangeradmin‘@‘localhost‘
IDENTIFIED BY ‘rangeradmin‘;

GRANT ALL PRIVILEGES ON *.* TO
‘rangeradmin‘@‘localhost‘  with grant option;

CREATE USER ‘rangeradmin‘@‘gmaster‘
IDENTIFIED BY ‘rangeradmin‘;

GRANT ALL PRIVILEGES ON *.* TO
‘rangeradmin‘@‘gmaster‘  with grant option;

FLUSH PRIVILEGES;

再查看权限:

SELECT DISTINCT CONCAT(‘User:
‘‘‘,user,‘‘‘@‘‘‘,host,‘‘‘;‘) AS query FROM mysql.user

select * from mysql.user where
user=‘rangeradmin‘ \G;

问题解决

5.15在ambari启动的时候出现错误:AmbariServer:820
- Failed to run the Ambari Server

这个问题困扰了我很久,最后通过查看源码找到了问题所在:

在/var/log/ambari-server/ambary-server.log文件中报有错误:

13 Apr 2016 14:16:01,723  INFO [main] StackDirectory:458 - Stack
‘/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS‘ doesn‘t contain an
upgrade directory

13 Apr 2016 14:16:01,723  INFO [main] StackDirectory:468 - Stack
‘/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS‘ doesn‘t contain
config upgrade pack file

13 Apr 2016 14:16:01,744  INFO [main] StackDirectory:484 - Role command
order info was loaded from file:
/var/lib/ambari-server/resources/stacks/HDP/2.1.GlusterFS/role_command_order.json

13 Apr 2016 14:16:01,840  INFO [main] StackDirectory:484 - Role command
order info was loaded from file:
/var/lib/ambari-server/resources/stacks/HDP/2.4/role_command_order.json

13 Apr 2016 14:16:01,927 ERROR [main]
AmbariServer:820 - Failed to run the Ambari Server

com.google.inject.ProvisionException: Guice
provision errors:

1) Error injecting method,
java.lang.NullPointerException

at
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:243)

at
org.apache.ambari.server.api.services.AmbariMetaInfo.class(AmbariMetaInfo.java:125)

while locating org.apache.ambari.server.api.services.AmbariMetaInfo

for field at
org.apache.ambari.server.controller.AmbariServer.ambariMetaInfo(AmbariServer.java:145)

at
org.apache.ambari.server.controller.AmbariServer.class(AmbariServer.java:145)

while locating org.apache.ambari.server.controller.AmbariServer

1 error

at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:987)

at
com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)

at
org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:813)

Caused by: java.lang.NullPointerException

at
org.apache.ambari.server.stack.StackModule.processRepositories(StackModule.java:665)

at
org.apache.ambari.server.stack.StackModule.resolve(StackModule.java:158)

at
org.apache.ambari.server.stack.StackManager.fullyResolveStacks(StackManager.java:201)

at
org.apache.ambari.server.stack.StackManager.(StackManager.java:119)

at
org.apache.ambari.server.stack.StackManager$$FastClassByGuice$$33e4ffe0.newInstance()

at
com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)

at
com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)

at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)

at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

at
com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)

at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)

at
com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:632)

at
com.sun.proxy.$Proxy26.create(Unknown Source)

at
org.apache.ambari.server.api.services.AmbariMetaInfo.init(AmbariMetaInfo.java:247)

at
org.apache.ambari.server.api.services.AmbariMetaInfo$$FastClassByGuice$$202844bc.invoke()

at
com.google.inject.internal.cglib.reflect.$FastMethod.invoke(FastMethod.java:53)

at
com.google.inject.internal.SingleMethodInjector$1.invoke(SingleMethodInjector.java:56)

at
com.google.inject.internal.SingleMethodInjector.inject(SingleMethodInjector.java:90)

at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)

at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)

at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at
com.google.inject.Scopes$1$1.get(Scopes.java:65)

at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)

at
com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)

at
com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)

at
com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)

at
com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)

at
com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)

at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)

at
com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)

at
com.google.inject.Scopes$1$1.get(Scopes.java:65)

at
com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)

at
com.google.inject.internal.InjectorImpl$4$1.call(InjectorImpl.java:978)

at
com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)

at
com.google.inject.internal.InjectorImpl$4.get(InjectorImpl.java:974)

...
2 more

解决方法:

后来发现在文件/var/lib/ambari-server/resources/stacks/HDP/2.4/repos/repoinfo.xml中的内容os这行原来的如下:

改成:

问题解决

5.16在启动hive的时候出现错误Error:
Duplicate key name ‘PCS_STATS_IDX‘ (state=42000,code=1061)

在HIVE安装的机器cd /var/lib/ambari-agent/data目录下有日志,日志中报如下错误:

Traceback (most recent call last):

File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 245, in

HiveMetastore().execute()

File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 219, in execute

method(env)

File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 58, in start

self.configure(env)

File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 72, in configure

hive(name = ‘metastore‘)

File
"/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py",
line 89, in thunk

return fn(*args, **kwargs)

File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py",
line 292, in hive

user = params.hive_user

File
"/usr/lib/python2.6/site-packages/resource_management/core/base.py",
line 154, in __init__

self.env.run()

File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 158, in run

self.run_action(resource, action)

File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 121, in run_action

provider_action()

File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 238, in action_run

tries=self.resource.tries, try_sleep=self.resource.try_sleep)

File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 70, in inner

result = function(command, **kwargs)

File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 92, in checked_call

tries=tries, try_sleep=try_sleep)

File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 140, in _call_wrapper

result = _call(command, **kwargs_copy)

File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py",
line 291, in _call

raise Fail(err_msg)

resource_management.core.exceptions.Fail:
Execution of ‘export
HIVE_CONF_DIR=/usr/hdp/current/hive-metastore/conf/conf.server ;
/usr/hdp/current/hive-metastore/bin/schematool -initSchema -dbType mysql
-userName hive -passWord [PROTECTED]‘ returned 1. WARNING: Use "yarn
jar" to launch YARN applications.

Metastore
connection URL:       
jdbc:mysql://a2slave1/hive?createDatabaseIfNotExist=true

Metastore
Connection Driver :   
com.mysql.jdbc.Driver

Metastore
connection User:       hive

Starting
metastore schema initialization to 1.2.1000

Initialization
script hive-schema-1.2.1000.mysql.sql

Error:
Duplicate key name ‘PCS_STATS_IDX‘ (state=42000,code=1061)

org.apache.hadoop.hive.metastore.HiveMetaException:
Schema initialization FAILED! Metastore state would be inconsistent !!

***
schemaTool failed ***

解决方法:

HIVE安装的机器拷贝脚本hive-schema-1.2.1000.mysql.sql到:

[[email protected] /]# scp /usr/hdp/2.4.0.0-169/hive/scripts/metastore/upgrade/mysql/hive-schema-1.2.1000.mysql.sql [email protected]:/usr/local/mysql

hive-schema-1.2.1000.mysql.sql                                                                  
100%   34KB  34.4KB/s  
00:00

在HIVE安装机器使用hive用户登陆,进入hive数据库,执行这个脚本

[[email protected] conf.server]# mysql -uhive
-p

Enter password:

Welcome to the MySQL monitor.  Commands end with ; or \g.

Your MySQL connection id is 505

Server version: 5.6.26-log MySQL Community
Server (GPL)

Copyright (c) 2000, 2015, Oracle and/or its
affiliates. All rights reserved.

Oracle is a registered trademark of Oracle
Corporation and/or its

affiliates. Other names may be trademarks
of their respective

owners.

Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to
clear the current input statement.

mysql> use hive;

Reading table information for completion of
table and column names

You can turn off this feature to get a
quicker startup with -A

Database changed

mysql> source
/usr/local/mysql/hive-schema-1.2.1000.mysql.sql;

问题解决

5.17在启动hive的时候出现错误Could
not create "increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.

在/var/log/hive中的日志文件hiveserver.log中记录有:

2016-04-15 10:45:20,446 INFO  [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(405)) - Starting HiveServer2

2016-04-15 10:45:20,573 INFO  [main]: metastore.ObjectStore
(ObjectStore.java:initialize(294)) - ObjectStore, initialize called

2016-04-15 10:45:20,585 INFO  [main]: metastore.MetaStoreDirectSql
(MetaStoreDirectSql.java:(140)) - Using direct SQL, underlying DB
is MYSQL

2016-04-15 10:45:20,585 INFO  [main]: metastore.ObjectStore (ObjectStore.java:setConf(277))
- Initialized ObjectStore

2016-04-15 10:45:20,590 WARN  [main]: metastore.ObjectStore
(ObjectStore.java:getDatabase(577)) - Failed to get database default, returning
NoSuchObjectException

2016-04-15 10:45:20,591 ERROR [main]: bonecp.ConnectionHandle
(ConnectionHandle.java:markPossiblyBroken(388)) - Database access problem.
Killing off this connection and all remaining connections in the connection
pool. SQL State = HY000

2016-04-15 10:45:20,600 WARN  [main]: metastore.HiveMetaStore
(HiveMetaStore.java:createDefaultDB(623)) - Retrying creating default database
after error: Could not create "increment"/"table"
value-generation container `SEQUENCE_TABLE` since autoCreate flags do not allow
it.

javax.jdo.JDOUserException: Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.

at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)

at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)

at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)

at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)

at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)

at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)

at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)

at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)

at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)

at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)

at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)

at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)

at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)

at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)

at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)

at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)

at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)

at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)

at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)

at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)

at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)

at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)

NestedThrowablesStackTrace:

Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.

org.datanucleus.exceptions.NucleusUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

at
org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)

at
org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)

at
org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)

at
org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)

at
org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)

at
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)

at
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)

at
org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)

at
org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)

at
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)

at
org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)

at
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)

at
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)

at
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)

at
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)

at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)

at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)

at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)

at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:621)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)

at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)

at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)

at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)

at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)

at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)

at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)

at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)

at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)

at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)

at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)

at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)

at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)

at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)

at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)

at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)

at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)

at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)

2016-04-15 10:45:20,607 WARN  [main]: metastore.ObjectStore
(ObjectStore.java:getDatabase(577)) - Failed to get database default, returning
NoSuchObjectException

2016-04-15 10:45:20,609 ERROR [main]:
bonecp.ConnectionHandle (ConnectionHandle.java:markPossiblyBroken(388)) -
Database access problem. Killing off this connection and all remaining
connections in the connection pool. SQL State = HY000

2016-04-15 10:45:20,617 INFO  [main]: server.HiveServer2
(HiveServer2.java:stop(371)) - Shutting down HiveServer2

2016-04-15 10:45:20,618 WARN  [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(442)) - Error starting HiveServer2 on
attempt 29, will retry in 60 seconds

java.lang.RuntimeException: Error applying
authorization policy on hive configuration: java.lang.RuntimeException: Unable
to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at
org.apache.hive.service.cli.CLIService.init(CLIService.java:114)

at
org.apache.hive.service.CompositeService.init(CompositeService.java:59)

at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:104)

at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:411)

at
org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:78)

at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:654)

at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:527)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at
org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: java.lang.RuntimeException:
java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:494)

at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)

at
org.apache.hive.service.cli.CLIService.init(CLIService.java:112)

...
12 more

Caused by: java.lang.RuntimeException:
Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1533)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)

at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)

at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)

at
org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)

at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:475)

...
14 more

Caused by:
java.lang.reflect.InvocationTargetException

at
sun.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source)

at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1531)

...
20 more

Caused by: javax.jdo.JDOUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

NestedThrowables:

org.datanucleus.exceptions.NucleusUserException:
Could not create "increment"/"table" value-generation
container `SEQUENCE_TABLE` since autoCreate flags do not allow it.

at
org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:549)

at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732)

at
org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752)

at
org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:530)

at
sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)

at
com.sun.proxy.$Proxy6.createDatabase(Unknown Source)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:605)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:625)

at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:462)

at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.(RetryingHMSHandler.java:66)

at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)

at
org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5789)

at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:199)

at
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.(SessionHiveMetaStoreClient.java:74)

...
24 more

Caused by:
org.datanucleus.exceptions.NucleusUserException: Could not create
"increment"/"table" value-generation container
`SEQUENCE_TABLE` since autoCreate flags do not allow it.

at
org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:261)

at
org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162)

at
org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197)

at
org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105)

at
org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005)

at
org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386)

at
org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827)

at
org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571)

at
org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513)

at
org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232)

at
org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414)

at
org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218)

at
org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065)

at
org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913)

at
org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217)

at
org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727)

...
39 more

最后发现是mysql数据库的binlog_format参数设置不正确,

原来设置的是STATEMENT,修改为MIXED

修改方法,在/etc/my.cnf文件中加上binlog_format=MIXED

然后重启mysql数据库

问题解决

5.18在使用hive命令进入hive的时候报错误Permission
denied: user=root, access=WRITE,
inode="/user/root":hdfs:hdfs:drwxr-xr-x  hive

解决方法:

1.       使用HDFS的命令行接口修改相应目录的权限,hadoop fs -chmod 777 /user,后面的/user是要上传文件的路径,不同的情况可能不一样,比如要上传的文件路径为hdfs://namenode/user/xxx.doc,则这样的修改可以,如果要上传的文件路径为hdfs://namenode/java/xxx.doc,则要修改的为hadoop
fs -chmod 777 /java或者hadoop fs -chmod 777 /,java的那个需要先在HDFS里面建立Java目录,后面的这个是为根目录调整权限。

脚本

su - hdfs

hadoop fs -chmod 777 /user

2.       在/etc/profile文件中加上系统的环境变量或java JVM变量里面添加export HADOOP_USER_NAME=hdfs(ambari使用的hadoop用户是hdfs),这个值具体等于多少看自己的情况,以后会运行HADOOP上的Linux的用户名。

export HADOOP_USER_NAME=hdfs

5.19Spark Thrift
Server组件启动后自动关闭

Spark Thrift Server组件在启动后自动关闭,查看/var/log/spark下的日志文件spark-hive-org.apache.spark.sql.hive.thriftserver.HiveThriftServer2-1-a2master.out,中有如下内容:

16/04/18
10:26:10 WARN DomainSocketFactory: The short-circuit local reads feature cannot
be used because libhadoop cannot be loaded.

16/04/18
10:26:10 INFO Client: Requesting a new application from cluster with 3
NodeManagers

16/04/18
10:26:10 INFO Client: Verifying our application has not requested more than the
maximum memory capability of the cluster (512 MB per container)

16/04/18
10:26:10 ERROR SparkContext: Error initializing SparkContext.

java.lang.IllegalArgumentException: Required executor memory (1024+384 MB) is
above the max threshold (512 MB) of this cluster! Please check the values of
‘yarn.scheduler.maximum-allocation-mb‘ and/or
‘yarn.nodemanager.resource.memory-mb‘.

at
org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:283)

at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:139)

at
org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)

at
org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)

at
org.apache.spark.SparkContext.(SparkContext.scala:530)

at
org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:56)

at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:76)

at
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)

at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)

at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)

at
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)

at
org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

16/04/18 10:26:11 INFO ContextHandler: stopped
o.s.j.s.ServletContextHandler{/stages/stage/kill,null}

分析:原因是spark的内存设置的太小

解决方法:在前台界面修改内存为1536M,并把配置在其它机器上更新,重启spark服务

5.20hbase在启动的时候出现错误Permission
denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,16020,1461130860883":hdfs:hdfs:drwxr-xr-x

在日志中有如下内容:

2016-04-20 15:42:11,640 INFO  [regionserver/gslave2/192.168.1.253:16020]
hfile.CacheConfig: Allocating LruBlockCache size=401.60 MB, blockSize=64 KB

2016-04-20 15:42:11,648 INFO  [regionserver/gslave2/192.168.1.253:16020]
hfile.CacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=433080,
freeSize=420675048, maxSize=421108128, heapSize=433080, minSize=400052704,
minFactor=0.95, multiSize=200026352, multiFactor=0.5, singleSize=100013176,
singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false,
cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false,
cacheDataCompressed=false, prefetchOnOpen=false

2016-04-20 15:42:11,704 INFO  [regionserver/gslave2/192.168.1.253:16020]
wal.WALFactory: Instantiating WALProvider of type class
org.apache.hadoop.hbase.wal.DefaultWALProvider

2016-04-20 15:42:11,729 INFO  [regionserver/gslave2/192.168.1.253:16020]
regionserver.HRegionServer: STOPPED: Failed initialization

2016-04-20 15:42:11,729 ERROR
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: Failed
init

org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

at
java.security.AccessController.doPrivileged(Native Method)

at
javax.security.auth.Subject.doAs(Subject.java:415)

at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)

at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)

at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)

at
org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)

at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)

at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)

at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)

at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)

at
org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)

at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)

at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)

at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)

at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)

at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)

at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)

at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)

at
java.lang.Thread.run(Thread.java:745)

Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

at
java.security.AccessController.doPrivileged(Native Method)

at
javax.security.auth.Subject.doAs(Subject.java:415)

at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

at
org.apache.hadoop.ipc.Client.call(Client.java:1411)

at
org.apache.hadoop.ipc.Client.call(Client.java:1364)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)

at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)

at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)

at
com.sun.proxy.$Proxy16.mkdirs(Unknown Source)

at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)

...
15 more

2016-04-20 15:42:11,732 FATAL
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer: ABORTING
region server gslave2,16020,1461138130424: Unhandled: Permission denied:
user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

at
java.security.AccessController.doPrivileged(Native Method)

at
javax.security.auth.Subject.doAs(Subject.java:415)

at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

org.apache.hadoop.security.AccessControlException:
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

at
java.security.AccessController.doPrivileged(Native Method)

at
javax.security.auth.Subject.doAs(Subject.java:415)

at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at
java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)

at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)

at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2589)

at
org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2558)

at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:820)

at
org.apache.hadoop.hdfs.DistributedFileSystem$16.doCall(DistributedFileSystem.java:816)

at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:816)

at
org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:809)

at
org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1815)

at
org.apache.hadoop.hbase.regionserver.wal.FSHLog.(FSHLog.java:488)

at
org.apache.hadoop.hbase.wal.DefaultWALProvider.init(DefaultWALProvider.java:97)

at
org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:147)

at
org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:179)

at
org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1624)

at
org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1362)

at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:899)

at
java.lang.Thread.run(Thread.java:745)

Caused by:
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
Permission denied: user=hbase, access=WRITE,
inode="/apps/hbase/data/WALs/gslave2,16020,1461138130424":hdfs:hdfs:drwxr-xr-x

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:319)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)

at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1771)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1755)

at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1738)

at
org.apache.hadoop.hdfs.server.namenode.FSDirMkdirOp.mkdirs(FSDirMkdirOp.java:71)

at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3905)

at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:1048)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:622)

at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)

at
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2151)

at
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2147)

at
java.security.AccessController.doPrivileged(Native Method)

at
javax.security.auth.Subject.doAs(Subject.java:415)

at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)

at
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2145)

at
org.apache.hadoop.ipc.Client.call(Client.java:1411)

at
org.apache.hadoop.ipc.Client.call(Client.java:1364)

at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)

at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)

at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)

at
com.sun.proxy.$Proxy15.mkdirs(Unknown Source)

at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:508)

at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at
java.lang.reflect.Method.invoke(Method.java:606)

at
org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279)

at
com.sun.proxy.$Proxy16.mkdirs(Unknown Source)

at
org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2587)

...
15 more

2016-04-20 15:42:11,732 FATAL
[regionserver/gslave2/192.168.1.253:16020] regionserver.HRegionServer:
RegionServer abort: loaded coprocessors are: []

2016-04-20 15:42:11,744 INFO  [regionserver/gslave2/192.168.1.253:16020]
regionserver.HRegionServer: Dump of metrics as JSON on abort: {

解决方法:

su - hdfs

hdfs dfs -chown -R hbase:hbase /apps/hbase

原文地址:https://www.cnblogs.com/tsxylhs/p/8716619.html

时间: 2024-08-30 05:06:42

Hdp安装问题杂解的相关文章

ambari+hdp安装

Yun2集群上Ambari+HDP安装 1. 安装环境说明: 系统版本:CentOS7 安装版本: Ambari版本:2.2.2.0 HDP版本:2.4.2.0 集群机器: Ambari Server :yun2 Ambari Agent:yun3 ,yun4,yun6,yun18,yun19,yun20,yun21 2. 操作系统环境准备 2.1 配置SSH免密码登录 主节点里root用户登录执行如下步骤: ssh-keygen –t rsa cd /root/.ssh cat id_rsa.

使用LVS实现负载均衡原理及安装配置详解

转:http://www.cnblogs.com/liwei0526vip/p/6370103.html 使用LVS实现负载均衡原理及安装配置详解 负载均衡集群是 load balance 集群的简写,翻译成中文就是负载均衡集群.常用的负载均衡开源软件有nginx.lvs.haproxy,商业的硬件负载均衡设备F5.Netscale.这里主要是学习 LVS 并对其进行了详细的总结记录. 一.负载均衡LVS基本介绍 LB集群的架构和原理很简单,就是当用户的请求过来时,会直接分发到Director

hbase-0.94安装方法详解

先决条件: 1)java环境,需要安装java1.6以上版本 2)hadoop环境,由于HBase架构是基于其他文件存储系统的,因此在分布式模式下安装Hadoop是必须的,但是,如果运行在单价模式下,此条件可以省略.Hadoop-1.2.1的安装方法参考 hadoop-1.2.1安装方法详解 注意:安装时要注意Hadoop和HBase之间的版本关系,如果不匹配,很可能会影响HBase系统的稳定性. 本帖教程采用的hadoop是hadoop-1.2.1,hbase采用的是hbase-0.94 hb

LNMP编译安装之msyql安装--图文详解

LNMP编译安装之msyql安装--图文详解 1.前言 本次安装采用源码安装,主要资源包从官网下载,次要依赖则使用yum进行安装,本篇只涉及mysql的安装,msyql远程登录,不涉及mysql具体配置.该教程纯属安装,不涉及任何重要知识点,老少皆宜. 2.安装步骤 2.1.下载boost(只下载不安装) wget http://nchc.dl.sourceforge.net/project/boost/boost/1.59.0/boost_1_59_0.tar.gz 2.2.解压bosot t

LNMP编译安装之nginx安装--图文详解

LNMP编译安装之nginx安装--图文详解 1.前言 本次安装采用源码安装,主要资源包从官网下载,次要依赖则使用yum进行安装,本篇只涉及nginx的安装,不涉及nginx的配置,对应nginx如何访问php,本篇也不涉及.该教程纯属安装,不涉及任何重要知识点,老少皆宜. 2.安装步骤 nginx官网:http://nginx.org/en/download.html 2.1.下载nginx wget http://nginx.org/download/nginx-1.12.0.tar.gz

使用 LVS 实现负载均衡原理及安装配置详解

使用 LVS 实现负载均衡原理及安装配置详解 来源:肖邦linux 发布时间:2017-02-19 阅读次数:106 0 负载均衡集群是 load balance 集群的简写,翻译成中文就是负载均衡集群.常用的负载均衡开源软件有nginx.lvs.haproxy,商业的硬件负载均衡设备F5.Netscale.这里主要是学习 LVS 并对其进行了详细的总结记录. 一.负载均衡LVS基本介绍 LB集群的架构和原理很简单,就是当用户的请求过来时,会直接分发到Director Server上,然后它把用

FY650四轴无人机安装全程详解(多图)

FY650四轴无人机安装全程详解(多图) 本文根据自己的安装实际过程,总结了箱后的一个比较合理的650四轴无人机安装顺序,以及各个步骤的注意事项,主要内容包括: 系统基本配置 机架安装 安装机臂与机架上板 安装机架底板与挂载架 安装折叠起落架 电子系统安装 安装马达与电调 安装飞控与电子设备 安装锂电池与云台 安装螺旋桨 一.基本配置 本次安装的基本配置如下: 机架: 飞越640sport机架×1 动力: 朗宇4110S 380kv马达×4 好盈铂金30A电调×4 锂电池:5200mAh 35C

Nagios服务器端安装部署详解(1)

下载地址如下: http://sourceforge.net/projects/nagios/files/ 具体详细下载地址:http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-4.0.6.tar.gz 1 开始下载: wget http://sourceforge.net/projects/nagios/files/nagios-4.x/nagios-4.0.6/nagios-4.0.6.tar.gz/download?us

LNMP编译安装之php安装--图文详解

LNMP编译安装之php安装--图文详解 1.前言 本次安装采用源码安装,主要资源包从官网下载,次要依赖则使用yum进行安装,本篇只涉及php的安装,不涉及php的配置,对应nginx如何访问php,本篇也不涉及.该教程纯属安装,不涉及任何重要知识点,老少皆宜. 2.安装步骤 php官网:http://www.php.net/ 2.1.下载php wget  http://php.net/get/php-7.1.7.tar.gz/from/a/mirror 2.2.解压php安装包 tar -z