当前位置:操作系统 > Unix/Linux >>

Hive Server 2 安装部署测试

Hive 0.11 包含了Hive Server 1 和 Hive Server 2,还包含1的原因是为了做到向下兼容性。从长远来看都会以Hive Server 2作为首选
 
 
 
 
1. 配置hive server监听端口和Host
[html]  
<property>  
  <name>hive.server2.thrift.port</name>  
  <value>10000</value>  
</property>  
<property>  
  <name>hive.server2.thrift.bind.host</name>  
  <value>test84.hadoop</value>  
</property>  
 
2. 配置kerberos认证,这样thrift client与hive server 2, hive server 2和hdfs 都由kerberos作认证
[html]  
<property>  
  <name>hive.server2.authentication</name>  
  <value>KERBEROS</value>  
  <description>  
    Client authentication types.  
       NONE: no authentication check  
       LDAP: LDAP/AD based authentication  
       KERBEROS: Kerberos/GSSAPI authentication  
       CUSTOM: Custom authentication provider  
               (Use with property hive.server2.custom.authentication.class)  
  </description>  
</property>  
<property>  
  <name>hive.server2.authentication.kerberos.principal</name>  
  <value>hadoop/_HOST@DIANPING.COM</value>  
</property>  
<property>  
  <name>hive.server2.authentication.kerberos.keytab</name>  
  <value>/etc/hadoop.keytab</value>  
</property>  
 
3. 设置impersonation,这样hive server会以提交用户的身份去执行语句,如果设置为false,则会以起hive server daemon的admin user来执行语句
[html]  
<property>  
  <name>hive.server2.enable.doAs</name>  
  <value>true</value>  
</property>  
 
执行命令$HIVE_HOME/bin/hive --service hiveserver2 会调用org.apache.hive.service.server.HiveServer2的main方法
hive log中输出日志信息如下:
[plain]  
2013-09-17 14:59:21,081 INFO  server.HiveServer2 (HiveStringUtils.java:startupShutdownMessage(604)) - STARTUP_MSG:   
/************************************************************  
STARTUP_MSG: Starting HiveServer2  
STARTUP_MSG:   host = test84.hadoop/10.1.77.84  
STARTUP_MSG:   args = []  
STARTUP_MSG:   version = 0.11.0  
STARTUP_MSG:   classpath = 略.................  
2013-09-17 14:59:21,957 INFO  security.UserGroupInformation (UserGroupInformation.java:loginUserFromKeytab(633)) - Login successful for user hadoop/test84.hadoop@DIANPING.COM using keytab file /etc/hadoop.keytab  
2013-09-17 14:59:21,958 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:OperationManager is inited.  
2013-09-17 14:59:21,958 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:SessionManager is inited.  
2013-09-17 14:59:21,958 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:CLIService is inited.  
2013-09-17 14:59:21,959 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:ThriftCLIService is inited.  
2013-09-17 14:59:21,959 INFO  service.AbstractService (AbstractService.java:init(89)) - Service:HiveServer2 is inited.  
2013-09-17 14:59:21,959 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:OperationManager is started.  
2013-09-17 14:59:21,960 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:SessionManager is started.  
2013-09-17 14:59:21,960 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:CLIService is started.  
2013-09-17 14:59:22,007 INFO  metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(409)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore  
2013-09-17 14:59:22,032 INFO  metastore.ObjectStore (ObjectStore.java:initialize(222)) - ObjectStore, initialize called  
2013-09-17 14:59:22,955 INFO  metastore.ObjectStore (ObjectStore.java:getPMF(267)) - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"  
2013-09-17 14:59:23,000 INFO  metastore.ObjectStore (ObjectStore.java:setConf(205)) - Initialized ObjectStore  
2013-09-17 14:59:23,909 INFO  metastore.HiveMetaStore (HiveMetaStore.java:logInfo(452)) - 0: get_databases: default  
2013-09-17 14:59:23,912 INFO  HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(238)) - ugi=hadoop/test84.hadoop@DIANPING.COM ip=unknown-ip-addr cmd=get_databases: default   
2013-09-17 14:59:23,933 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:ThriftCLIService is started.  
2013-09-17 14:59:23,948 INFO  service.AbstractService (AbstractService.java:start(104)) - Service:HiveServer2 is started.  
2013-09-17 14:59:24,025 INFO  security.UserGroupInformation (UserGroupInformation.java:loginUserFromKeytab(633)) - Login successful for user hadoop/test84.hadoop@DIANPING.COM using keytab file /etc/hadoop.keytab  
2013-09-17 14:59:24,047 INFO  thrift.ThriftCLIService (ThriftCLIService.java:run(435)) - ThriftCLIService listening on test84.hadoop/10.1.77.84:10000  
可以看到在HiveServer2已经变成一个CompisiteService了,它包含了一组service,包括OperationManager,SessionManager,CLIService,ThriftCLIService。并且在初始化的时候会建立HiveMetaStore连接,并调用get_databases命令来测试。最后启动thrift server,监听在test84.hadoop/10.1.77.84:10000端口上
 
1. Beeline访问hive server 2
Beeline是hive 0.11引入的新的交互式CLI,它基于SQLLine,可以作为Hive JDBC Client端访问Hive Server 2,启动一个beeline就是维护了一个session。
由于采用了kerberos认证方式,所以需要在本地有kerberos ticket,并且在connection url中指定hive server 2的service principal,此处为principal=hadoop/test84.hadoop@DIANPING.COM,另外用户名和密码可以不用填写,之后的语句会以当前ticket cache中principal的用户身份来执行。
[plain]  
-dpsh-3.2$ bin/beeline   
Beeline version 0.11.0 by Apache Hive  
beeline> !connect jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM  
scan complete in 2ms  
Connecting to jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM  
Enter username for jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM:   
Enter password for jdbc:hive2://test84.hadoop:10000/default;principal=hadoop/test84.hadoop@DIANPING.COM:   
Connected to: Hive (version 0.11.0)  
Driver: Hive (version 0.11.0)  
Tra
CopyRight © 2022 站长资源库 编程知识问答 zzzyk.com All Rights Reserved
部分文章来自网络,