hive高级数据类型

hive的高级数据类型主要包括：数组类型、map类型、结构体类型、集合类型，以下将分别详细介绍。

1）数组类型

array_type：array<data_type>

-- 建表语句

create table test.array_table(

name string,

age int,

addr array<string>

)

row format delimited fields terminated by ‘,‘

collection items terminated by ‘:‘

;

hive> desc test.array_table;

name string

age int

addr array<string>

-- 测试数据

chavin,28,beijing:shanghai:shenyang

nope,28,beijing:shanghai:taiyuan

wzq,28,beijing:dalian:shenyang:fuxin

zhangshuai,28,beijing:shenyang:fuxin

-- 加载数据

hive> load data local inpath ‘/home/hadoop/array_table.txt‘ into table test.array_table;

Loading data to table test.array_table

Time taken: 0.918 seconds

-- 查询

hive> select * from test.array_table;

chavin 28 ["beijing","shanghai","shenyang"]

nope 28 ["beijing","shanghai","taiyuan"]

wzq 28 ["beijing","dalian","shenyang","fuxin"]

zhangshuai 28 ["beijing","shenyang","fuxin"]

Time taken: 0.199 seconds, Fetched: 4 row(s)

hive> select addr[1] from test.array_table;

shanghai

dalian

shenyang

Time taken: 0.186 seconds, Fetched: 4 row(s)

hive> select addr[0] from test.array_table;

beijing

Time taken: 0.212 seconds, Fetched: 4 row(s)

2）map类型

map_type：map<primary_type,data_type>

-- 建表语句

create table test.map_table(

name string,

detail map<string,string>

)

row format delimited fields terminated by ‘\t‘

collection items terminated by ‘,‘

map keys terminated by ‘:‘

;

hive> desc test.map_table;

name string

detail map<string,string>

Time taken: 0.146 seconds, Fetched: 2 row(s)

-- 测试数据

chavin age:28,addr:beijing,phone:15998888888,email:[email protected]

nope age:28,addr:beijing,phone:13899998888,email:[email protected]

wzq age:28,addr:shanghai,phone:13899998888,email:[email protected]

zhangshuai age:28,addr:tianjing,phone:13899998888,email:[email protected]

-- 加载数据

hive> load data local inpath ‘/home/hadoop/map.txt‘ into table test.map_table;

Loading data to table test.map_table

Time taken: 0.641 seconds

-- 查询数据

hive> select * from test.map_table;

chavin {"age":"28","addr":"beijing","phone":"15998888888","email":"[email protected]"}

nope {"age":"28","addr":"beijing","phone":"13899998888","email":"[email protected]"}

wzq {"age":"28","addr":"shanghai","phone":"13899998888","email":"[email protected]"}

zhangshuai {"age":"28","addr":"tianjing","phone":"13899998888","email":"[email protected]"}

Time taken: 0.183 seconds, Fetched: 4 row(s)

hive> select detail from test.map_table;

{"age":"28","addr":"beijing","phone":"15998888888","email":"[email protected]"}

{"age":"28","addr":"beijing","phone":"13899998888","email":"[email protected]"}

{"age":"28","addr":"shanghai","phone":"13899998888","email":"[email protected]"}

{"age":"28","addr":"tianjing","phone":"13899998888","email":"[email protected]"}

Time taken: 0.177 seconds, Fetched: 4 row(s)

hive> select name,detail[‘email‘] from test.map_table;

chavin [email protected]

nope [email protected]

wzq [email protected]

zhangshuai [email protected]

Time taken: 0.179 seconds, Fetched: 4 row(s)

3）结构体类型

struct_type：struct<col_name:data_type,... ...>

-- 建表语句

create table test.struct_table(

name string,

detail struct<age:int,phone:string,email:string>

)

row format delimited fields terminated by ‘\t‘

collection items terminated by ‘,‘

;

hive> desc test.struct_table;

name string

detail struct<age:int,phone:string,email:string>

Time taken: 0.139 seconds, Fetched: 2 row(s)

-- 测试数据

chavin 28,15998888888,[email protected]

nope 28,13899998888,[email protected]

wzq 28,13899998888,[email protected]

zhangshuai 28,13899998888,[email protected]

-- 加载数据

hive> load data local inpath ‘/home/hadoop/struct.txt‘ into table test.struct_table;

Loading data to table test.struct_table

Time taken: 0.564 seconds

-- 查询数据

hive> select * from test.struct_table;

chavin {"age":28,"phone":"15998888888","email":"[email protected]"}

nope {"age":28,"phone":"13899998888","email":"[email protected]"}

wzq {"age":28,"phone":"13899998888","email":"[email protected]"}

zhangshuai {"age":28,"phone":"13899998888","email":"[email protected]"}

Time taken: 0.183 seconds, Fetched: 4 row(s)

hive> select name,detail.email from test.struct_table;

chavin [email protected]

nope [email protected]

wzq [email protected]

zhangshuai [email protected]

Time taken: 0.869 seconds, Fetched: 4 row(s)

4）集合类型

union_type：uniontype<data_type,data_type,... ...>

-- 建表语句

create table test.union_table(

detail uniontype<int,string>

)

row format delimited fields terminated by ‘\t‘

collection items terminated by ‘,‘

;

hive> desc test.union_table;

detail uniontype<int,string>

Time taken: 0.141 seconds, Fetched: 2 row(s)

原文地址：https://www.cnblogs.com/wcwen1990/p/9771728.html

时间： 2024-07-29 02:36:19

hive高级数据类型的相关文章

hadoop之hive集合数据类型

除了string,boolean,date等基本数据类型之外,hive还支持三种高级数据类型: 1.ARRAY ARRAY类型是由一系列相同数据类型的元素组成,这些元素可以通过下标来访问.比如有一个ARRAY类型的变量fruits,它是由['apple','orange','mango']组成,那么我们可以通过fruits[1]来访问元素orange,因为ARRAY类型的下标是从0开始的: 2.MAP MAP包含key->value键值对,可以通过key来访问元素.比如"userlist&

hive高级操作（优化，数据倾斜优化）

2019/2/21 星期四 hive高级操作(优化,数据倾斜优化) 分区表/桶表应用,skew,map-join //见hive的基本语法行列转换 hive 优化hive 优化思想Explain 的使用经典案例(distinct count) 数据倾斜的原因操作:关键词情形后果1.Join 其中一个表较小,但是key 集中分发到某一个或几个Reduce 上的数据远高于平均值 :2.大表与大表,但是分桶的判断字段0 值或空值过多这些空值都由一个reduce 处理,非常慢:3.group by

大数据技术之_08_Hive学习_04_压缩和存储（Hive高级）+ 企业级调优（Hive优化）

第8章压缩和存储(Hive高级)8.1 Hadoop源码编译支持Snappy压缩8.1.1 资源准备8.1.2 jar包安装8.1.3 编译源码8.2 Hadoop压缩配置8.2.1 MR支持的压缩编码8.2.2 压缩参数配置8.3 开启Map输出阶段压缩8.4 开启Reduce输出阶段压缩8.5 文件存储格式8.5.1 列式存储和行式存储8.5.2 TextFile格式8.5.3 Orc格式8.5.4 Parquet格式8.5.5 主流文件存储格式对比实验8.6 存储和压缩结合8.6.1 修

Hive 5、Hive 的数据类型和 DDL Data Definition Language) (1)

官方帮助文档:https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL Hive的数据类型 -- 扩展数据类型data_type : primitive_type | array_type | map_type | struct_type | union_type -- (Note: Available in Hive 0.7.0 and later) array_type : ARRAY < data_type >

hadoop笔记之Hive的数据类型

Hive的数据类型 Hive的数据类型前面说过,Hive是一个数据仓库,相当于一个数据库.既然是数据库,那么就必须能创建表,既然有表,那么当中就有列,列中就有对应的类型总的来讲,hive的数据类型可以分为三类: 基本数据类型: tinyint/smallint/int/bigint:整数类型 float/double:浮点数类型 boolean:布尔类型 string:字符串类型复杂数据类型 Array:数组类型,由一系列相同数据类型的元素组成 Map:集合类型,包含key–>value键

hive的数据类型和数据模型

关系数据库里有表(table),分区,hive里也有这些东西,这些东西在hive技术里称为hive的数据模型.今天本文介绍hive的数据类型,数据模型以及文件存储格式.这些知识大家可以类比关系数据库的相关知识. 首先我要讲讲hive的数据类型. Hive支持两种数据类型,一类叫原子数据类型,一类叫复杂数据类型. 原子数据类型包括数值型.布尔型和字符串类型,具体如下表所示: 基本数据类型类型描述示例 TINYINT 1个字节(8位)有符号整数 1 SMALLINT 2字节(16位)有符号整数

hive 高级查询1

hadoop hive 高级查询 select基础 1.0 一般查询 1)select * from table_name 2)select * from table_name where name='....' limit 1; 1.1cte和嵌套查询 1)with t as(select....) select * from t; 2)select * from(select....) a;(a一定要添加) 1.2列匹配正则表达式在添加数据前:SET hive.support.quoted

hive 高级查询

hadoop hive 高级查询 Hive聚合运算 - Group by (基本内置聚合函数)nmax, min, count, sum, avg 1)Hive基本内置聚合函数与group by 一起使用 2)支持按位置编号分组 set hive.groupby.orderby.position.alias=true; select name,sum(score) from table_name group by name;——>使用表达式 Hive聚合运算-hiving 1)对group by

高级数据类型（arrary、slice、map、ptr）

高级数据类型: 高级数据类型有数组.切片.map.指针.结构体.函数.接口.通道等,本文只介绍Arrary.Slice.map.ptr. 数组: (1)概念: 数组是同一种数据类型的集合.数组从声明时大小就已经确定,使用过程中可以修改值,但是数组的大小不可改变. (2)初始化方法一:使用初始化列表来设置数组元素的值 var testArray[3]int var numArray=[3]int{4,5} 方法二:自行推断数组的长度的方式 var numArray=[...]int{4,5} 方