你会在Hive中使用GIS function么？

应用场景

目前，Hive表中存在需要把经纬度转化为距离，和一些其他相关的计算函数，但是在hive中并没有集成这些函数。当然我们可以自定义UDF函数，但是现在提供一个更加简单的方法，通过源码编译的方式，集成GIS函数。

环境

<profile>
      <id>hadoop-2.7</id>
      <properties>
        <hadoop.version>2.7.3</hadoop.version>
      </properties>
</profile>

 <profile>
      <id>hive-1.2</id>
      <properties>
        <hive.version>1.2.1</hive.version>
      </properties>
 </profile>

<java.source.version>1.8</java.source.version>
<java.target.version>1.8</java.target.version>
<hadoop.version>2.7.3</hadoop.version>
<hive.version>1.2.1</hive.version>

下载源码：

源码地址
在github上下载源码在本地idea进行编译，修改最外层的pom.xml,修改hadoop、hive、java版本为生产环境中版本

  • idea中添加项目

  • 打包

    打包后会出现如下两个jar包：

然后下载最新的esri-geometry-java中的esri-geometry-api-2.2.1.jar，上传三个jar到linux系统(jar权限设置成最高),在hive的shell控制台输入添加jar和创建函数语句。

spatial-sdk-json-2.1.1-SNAPSHOT.jar
spatial-sdk-hive-2.1.1-SNAPSHOT.jar

然后下载最新的esri-geometry-java中的esri-geometry-api-2.2.1.jar，上传三个jar到linux系统(jar权限设置成最高),在hive的shell控制台输入添加jar和创建函数语句。

创建函数

如果jar包在本地则创建临时函数，只在当前session有效，我们可以把jar包上传到HDFS上创建永久函数。

add jar  /iDatalight/jars/esri-geometry-api-2.2.1.jar;
add jar  /iDatalight/jars/spatial-sdk-json-2.1.0.jar;
add jar  /iDatalight/jars/spatial-sdk-hive-2.1.0.jar;

自定义函数使用可以参考官方文档UDF文档,创建自定义函数可以参考spatial-framework-for-hadoop项目的hive下function-ddl.sql，但创建永久函数需要去掉temporary。

CREATE FUNCTION STPoint as ‘com.esri.hadoop.hive.ST_Point‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create function ST_AsBinary as ‘com.esri.hadoop.hive.ST_AsBinary‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_AsGeoJSON as ‘com.esri.hadoop.hive.ST_AsGeoJson‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_AsJSON as ‘com.esri.hadoop.hive.ST_AsJson‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_AsShape as ‘com.esri.hadoop.hive.ST_AsShape‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_AsText as ‘com.esri.hadoop.hive.ST_AsText‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeomFromJSON as ‘com.esri.hadoop.hive.ST_GeomFromJson‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeomFromGeoJSON as ‘com.esri.hadoop.hive.ST_GeomFromGeoJson‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeomFromShape as ‘com.esri.hadoop.hive.ST_GeomFromShape‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeomFromText as ‘com.esri.hadoop.hive.ST_GeomFromText‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeomFromWKB as ‘com.esri.hadoop.hive.ST_GeomFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_PointFromWKB as ‘com.esri.hadoop.hive.ST_PointFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_LineFromWKB as ‘com.esri.hadoop.hive.ST_LineFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_PolyFromWKB as ‘com.esri.hadoop.hive.ST_PolyFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MPointFromWKB as ‘com.esri.hadoop.hive.ST_MPointFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MLineFromWKB as ‘com.esri.hadoop.hive.ST_MLineFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MPolyFromWKB as ‘com.esri.hadoop.hive.ST_MPolyFromWKB‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeomCollection as ‘com.esri.hadoop.hive.ST_GeomCollection‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create function ST_GeometryType as ‘com.esri.hadoop.hive.ST_GeometryType‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_LineString as ‘com.esri.hadoop.hive.ST_LineString‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Polygon as ‘com.esri.hadoop.hive.ST_Polygon‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MultiPoint as ‘com.esri.hadoop.hive.ST_MultiPoint‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MultiLineString as ‘com.esri.hadoop.hive.ST_MultiLineString‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MultiPolygon as ‘com.esri.hadoop.hive.ST_MultiPolygon‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_SetSRID as ‘com.esri.hadoop.hive.ST_SetSRID‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_SRID as ‘com.esri.hadoop.hive.ST_SRID‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_IsEmpty as ‘com.esri.hadoop.hive.ST_IsEmpty‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_IsSimple as ‘com.esri.hadoop.hive.ST_IsSimple‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Dimension as ‘com.esri.hadoop.hive.ST_Dimension‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_X as ‘com.esri.hadoop.hive.ST_X‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Y as ‘com.esri.hadoop.hive.ST_Y‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MinX as ‘com.esri.hadoop.hive.ST_MinX‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MaxX as ‘com.esri.hadoop.hive.ST_MaxX‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MinY as ‘com.esri.hadoop.hive.ST_MinY‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MaxY as ‘com.esri.hadoop.hive.ST_MaxY‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_IsClosed as ‘com.esri.hadoop.hive.ST_IsClosed‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_IsRing as ‘com.esri.hadoop.hive.ST_IsRing‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Length as ‘com.esri.hadoop.hive.ST_Length‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeodesicLengthWGS84 as ‘com.esri.hadoop.hive.ST_GeodesicLengthWGS84‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Area as ‘com.esri.hadoop.hive.ST_Area‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Is3D as ‘com.esri.hadoop.hive.ST_Is3D‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Z as ‘com.esri.hadoop.hive.ST_Z‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MinZ as ‘com.esri.hadoop.hive.ST_MinZ‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MaxZ as ‘com.esri.hadoop.hive.ST_MaxZ‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_IsMeasured as ‘com.esri.hadoop.hive.ST_IsMeasured‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_M as ‘com.esri.hadoop.hive.ST_M‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MinM as ‘com.esri.hadoop.hive.ST_MinM‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_MaxM as ‘com.esri.hadoop.hive.ST_MaxM‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_CoordDim as ‘com.esri.hadoop.hive.ST_CoordDim‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_NumPoints as ‘com.esri.hadoop.hive.ST_NumPoints‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_PointN as ‘com.esri.hadoop.hive.ST_PointN‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_StartPoint as ‘com.esri.hadoop.hive.ST_StartPoint‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_EndPoint as ‘com.esri.hadoop.hive.ST_EndPoint‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_ExteriorRing as ‘com.esri.hadoop.hive.ST_ExteriorRing‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_NumInteriorRing as ‘com.esri.hadoop.hive.ST_NumInteriorRing‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_InteriorRingN as ‘com.esri.hadoop.hive.ST_InteriorRingN‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_NumGeometries as ‘com.esri.hadoop.hive.ST_NumGeometries‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_GeometryN as ‘com.esri.hadoop.hive.ST_GeometryN‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create function ST_Centroid as ‘com.esri.hadoop.hive.ST_Centroid‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create  function ST_Contains as ‘com.esri.hadoop.hive.ST_Contains‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Crosses as ‘com.esri.hadoop.hive.ST_Crosses‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Disjoint as ‘com.esri.hadoop.hive.ST_Disjoint‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_EnvIntersects as ‘com.esri.hadoop.hive.ST_EnvIntersects‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Envelope as ‘com.esri.hadoop.hive.ST_Envelope‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Equals as ‘com.esri.hadoop.hive.ST_Equals‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Overlaps as ‘com.esri.hadoop.hive.ST_Overlaps‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Intersects as ‘com.esri.hadoop.hive.ST_Intersects‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Relate as ‘com.esri.hadoop.hive.ST_Relate‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Touches as ‘com.esri.hadoop.hive.ST_Touches‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Within as ‘com.esri.hadoop.hive.ST_Within‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create  function ST_Distance as ‘com.esri.hadoop.hive.ST_Distance‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Boundary as ‘com.esri.hadoop.hive.ST_Boundary‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Buffer as ‘com.esri.hadoop.hive.ST_Buffer‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_ConvexHull as ‘com.esri.hadoop.hive.ST_ConvexHull‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Intersection as ‘com.esri.hadoop.hive.ST_Intersection‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Union as ‘com.esri.hadoop.hive.ST_Union‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Difference as ‘com.esri.hadoop.hive.ST_Difference‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_SymmetricDiff as ‘com.esri.hadoop.hive.ST_SymmetricDiff‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_SymDifference as ‘com.esri.hadoop.hive.ST_SymmetricDiff‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create  function ST_Aggr_ConvexHull as ‘com.esri.hadoop.hive.ST_Aggr_ConvexHull‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Aggr_Intersection as ‘com.esri.hadoop.hive.ST_Aggr_Intersection‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;
create  function ST_Aggr_Union as ‘com.esri.hadoop.hive.ST_Aggr_Union‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create  function ST_Bin as ‘com.esri.hadoop.hive.ST_Bin‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

create  function ST_BinEnvelope as ‘com.esri.hadoop.hive.ST_BinEnvelope‘ using jar ‘hdfs:///iDatalight/jars/esri-geometry-api-2.2.2.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-hive-2.1.1.jar‘,jar ‘hdfs:///iDatalight/jars/spatial-sdk-json-2.1.1.jar‘;

原文地址：https://blog.51cto.com/14309075/2416059

时间： 2024-11-06 07:23:43

你会在Hive中使用GIS function么？的相关文章

Hive中分组取前N个值

分享两篇文章,结合看更清楚一点. 背景 假设有一个学生各门课的成绩的表单,应用hive取出每科成绩前100名的学生成绩. 这个就是典型在分组取Top N的需求. 解决思路 对于取出每科成绩前100名的学生成绩,针对学生成绩表,根据学科,成绩做order by排序,然后对排序后的成绩,执行自定义函数row_number(),必须带一个或者多个列参数,如ROW_NUMBER(col1, ....),它的作用是按指定的列进行分组生成行序列.在ROW_NUMBER(a,b) 时,若两条记录的a,b列相同

使用sqoop从mysql往hive中增量导数据shell脚本

一:sqoop增量导入的两种方式 Incremental import arguments: Argument Description --check-column (col) Specifies the column to be examined when determining which rows to import. (the column should not be of type CHAR/NCHAR/VARCHAR/VARNCHAR/ LONGVARCHAR/LONGNVARCHA

hive中udf读写hbase

在大数据开发过程中经常会遇到,将hive中处理后的结果写入hbase中,每次都要写java程序会非常浪费时间,我们就想了一个办法 ,用hive的udf来实现. 只需要调用同一个udf,将表名字段名以及每一个字段的值作为udf的参数,就可以实现写hbase了. 这样大大的节省了开发时间,提升了开发效率. 大家可以按照这种思路写自己需要的功能.这里只简单的列举几个供大家参考,具体操作如下: 一.依赖的jar包 commons-codec-1.7.jar commons-collections-3.2

Hive中基本语法

1.Hive中数据库基本操作1.1 Create/Drop/Alter/Use Database1.2 create table2. Hive中表的操作案例1.3 LOAD DATA1.4 导出Hive表数据3.Hive中的查询语句4. 数据导入导出5.Order, Sort, Cluster, and Distribute By6.Hive UDF编程Hive相关文章 1.Hive中数据库基本操作 1.1 Create/Drop/Alter/Use Database ```CREATE (DA

hive中order by,sort by, distribute by, cluster by作用以及用法

1. order by Hive中的order by跟传统的sql语言中的order by作用是一样的,会对查询的结果做一次全局排序,所以说,只有hive的sql中制定了order by所有的数据都会到同一个reducer进行处理(不管有多少map,也不管文件有多少的block只会启动一个reducer).但是对于大量数据这将会消耗很长的时间去执行. 这里跟传统的sql还有一点区别:如果指定了hive.mapred.mode=strict(默认值是nonstrict),这时就必须指定limit来

kettle连接Hive中数据导入导出（6）

1.hive往外写数据 http://wiki.pentaho.com/display/BAD/Extracting+Data+from+Hive+to+Load+an+RDBMS 连接hive 表输入 1)往excel中写数据 2)往文本文件中写数据 注:这里需要填上hive的表名.字段名,不然会报如下错误: 2016/07/27 15:43:01 - 表输入.0 - ERROR (version 3.2.0, build 1 from 2016-07-07 10.46.10 by xnren

hive中partition如何使用

1.背景 1.在Hive Select查询中一般会扫描整个表内容,会消耗很多时间做没必要的工作.有时候只需要扫描表中关心的一部分数据,因此建表时引入了partition概念. 2.分区表指的是在创建表时指定的partition的分区空间. 3.如果需要创建有分区的表,需要在create表的时候调用可选参数partitioned by,详见表创建的语法结构. 2.细节 1.一个表可以拥有一个或者多个分区,每个分区以文件夹的形式单独存在表文件夹的目录下. show partitions stage_

【转】Predicate和Consumer接口– Java 8中java.util.function包下的接口

原文链接 http://ifeve.com/predicate-and-consumer-interface-in-java-util-function-package-in-java-8/ 原文链接 作者:   Mohamed Sanaulla  译者: 李璟([email protected]) 早先我写了一篇<函数式接口>,探讨了部分Java 8中函数式接口的用法.我也提及了Predicate接口属于java.util.function包, 在这篇文章中,我将展示如何应用Predicat

使用Sqoop，最终导入到hive中的数据和原数据库中数据不一致解决办法

Sqoop是一款开源的工具,主要用于在Hadoop(Hive)与传统的数据库(mysql.postgresql...)间进行数据的传递,可以将一个关系型数据库(例如 : MySQL ,Oracle ,Postgres等)中的数据导进到Hadoop的HDFS中,也可以将HDFS的数据导进到关系型数据库中. 1.问题背景 使用Sqoop把oracle数据库中的一张表,这里假定为student,当中的数据导入到hdfs中,然后再创建hive的external表,location到刚才保存到hdfs中数