Greenplum query Oracle via DLINK

在系统日常生产过程中,如何从Oracle数据库同步一些配置表到GP库个问题。

先来说说之前使用的方式:

0. 使用第3方工具从Oracle导出纯文本格式,存放在本地;

1. 使用GP外部表加载入库

这种方式优点是对于需要同步大数据量的表的时候性能非常高;缺点是稍显麻烦,最主要的是非实时的,存在着任务调度周期和导出导入时间周期。

对此我的解决办法是简化整个过程并实时查询:

0. 使用GP WEB外部表技术直接查询Oracle数据表

由于是配置表,数据量一般都非常小,几k到几w条的记录采用这种方式几乎是瞬间的消耗。

首先是Oracle的源表:

[[email protected] ~]$ sqlplus gtlions/gtlions

SQL*Plus: Release 10.2.0.5.0 - Production on Thu May 8 15:41:06 2014

Copyright (c) 1982, 2010, Oracle.  All Rights Reserved.

Connected to:

Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production

With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> select * from test;

ID NAME

---------- -----

1 a

2 b

然后在GP中直接查询Oracle的表:

[[email protected] ~]$ psql gtlions

psql (8.2.15)

Type "help" for help.

gtlions=# drop external table if exists sqlplus_test;

DROP EXTERNAL TABLE

gtlions=# create external web table sqlplus_test(id int,name varchar(5)) execute ‘sqlplus -S gtlions/[email protected] <<EOF

gtlions‘# set echo OFF;

gtlions‘# set heading OFF;

gtlions‘# set feedback off;

gtlions‘# set linesize 32767;

gtlions‘# set pagesize 0;

gtlions‘# select id||‘‘,‘‘||name from test ;

gtlions‘# exit;

gtlions‘# EOF‘ on master  format ‘csv‘ (header);

NOTICE:  HEADER means that each one of the data files has a header row.

CREATE EXTERNAL TABLE

gtlions=# select * from sqlplus_test;

id | name

----+------

1 | a

2 | b

(2 rows)

so easy,Bingo!

-EOF

Greenplum query Oracle via DLINK

时间: 2024-10-06 02:51:57

Greenplum query Oracle via DLINK的相关文章

Query oracle space occupation

ALTER DATABASE DATAFILE 'C:\ORACLEXE\APP\ORACLE\ORADATA\XE\SYSTEM.DBF' resize 4096M; SELECT USED.TABLESPACE_NAME, USED.USED_BYTES AS "USED SPACE(IN GB)", FREE.FREE_BYTES AS "FREE SPACE(IN GB)" FROM (SELECT TABLESPACE_NAME,TO_CHAR(SUM(N

Oracle学习基本知识点总结

 以下是我总结的OCP教程的知识点,以备参考之用! 1, What's Oracle Server? ·         It's a database management system that provides an open, comprehensive, integrated approach to information management. ·         Consists of an Oracle instance and an Oracle database 2,What

Oracle 复制表创建的sql语句

http://cache.baiducontent.com/c?m=9d78d513d99e01fc09b3c3690d67c0161343f0652ba1d4020ed08449e3732b42501195ac26520774d5d20c6116df3f4b99fa2235775d2feddd8eca5ddcc88f3579db6167204fc71605d36ea8c817639c37902db8f246fae9b0748eaed0d0dd5655d7500e7a85fb894d006390

ORACLE ERP相关整理汇总

摘自:http://cache.baiducontent.com/c?m=9f65cb4a8c8507ed4fece763104687270e54f7633f8883492c81c40884642c101a39feed7c63525483846b6777ac4f5aeefb3765377523a09bbfd20c82e5866d72c8713b2d5cd5104c840eafba11658337902bb6e947f0bb802594dac5d2af0b008c044521d3ae9c5b700

大数据分析案例

部分数据来源于网络,如有侵权请告知. 一.大数据分析在商业上的应用 1.体育赛事预测 世界杯期间,谷歌.百度.微软和高盛等公司都推出了比赛结果预测平台.百度预测结果最为亮眼,预测全程64场比赛,准确率为67%,进入淘汰赛后准确率为94%.现在互联网公司取代章鱼保罗试水赛事预测也意味着未来的体育赛事会被大数据预测所掌控. “在百度对世界杯的预测中,我们一共考虑了团队实力.主场优势.最近表现.世界杯整体表现和博彩公司的赔率等五个因素,这些数据的来源基本都是互联网,随后我们再利用一个由搜索专家设计的机

Cloudera Hadoop 5&amp; Hadoop高阶管理及调优课程(CDH5,Hadoop2.0,HA,安全,管理,调优)

1.课程环境 本课程涉及的技术产品及相关版本: 技术 版本 Linux CentOS 6.5 Java 1.7 Hadoop2.0 2.6.0 Hadoop1.0 1.2.1 Zookeeper 3.4.6 CDH Hadoop 5.3.0 Vmware 10 Hive 0.13.1 HBase 0.98.6 Impala 2.1.0 Oozie 4.0.0 Hue 3.7.0 2.内容简介 本教程针对有一定Hadoop基础的学员,深入讲解如下方面的内容: 1.Hadoop2.0高阶运维,包括H

OCP学习基本知识点总结

 下面是我总结的OCP教程的知识点.以备參考之用. 1, What's Oracle Server? ·         It's a database management system that provides an open, comprehensive, integrated approach to information management. ·         Consists of an Oracle instance and an Oracle database 2,What

Enhancing the Application: Advanced JDBC Features(转)

Enhancing the Application: Advanced JDBC Features This chapter describes additional functionality that you can use in your Java application. Some of these features have not been implemented in the sample application, while some features are enhanceme

Oracle_052_lesson_p10

Managing Undo Data 管理UNDO数据 you should be able to:1.Explain DML and undo data generation2.Monitor and administer undo data3.Describe the difference between undo data and redo data4.Configure undo retention5.Guarantee undo retention6.Use the Undo Advi