OGG "Loading data from file to Replicat"table静态数据同步配置过程

OGG "Loading data from file to Replicat"table静态数据同步配置过程

一个、mgr过程

GGSCI (lei1) 3> view params mgr

port 7809

二、抽取进程extftor

GGSCI (lei1) 4> view params extftor

SOURCEISTABLE

userid goldengate, password yyyyy
rmthost 192.168.100.189, mgrport 7809

RMTFILE /u02/ggs/dirdat/pp, MAXFILES 1000, MEGABYTES 100

table test.xxx;

三、复制进程repftor

GGSCI (lei1) 5> view params repftor

SPECIALRUN
END RUNTIME
userid goldengate, password yyyy
exttrail /u02/ggs/dirdat/pp
ASSUMETARGETDEFS

map test.xxx target testhvr.xxxx;

请注意：

上面的參数是exttrail（不是extfile），exttrail參数是与抽取进程的MAXFILES和MEGABYTES配合使用的，通俗的说，exttrail是相应的/u02/ggs/dirdat/pp000000。/u02/ggs/dirdat/pp000001，/u02/ggs/dirdat/pp000002。/u02/ggs/dirdat/pp000003等等。

而extfile 參数，相应的是/u02/ggs/dirdat/pp

四、从操作系统命令行发起extract（注意：此种extract，ggsci的info all是看不到的）

[[email protected] ggs]$ cd /u02/ggs
[[email protected] ggs]$ pwd
/u02/ggs
[[email protected] ggs]$ extract paramfile /u02/ggs/dirprm/extftor.prm reportfile /u02/ggs/dirrpt/EXTFTOR.rpt
---->请务必确保发起上面extract的securecrt不中断！

建议在vncserver或者以nohup方式运行此extract。

nohup方式运行请參考：http://blog.csdn.net/msdnchina/article/details/46333375

此时，extract会在/u02/ggs/dirdat/下例如以下的文件：

[[email protected] dirdat]$ pwd
/u02/ggs/dirdat
[[email protected] dirdat]$ ls -lrt pp* | wc -l
424
[[email protected] dirdat]$

等待extract完毕（能够从/u02/ggs/dirrpt/EXTFTOR.rpt中推断）：

---------------------------------------以上省略-------------------------------------------
2015-06-02 01:35:10  INFO    OGG-01026  Rolling over remote file /u02/ggs/dirdat/pp000422.

2015-06-02 01:35:22  INFO    OGG-01026  Rolling over remote file /u02/ggs/dirdat/pp000423.

***********************************************************************
*                   ** Run Time Statistics **                         *
***********************************************************************

Report at 2015-06-02 01:35:32 (activity since 2015-06-02 00:02:59)

Output to /u02/ggs/dirdat/pp:

From Table TEST.XXXX:
       #                   inserts:      3005    --------->抽取了3005条记录
       #                   updates:         0
       #                   deletes:         0
       #                  discards:         0

REDO Log Statistics
  Bytes parsed                    0
  Bytes output          43269697047
--->此处是该rpt文件的终点。

五、从操作系统命令行发起repftor（注意：此种repftor，ggsci的info all是看不到的）

[[email protected] ggs]$ cd /u02/ggs
[[email protected] ggs]$ pwd
/u02/ggs
[[email protected] ggs]$replicat paramfile /u02/ggs/dirprm/repftor.prm reportfile /u02/ggs/dirrpt/REPFTOR.rpt
---->请务必确保发起上面replicat的securecrt不中断！建议在vncserver或者以nohup方式运行此replicat。
nohup方式运行请參考：http://blog.csdn.net/msdnchina/article/details/46333375

等待replicat完毕（能够从/u02/ggs/dirrpt/REPFTOR.rpt中推断）：

---------------------------------------以上省略-------------------------------------------
Switching to next trail file /u02/ggs/dirdat/pp000422 at 2015-06-02 04:48:11 due to EOF, with current RBA 98987715
Opened trail file /u02/ggs/dirdat/pp000422 at 2015-06-02 04:48:11

Switching to next trail file /u02/ggs/dirdat/pp000423 at 2015-06-02 04:48:39 due to EOF, with current RBA 97717950
Opened trail file /u02/ggs/dirdat/pp000423 at 2015-06-02 04:48:39

***********************************************************************
*                   ** Run Time Statistics **                         *
***********************************************************************

Last record for the last committed transaction is the following:
___________________________________________________________________
Trail name :  /u02/ggs/dirdat/pp000423
Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)
UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)
RecLength  :    83 (x0053)    IO Time    : 2015-06-02 00:03:04.183247
IOType     :     5  (x05)     OrigNode   :   255  (xff)
TransInd   :     .  (x03)     FormatType :     R  (x52)
SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)
AuditRBA   :          0       AuditPos   : 0
Continued  :     Y  (x01)     RecCount   :     1  (x01)

2015-06-02 00:03:04.183247 Insert             Len    83 RBA 82048951
Name: TEST.XXXX
___________________________________________________________________

Reading /u02/ggs/dirdat/pp000423, current RBA 96448185, 3005 records

Report at 2015-06-02 04:48:52 (activity since 2015-06-02 03:03:36)

From Table TEST.XXXX to TESTHVR.XXXX:
       #                   inserts:      3005 --------->插入了3005条记录
       #                   updates:         0
       #                   deletes:         0
       #                  discards:         0

Last log location read:
     FILE:      /u02/ggs/dirdat/pp000423
     SEQNO:     423
     RBA:       96448185
     TIMESTAMP: 2015-06-02 00:03:04.183247
     EOF:       NO
     READERR:   400

CACHE OBJECT MANAGER statistics

CACHE MANAGER VM USAGE
vm current     =  23.82M   vm anon queues =  23.82M
vm anon in use =      0    vm file        =      0
vm used max    =  23.82M   ==> CACHE BALANCED

CACHE CONFIGURATION
cache size       = 512M   cache force paging = 881M
buffer min       =  64K   buffer highwater   =   4M
pageout eligible size =   4M

================================================================================
RUNTIME STATS FOR SUPERPOOL

CACHE Transaction Stats
trans active   =      0    max concurrent =      3
non-zero total =   8.81K   trans total    =   8.81K

CACHE File Caching
disk current   =      0    disk total  =      0
disk caching   =      0    file cached =      0
file retrieves =      0 

CACHE MANAGEMENT
buffer links  =  52.83K  anon gets   =      0
forced unmaps =      0   cnnbl try   =      0
cached out    =      0   force out   =      0 

Allocation Request Distribution
< 128B:       0
 128B:        0         0     | 512B:     2.94K     5.87K
   2K:    19.79M        0     |   8K:        0         0
  32K:        0         0     | 128K:        0         0
 512K:        0         0     |   2M:        0         0
   8M:        0         0     |  32M:        0         0
 128M:        0         0     | 512M:        0         0
   2G:        0         0     |   8G:        0 

Cached Transaction Size Distribution
    0:        0
 < 4K:        0
   4K:        0         0     |  16K:        0         0
  64K:        0         0     | 256K:        0         0
   1M:        0         0     |   4M:     8.81K        0
  16M:        0         0     |  64M:        0         0
 256M:        0         0     |   1G:        0         0
   4G:        0         0     |  16G:        0         0
  64G:        0         0     | 256G:        0         0
   1T:        0         0     |   4T:        0         0
  16T:        0         0     |  64T:        0         0
 256T:        0         0     |1024T:        0         0 

================================================================================
CUMULATIVE STATS FOR SUPERPOOL

CACHE Transaction Stats
trans active   =      0    max concurrent =      0
non-zero total =      0    trans total    =      0 

CACHE File Caching
disk current   =      0    disk total  =      0
disk caching   =      0    file cached =      0
file retrieves =      0 

CACHE MANAGEMENT
buffer links  =      0   anon gets   =      0
forced unmaps =      0   cnnbl try   =      0
cached out    =      0   force out   =      0 

Allocation Request Distribution
< 128B:       0
 128B:        0         0     | 512B:        0         0
   2K:        0         0     |   8K:        0         0
  32K:        0         0     | 128K:        0         0
 512K:        0         0     |   2M:        0         0
   8M:        0         0     |  32M:        0         0
 128M:        0         0     | 512M:        0         0
   2G:        0         0     |   8G:        0 

Cached Transaction Size Distribution
    0:        0
 < 4K:        0
   4K:        0         0     |  16K:        0         0
  64K:        0         0     | 256K:        0         0
   1M:        0         0     |   4M:        0         0
  16M:        0         0     |  64M:        0         0
 256M:        0         0     |   1G:        0         0
   4G:        0         0     |  16G:        0         0
  64G:        0         0     | 256G:        0         0
   1T:        0         0     |   4T:        0         0
  16T:        0         0     |  64T:        0         0
 256T:        0         0     |1024T:        0         0 

QUEUE Statistics:
num queues    =     15     default index =      0
cur len       =      0     max len       =      0
q vm current  =      0     vm max        =      0
q hits        =  61.61K    q misses      =     21 

queue size  q hits  curlen  maxlen     cannibalized
  0   64K   8.81K      3       3       0
  1  128K   8.81K      3       3       0
  2  256K   8.81K      3       3       0
  3  512K   8.81K      3       3       0
  4    1M   8.81K      3       3       0
  5    2M   8.81K      3       3       0
  6    4M   8.81K      3       3       0
  7    8M      0       0       0       0
  8   16M      0       0       0       0
  9   32M      0       0       0       0
 10   64M      0       0       0       0
 11  128M      0       0       0       0
 12  256M      0       0       0       0
 13  512M      0       0       0       0
 14    1G      0       0       0       0 

================================================================================
RUNTIME STATS FOR CACHE POOL #0
POOL INFO   group: repftor  id: p22586_BLOB
trans active  =       0   trans concurrent (max) =     3
trans total   =    8.81K  (9015 )
flag          = 0x00000030
last error    = (0=<none>)

Allocation Request Distribution
< 128B:       0
 128B:        0         0     | 512B:     2.94K     5.87K
   2K:    19.79M        0     |   8K:        0         0
  32K:        0         0     | 128K:        0         0
 512K:        0         0     |   2M:        0         0
   8M:        0         0     |  32M:        0         0
 128M:        0         0     | 512M:        0         0
   2G:        0         0     |   8G:        0 

================================================================================
CUMULATIVE STATS FOR CACHE POOL #0
POOL INFO   group: repftor  id: p22586_BLOB
trans active  =       0   trans concurrent (max) =     0
trans total   =       0   (0 )
flag          = 0x00000030
last error    = (0=<none>)

Allocation Request Distribution
< 128B:       0
 128B:        0         0     | 512B:        0         0
   2K:        0         0     |   8K:        0         0
  32K:        0         0     | 128K:        0         0
 512K:        0         0     |   2M:        0         0
   8M:        0         0     |  32M:        0         0
 128M:        0         0     | 512M:        0         0
   2G:        0         0     |   8G:        0 

QUEUE Statistics:
num queues    =     15     default index =      0
cur len       =      0     max len       =      0
q vm current  =      0     vm max        =      0
q hits        =  61.61K    q misses      =     21 

queue size  q hits  curlen  maxlen     cannibalized
  0   64K   8.81K      3       3       0
  1  128K   8.81K      3       3       0
  2  256K   8.81K      3       3       0
  3  512K   8.81K      3       3       0
  4    1M   8.81K      3       3       0
  5    2M   8.81K      3       3       0
  6    4M   8.81K      3       3       0
  7    8M      0       0       0       0
  8   16M      0       0       0       0
  9   32M      0       0       0       0
 10   64M      0       0       0       0
 11  128M      0       0       0       0
 12  256M      0       0       0       0
 13  512M      0       0       0       0
 14    1G      0       0       0       0 

================================================================================
RUNTIME STATS FOR CACHE POOL #0
POOL INFO   group: repftor  id: p22586_BLOB
trans active  =       0   trans concurrent (max) =     3
trans total   =    8.81K  (9015 )
flag          = 0x00000030
last error    = (0=<none>)

Allocation Request Distribution
< 128B:       0
 128B:        0         0     | 512B:     2.94K     5.87K
   2K:    19.79M        0     |   8K:        0         0
  32K:        0         0     | 128K:        0         0
 512K:        0         0     |   2M:        0         0
   8M:        0         0     |  32M:        0         0
 128M:        0         0     | 512M:        0         0
   2G:        0         0     |   8G:        0 

================================================================================
CUMULATIVE STATS FOR CACHE POOL #0
POOL INFO   group: repftor  id: p22586_BLOB
trans active  =       0   trans concurrent (max) =     0
trans total   =       0   (0 )
flag          = 0x00000030
last error    = (0=<none>)

Allocation Request Distribution
< 128B:       0
 128B:        0         0     | 512B:        0         0
   2K:        0         0     |   8K:        0         0
  32K:        0         0     | 128K:        0         0
 512K:        0         0     |   2M:        0         0
   8M:        0         0     |  32M:        0         0
 128M:        0         0     | 512M:        0         0
   2G:        0         0     |   8G:        0
--->此处是该rpt文件结尾。

版权声明：本文博客原创文章，博客，未经同意，不得转载。

时间： 2024-10-13 22:20:57

OGG &quot;Loading data from file to Replicat&quot;table静态数据同步配置过程的相关文章

使用OGG&amp;quot;Loading data from file to Replicat&amp;quot;的方法应该注意的问题：replicat进程是前台进程

使用OGG的 "Loading data from file to Replicat"的方法应该注意的问题:replicat进程是前台进程 因此.最好是在vncserver中调用该replicat进程或者以nohup方式放在后台执行.以下的是使用nohup方式放在后台执行. [[email protected] ~]$ ll rep_backgroud.sh -rwxr-xr-x 1 oracle oinstall 98 Jun 2 03:02 rep_backgroud.sh [[e

使用OGG&quot;Loading data from file to Replicat&quot;的方法应该注意的问题：replicat进程是前台进程

使用OGG的 "Loading data from file to Replicat"的方法应该注意的问题:replicat进程是前台进程 因此,最好是在vncserver中调用该replicat进程或者以nohup方式放在后台运行.下面的是使用nohup方式放在后台运行. [[email protected] ~]$ ll rep_backgroud.sh -rwxr-xr-x 1 oracle oinstall 98 Jun 2 03:02 rep_backgroud.sh [[e

关于 OGG &quot;Loading data from file to Replicat&quot;同步含有lob字段表的部分记录的关键参数

首先说明一点: Loading data with an Oracle GoldenGate direct load 这个方法,对含有如下数据类型的table,是无法使用的: LOBs, LONGs, user-defined types (UDT), or any other large data type that is greater than 4 KB in size. 因此,遇到如下需求时,就不得不用OGG "Loading data from file to Replicat&quo

OGG &quot;Loading data from file to Replicat&quot;table数据的静态同步配置过程

一.mgr进程 GGSCI (lei1) 3> view params mgr port 7809 二.抽取进程extftor GGSCI (lei1) 4> view params extftor SOURCEISTABLE userid goldengate, password yyyyy rmthost 192.168.100.189, mgrport 7809 RMTFILE /u02/ggs/dirdat/pp, MAXFILES 1000, MEGABYTES 100 table

Loading Data into HDFS

How to use a PDI job to move a file into HDFS. Prerequisites In order to follow along with this how-to guide you will need the following: Hadoop Pentaho Data Integration Sample Files The sample data file needed for this guide is: File Name Content we

Cocos2d-x执行时错误：Cocos2d: Get data from file(xxx.xxx) failed!

取资源图片时遇到执行时错误: Cocos2d: Get data from file(xxx/xxx.xxx) failed! 原因是我加入资源目录的方式不正确,例如以下图,我选择的是在Resource中加入Group.这样加入完menuscene目录是黄色的,我在代码中引用资源时路径是"menuscene/background.png",可是在项目看来menuscene并非一个真实存在的Folder,而是一个Group,所以会找不到. 解决方法是加入文件时选择Create folde

Loading Data From Oracle To Hive By ODI 12c

本文描述如何通过ODI将Oracle表数据同步到Hive.1.准备工作在hadoop集群的各个节点分别安装Oracle Big Data Connectors,具体的组件如下图所示:这里只需安装Oracle Loader For Hadoop(oraloader)以及Oracle SQL Connector for Hadoop Distributed File System (oraosch)两个软件.安装比较简单,直接解压即可使用(这里的ODI.oraosch以及oraloader组件都是以

OGG ORA-1403 NO DATA FOUND

近期开始做OGG的项目,测试过程中遇到一个问题.使用CTAS创建一张表,表中有数据,然后使用delete from xxx:删除表之后,replicat进程异常.查看ogg日志,查看官网找到原因. 官方文档号 How to Capture the DML(s) Executed by a DDL Trigger (OGG) (文档 ID 1509900.1) 解决方案 SOLUTION By design, Extract (Classic/ Integrated) doesn't captur

Oracle GoldenGate (以下简称ogg)在异种os上同种db之间的数据同步。

ogg能实现的功能: 可以细化到单表同步,符合特定where条件的rows同步,指定column的同步. ogg 的竞争对手:ibm cdc:dell shareplex:dsg:dds等等 环境描述: 源头:windows 2003x64bit, oracle db 11.2.0.3 x64bit 目的端:suse linux ,oracle db 11.2.0.3 预先准备: 1.Microsoft Visual C ++ 2005 SP1 Redistributable Package 注