Spark转GemFire任务（一）

ADMG-40 2.2.2.4 - Global Tax Warehouse (GTW) CoA mapping -- Both BtB and Project 1

Revision 7/15, change all latam to mdg

Create GFS account mapped to GTW using FAGL_011ZC, write to /tax/gfs_gtw_mapping

and go through /btb_mdg/ska1 to check all 10 digit GL account can be mapped, if not, write to exception region /tax/master_data_exception

Use BTB_MDG.SKA1 (if BTB_MDG is not ingested, use BTB_LATAM.SKA1):

1. For every record of SKA1 (select every record from SKA1 where KTOPL = ‘JNJG’ and XSPEB <> ‘X’ )  -- collection A

collectionA:

Path: /btb_mdg/ska1

Filter: KTOPL  =  ‘JNJG’   and  XSPEB  <>  ‘X’

Column: saknr

自己本身

1. select records in BTB_MDG.FAGL_011ZC (where VERSN=’GTW’ AND KTOPL=’JNJG’) – collection B

collectionB:

Path: /btb_mdg/fagl_011zc

Filter: trim(versn) = ‘GTW’ and ktopl = ‘JNJG’

Column:  from  vonkt

To     biskt

自己本身的

1. match each A.SAKNR with [B. VONKT , B. BISKT], B.VONKT is the starting number, B. BISKT is the ending number, both numbers are included in the range,  when a match is found,

Validate wheather each item in collectionA in the range of collectionB

判断是否 A包含于B

4. Get gfsCode (A.SAKNR) and gtwCode (B.ERGSL) description:

for gfsCode -- /btb_mdg/skat: spras:E AND ktopl: JNJG AND saknr:%A.SAKNR%, let it be D, D.txt50 is the gfsCode description.

gtwCode -- /btb_mdg/fagl_011qt : versn:GTW AND spras:E AND ergsl:%gtwCode%, let it be E, E.txt45 is the gtwCode description

gfsCode:

Select  txt50 as gfsCodeDescription from  /btb_mdg/skat where spras = ‘E’ and ktopl = ‘JNJG’ and saknr in collectionA.saknr

和A的saknr有关系

gtwCode:

Select txt45 as gtwCodeDescription from /btb_mdg/fagl_011qt where trim(versn) = ‘GTW’ and spras = ‘E’ and  ergsl in collectionB.ergsl

和B的ergsl有关系

1. save it into /tax/gfs_gtw_mapping: <gfs:A.SAKNR, gfs_description:D.txt50, gtw:B.ERGSL, gtw_description:E.txt45>

Insert into /tax/gfs_gtw_mapping(

Gfs  :   collectionA.saknr,

Gfs_description  :  txt50,

Gtw  :  collectionB.ergsl,

Gtw_Description : txt45

)

Validation:

3.1 If there is no match, write exceptionCode:INVALID_GTW_MAPPING, excepitonMessage: No Match on GTW Code for GFS code % A.SAKNR%

items in CollectionA but not in the range of CollectionB will be written into Exception

exceptionCode :  INVALID_GTW_MAPPING

exceptionMessage :  No Match on GTW Code for GFS code % A.SAKNR%

all  [B. VONKT , B. BISKT] should not have overlap, check all [B. VONKT , B. BISKT] do not overlap by sorting B. VONKT and B. BISKT and make sure B.VONKT(n+1) < B. BISKT (n), ), if there is invalid range, exceptionCode: INVALID_GFS_MAPPING, exceptionMessage  overlapping range is found, [B(n). valfrom, B(n).valto] and [B(n+1). valfrom, B(n+1).valto],

All the ranges in CollectionB cannot have overlap,if it does,written into Exception

exceptionCode: INVALID_GFS_MAPPING

exceptionMessage  overlapping range is found, [B(n). valfrom, B(n).valto] and [B(n+1). valfrom, B(n+1).valto]

6. Saving exception:

Exception should be saved to /tax/master_data_exception

{

sourceSystem: btb_mdg,

exceptionCode:

exceptionMessage:

timestamp

}