Collections - PDP and ODS

A) Planning Data Pull
   This program refreshes the snapshots on source instance
   Puts data into staging tables of VCP
   1) Refresh snapshots on source instance    
      Snapshots are materialized views. These views gets refreshed by a program called 'Refresh Collection Snapshots'.This program gets triggered on source instance. It refreshes all the snapshots of those entities that are being collected in the current collections run.

Examples of Snapshots.
      MTL_SYS_ITEMS_SN     -- Snapshot for Items
      MTL_ITEM_CATS_SN     -- Snapshot for Item Categories
      MTL_SUPPLY_SN          -- Snapshot for Onhand Inventory details

Below figure showing few of the MTL Snapshots.

   2) After the 'Refresh Collections Snapshots' program is completed on source,the 'Planning Data Pull' process submits 'Planning Data Pull Workers' to load the data into VCP staging tables. The 'Planning Data Pull Workers' take the data from the refreshed snaphsot views and load the data into VCP staging tables

Examples of staging tables
      MSC_ST_SUPPLIES              -- Staging table for supply details
      MSC_ST_SYSTEM_ITEMS       -- Staging table for items
      MSC_ST_ITEM_CATEGORIES  -- Staging table for item categories

B) Planning ODS Load
After the staging tables are populated by Planning Data Pull and Planning Data Pull Workers,The next program in the collections process is the Planning ODS Load.This program performs validations on the staging tables and submits 'Planning ODS Load Workers' to load the data into VCP base tables.

Examples of VCP base tables
  MSC_SUPPLIES               -- Base table for supply details
  MSC_SYSTEM_ITEMS       -- Base table for items
  MSC_ITEM_CATEGORIES  -- Base table for item cateogries

After the data is loaded into base tables, the program 'Planning Data Collection - Purge Staging Tables' program is submitted.This program purges the data from staging tables.



Collection Method

There are three ways in which we can collect data to VCP - Complete Refresh, Target Refresh, Netchange Refresh
Complete Refresh - In Complete Refresh, all the entities that were previously collected into VCP will be removed and all the entities will be recollected from source.
Target Refresh   - In Target Refresh, only those entities that are being collected as part of the collection run will be removed from VCP and recollected from source.
Netchange Refresh - In NetChange Refresh, only the difference of the data that is on source and not on VCP will get collected from source.