Overview BW Delta logic for data extraction
Author: D033258
IMS Financials, SAP AG
BW Architecture
Why BW?
BW as an OLAP system improves the reporting functionality.
- performance
- efficiency
- flexibility
- multiple system landscape
Main Extraction Methods
- Full Update
- DeltaInit Update (Initialization of Delta Process)
- Delta
Delta Logic
Delta extraction loads only new and changed data into BW from the point of the last extraction.
Data transferred already into BW remains and does not have to be loaded again.
Advantage: Performance Improvement by decreasing the extraction time and data volume --compared to the FULL Update logic
Disadvantage: very complex logic necessary in OLTP to extract the data and determine which data has been changed.
Delta enabled DataSources in FI-AA
0ASSET_ATTR_TEXT | Texts and attributes for asset master data (ANLA, ANLZ, ANLU) |
0ASSET_AFAB_ATTR | Asset depreciation parameters (ANLB) |
0FI_AA_11 | Transactions and annual values (ANLC, ANEK, ANEP, ANEA) |
0FI_AA_12 | Period depreciation planned and simulated (ANLP) This extractor has no dedicated fields for legacy values in ANLC, instead it just uses ANLC data to calculate the takeover values and then create some kind of a "virtual" ANLP from it. So actually 0FI_AA_12 does not really load ANLC. ANLC is just part of the calculation, which is necessary because the information of the legacy values does not directly exist in any database table. |
Asset Master Data
- Asset Subnumber (Flexible Update)
- InfoObject 0ASSET
- InfoSource 0ASSET_ATTR_TEXT
- Asset Depreciation Area
- InfoObject 0ASSET_AFAB
- InfoSource 0ASSET_AFAB_ATTR
Asset Master Data : Data sources and targets (today)
Asset values
- Transactions (Delta)
- InfoProvider 0FIA_DS11
- InfoSource 0FI_AA_11
- Annual Values (Delta)
- InfoProvider 0FIA_DS12
- InfoSource 0FIA_DS11 (DataMart Scenario)
- Period Values (Delta)
- InfoProvider 0FIA_DS13
- InfoSource 0FI_AA_12 and InfoSource 0FIA_DS11 (DataMart Scenario)
Asset values: Data sources and targets (before)
Asset values: Data sources and targets (today)
SAP BW 3.2 Content: New Data Model
DataSource definition (RSA2)
The definition of DataSources can be checked in transaction RSA2. Be aware that this transaction is only available in SAP owned systems. In customer systems, this transaction is not available!
Change pointers I
As there is no reliable timestamp information in most of the Asset tables, we had to implement change pointer tables which protocol changes to asset and asset values.
- BWFIAA_AEDAT_AS -> Asset Master record changes
- BWFIAA_AEDAT_AB -> Asset depreciation parameter changes
- BWFIAA_AEDAT_TR -> all changes to asset values, such as transactions, depreciation changes, depreciation postings
Change pointers II
Change pointers are stored by Business Add-In (BADI) FIAA_BW_DELTA_UPDATE. This BADI is called from every voucher function in FI-AA and contains 3 methods:
- UPDATE_POSTING
- UPDATE_MASTERDATA
- UPDATE_DEPRECIATIONS
FIAA_BW_DELTA_UPDATE is delivered inactive. So originally no change pointers are stored in change tables. If the customer wants to use FI-AA Business Content, he has to activate this BADI. First when this is done, change pointers are created for the Delta Extractors.
Important: Leading Source FI-AA (0ASSET_ATTR_TEXT) checks on every Delta or DeltaInit extraction if the BADI is active. If it is not acitve, extraction is stopped and errors are raised.
Example change pointers (BWFIAA_AEDAT_TR)
Sequence for data requests
The aggregation of line items according to master data attributes (e.g. cost center) during the update of Info Cubes from the ODS objects makes it necessary that master data is available in BW before transactions are loaded. Out of this necessity, the following load sequence must be executed:
- 0FI_GL_4
General Ledger: Line items (only if used) - 0ASSET_ATTR_TEXT
Asset master data - 0ASSET_AFAB
Asset depreciation parameters
And then the transaction data
- 0FI_AA_11
Transactions and annual values 0FI_AA_12
Posted depreciation (period values)The correct load sequence is checked by the extraction routines!
Extraction Routines
Central Routine for extraction is Function Module BWFIAA_EXTRACT_DELTA
DataSource Definition in RSA2
Extraction Phases
The extraction can be split at least into 3 phases. Extractor is called at least three times from the BW API.
- INITIALIZATION PHASE
- EXTRACTION PHASE
- FINISHING PHASE
Initialization Phase
Prerequisite: Importing Parameter I_INITFLAG is not initial.
Checks:
- Get selection data, build and convert internal selection tables
- Get timestamps of the leading source (0FI_GL_4 or when this is not used 0ASSET_ATTR_TEXT)
- Get timestamp of the extraction source
- Build Selection Ranges for the extractor
Extraction Phase
Prerequisite: I_INITFLAG is initial.
Depending on extraction source call single extractors
- BWFIAA_GET_MASTER for 0ASSET_ATTR_TEXT and 0ASSET_AFAB_ATTR
- BWFIAA_GET_TRANS for 0FI_AA_11
- BWFIAA_GET_DEPR for 0FI_AA_12
When all data is extracted set NO_MORE_DATA_FLAG -> the next call from API will be the FINISHING PHASE then
Finishing Phase
Prerequisite: I_INITFLAG is intial AND
NO_MORE_DATA_FLAG is NOT initial
Two steps are remaining
- Update Timestamps for extraction source
- Delete "old" timestamps to keep extraction pointer tables as small as possible
Extraction Phase 0ASSET_ATTR_TEXT and 0ASSET_AFAB_ATTR
- Done in Function BWFIAA_GET_MASTER
- IF Delta Logic (DeltaInit, Delta) is chosen, active BADI Implementation for BADI FIAA_BW_DELTA_UPDATE is checked. If BADI is inactive, extraction is stopped!
- DeltaInit Extraction calls old extraction module
ARBW_BIW_GET_MASTER_DATA
for both extractors. - Delta Extraction 0ASSET_ATTR_TEXT uses Function
BWFIAA_GET_ASSET_DELTA.
Delta timestamps are read from BWFIAA_AEDAT_AS. - Delta Extraction 0ASSET_AFAB_ATTR uses Function
BWFIAA_GET_ASSET_AFAB_DELTA.
Delta timestamps are read from BWFIAA_AEDAT_AB.
Extraction Phase 0FI_AA_11 and 0FI_AA_12
- 0FI_AA_11 with Function BWFIAA_GET_TRANS
- DeltaInit Extraction uses Function BWFIAA_GET_TRANS_INIT
- Delta Extraction uses Function
BWFIAA_GET_TRANS_DELTA.
Delta timestamps are read from BWFIAA_AEDAT_TR.
- 0FI_AA_12 with Function BWFIAA_GET_DEPR
- DeltaInit Extraction uses Function BWFIAA_GET_DEPR_INIT
- Delta Extraction uses Function BWFIAA_GET_DEPR_DELTA.
Delta timestamps are read from table BWFIAA_AEDAT_TR or Schedule Monitor logs.
Extraction Phase 0FI_AA_11 - special logic
Source data of this extractor is ANEK, ANEP, ANEA and ANLC. To extract ANLC data in one extraction with transactions (ANEK, ANEP, ANEA), the ANLC data is split into 2 records and masked as transactions with special transaction types.
One ANLC record is split as follows:
- Cumulative values with transaction type CUM
- Planned annual values with transaction type PLN
ANLC values are converted to extract structure in Function BWFIAA_FILL_ANLC_TO_TRAN.
ANEK, ANEP and ANEA data is converted to extract structure BWFIAA_FILL_ANEPV_TO_TRAN.
Extraction Phase 0FI_AA_12 - special logic I
Extraction is split in 2 phases.
- Phase 1: posted depreciation
-> ANLP is read and extracted for all posted periods
- Phase 2: simulated depreciation
-> All periods not posted yet in all open fiscal years are simulated and extracted.
-> Simulated depreciation is marked by field SIM in extract structure.
If simulated periods are posted then and extracted to BW, the simulated depreciation records are overwritten with the posted depreciation in ODS. So step by step the annual period depreciation is extracted to BW.
Extraction Phase 0FI_AA_12 - special logic II
Delta Extraction uses 2 different methods to determine DELTA data, depending on the processed phase.
- Phase 1: posted depreciation
-> Schedule Monitor monitors all executions for a productive depreciation run. To get the periods posted from the last extraction, Schedule Monitor tables are read. With this method, periods can be determined
-> ANLP for these periods is read and passed to BW as real depreciation - Phase 2: simulated depreciation
-> assets changed since the last extraction determined by change pointer table BWFIAA_AEDAT_TR
Extraction Phase 0FI_AA_12 - Depreciation Simulation
Simulation Methods set up in table BWOM_SETTINGS, parameter BWFIAASIM
- NO simulation (BWFIAASIM = 2)
- FAST simulation (simplified RAPOST00-method,
BWFIAASIM = 0 ) - EXACT simulation (exact RAPOST2000-method,
BWFIAASIM = 1 )
Further information in note 614804.
How can I test the extraction in R/3?
Transaction RSA3 - Extractor Checker:
Tips and Tricks I
1) Restrict data selection 0FI_AA_11 and 0FI_AA_12 to limited range of assets:
- Breakpoint in Function Module ANLC_READ_RANGE_TRIGGER
- Insert selection criteria in selection table t_dopt
Sample content of T_DOPT:
- ( ANLN1 = '000000003370' )
- ( ANLN1 = '000300000000' )
- ( ANLN1 = '000200565799' AND
ANLN2 = '0002' )
- ( ANLN1 GE '000000003021' )
AND ( ANLN1 LE '000000003024' )
Tips and Tricks II
2) Timestamp conversion
- Select extraction in table BWOM2_TIMEST
- Fill Function Module RKE_TIMESTAMP_CONVERT_OUTPUT with parameters from BWOM2_TIMEST
Data exploration in BW
Data exploration in BW - step by step
How do I start the extraction from BW?
Transaction RSA1 - Administrator Workbench:
InfoPackage
Request Monitor
Status of extraction requests - Error I
Status of extraction requests - Error II
Status of extraction requests - Success
Content of InfoProviders - technical
Content of InfoProviders - in BEx Analyzer
6 Comments
Unknown User (imwo95f)
Hi Daniella,
Excellent tutorial, but I have a question regarding your statement:
Be aware that this transaction is only available in SAP owned systems. In customer systems, this transaction is not available!
How can this be implemented on customer site or tested in an non-SAP owned IDES system?
Nathan Genez
RSA2 is available in all systems but it's what SAP Developers use to initially create and define the datasource. It's useful in this context because it shows you the specific FM and delta types used by the datasource. For a customer's SAP system, it's informative only... similar to just looking at ROOSOURCE.
vinod bokade
Nicly documented.
Thanks for sharing.
Regards,
VB
Kishore Kamarthapu
Good one
Former Member
Excellent (technical) overview!
Rainer Winkler
I am irritated by the chapter "Sequence for data requests" regarding the order of the loads.
The standard documentation for 7.50 (http://help.sap.com/saphelp_nw75/helpdata/en/fe/55c554ed0fb209e10000000a423f68/content.htm) writes: "The following sequence should be kept during extraction:", this document says "must" which is stronger and to my understanding not justified by the official documentation.
The order is explained in this document by reading of master data during SAP BW transformations. Whereas no explanation is given in the SAP documentation.
The SAP documentation links 0FI_AA_11 and 0FI_AA_12 with "and". I understand this, that both loads can be done in parallel. Wheras this document makes the impression that 0FI_AA_11 has to be loaded before 0FI_AA_12.
I fear that people therefore start to be carefull and load everything strictly one after another. But this causes long runtimes which may be problematic during certain projects.
But I am also not sure whether I understand the SAP documentation correctly as the formulations there are not as explicit as I would like to have it.