Setup & Administration
With SAP Solution Manager 7.1 only 2 extractor setup types (SOLMAN and RCA) existed. All extractors were scheduled during Basic Configuration or the Managed System Configuration.
With SAP Solution Manager 7.1 we have currently 24 different setup types. The extractors are scheduled during the setup of their different use cases. The setup and the scheduling of the extractors is mostly automated. Though there are some important steps during the Basic Configuration and in Managed System Configuration that influences the settings of the scheduled extractors.
The first setup step that influences the extractor setup is in the step “Specify SAP BW System” in the Basic Configuration. The extractors write data to the BW system in SAP Solution Manager, so the decision which BW configuration you want to use in your Solution Manager, influences this setup. If you change the BW system setup you will have to make sure the extractors are adjusted accordingly.
As explained earlier, the Resource Manager is responsible for scheduling extractors when they are due for running. The resource manager is a batch job which is scheduled on the step “Configure Automatically” in the activity “Schedule Sol. Manager Background Jobs”
The following jobs relevant for the EFWK are scheduled:
- Resource Manager (E2E_EFWK_RESOURCE_MGR) runs every minute
- EFWK Housekeeper (E2E_EFWK_HOUSEKEEPER) runs daily
- BI Housekeeper (E2E_HK_CONTROLLER) runs daily
In the activity “Setup Extractor Framework” the extractors of setup type SOLMAN are scheduled. This includes extractors for CCDB and Change Analysis.
During the managed system setup the extractors for root cause analysis (setup type RCA) are scheduled for the managed systems.
The extractors for the other setup types are scheduled in the actual scenario setup. This is not always done in a separate step or activity, but sometimes in the background during the activation of the functionality.
Extractor Setup Types
The following table contains the currently used setup types for the extractors.
Use Case / Setup Type
Root Cause Analysis (Mainextractor <> E2E*SSR*)
SV-SMG-DIA-WA / SV-SMG-DIA-EA
Statistical Records (Mainextractor = E2E*SSR*)
Root Cause Analysis (SolMan System only)
SV-SMG-DIA-WA / SV-SMG-DIA-EA
End User Experience Monitoring (Technical Scenario)
End User Experience Monitoring (SolMan System only)
Monitoring and Alerting relevant Extractors
Service Provider Reporting
Test Workbench Reporting
PI Message Flow
Business Process Monitoring
IT Management Reporting – BI Self Service
Usage and Landscape Data Exchange
SUGEN Data Exchange
CCDB Technical System Extractor
Data Volume Management
SAP Solution Manager Usage Scenario
Setup for Benchmark Project (Massman Data)
Customer Usage Provision
Transport Request Management
Interface Channel Monitoring
Job Scheduling Management
If you have issues with an extractor and want to know the setup type to choose the correct CSN component, open the EFWK Admin UI via transaction SM_WORKCENTER -> Solution Manager Administration -> Infrastructure -> Framework -> Extractor Framework
Change the settings of the POWL table that contains the extractors, by clicking on “Settings” in the upper right corner of the table.
Make the column “Setup Type” visible
Now you find the information on the extractor’s setup type in the last column of the POWL table.
EFWK Admin UI
The Extractor Framework is administered using the new Administration UI. From here you can display the status of the extractors and start an error analysis or configuration of extractors.
To open the Admin UI call transaction SM_WORKCENTER and switch to the Solution Manager Administration workcenter. Depending if you want to check extractors in general or for a specific technical system you have to choose either the Landscape view (system specific) or the Infrastructure view (general)
The first screen is the status overview screen for the extractor framework. Here you see status information for all registered extractors, except MAI extractors. MAI extractors are administered in the “Alerting Framework”.
Extractors have 2 statuses, the status of activation and the status of the last run.
An Extractor can become inconsistent, e.g. if the technical configuration of the system is changed. Banned means the extractor is not executed anymore, due to previous problems it was causing. An extractor gets banned if he causes 3 timeouts in the EFWK. A timeout can occur if the extractor runs too long and is blocking a resource (i.e. an RFC destination) for too long.
Please note that all data transported in the Extractor Framework is always in UTC time zone. The Required Record algorithm and the Logging are also in UTC. The extractor makes sure the data is delivered in UTC time zone. However, before the collected data is written to the BW, that data is converted into the time zone of Solution Manager.
When you select an extractor in a table below you find the status of the last N runs (per default N would be 20). When you select a run you find details on the selected extractor run in the table below. Which phases an extractor has, depends on the extractor.
In case of an error you will an error message in the column Message.
On the tab “Extractor Detail” you find further details on the extractor:
On the tab “Resource Manager” you find status information for the Resource Manager job.
Exceptions for the extractor framework that have been written to application log, can also be found on the tab “Exception”
Extractor Framework Configuration Options
There are certain settings than can be made to tune the EFWK. DO only change these settings if you know exactly what you are doing or when you are adviced to do so by SAP.
Configuration changes are made via the tab "Configuration" of the EFWK Admin UI.
Changing the Resource Cap for RFC resources
The EFWK is controlled by the Resource Manager and is subject to the so called resource cap. This resource cap makes sure that the Resource Manager doesn’t open too many RFC destinations to a managed system and might overload the managed system. The resource cap applies to all extractors for a system, RCA as well as E2E Monitoring and Alerting and other use cases.
Usually a resource is blocked as long as the extractor is running. Additionally an enqueue lock is set per extractor to make sure it cannot be started several time at once in Solution Manager. There is also a timeout in EFWK, after which a resource is released no matter if the extractor finished or not. If an extractor hits a timeout 3 times, it is banned from further execution.
Sometimes it can be necessary to adjust the resource cap. Per default it is set to 1. But if the managed system is very busy or huge and a lot of data has to be collected and if RCA and E2E Monitoring and Alerting is both active, it can be necessary to adjust the resource cap.
You can set resource caps for each READ RFC destination to a managed system and local RFC destinations. Also the number of ABAP dialog work processes that can be used by the EFWK is subject to the resource cap. To change the resource cap switch to the tab “Configuration” in the EFWK Admin UI and change resource cap for the desired RFC destination.
RFC resources are:
- SOLMAN_DIALOG_WORKPROCESSES: Maximum number of Dialog workprocesses on Solution Manager, which can be used by the Extractor Framework
- SM_<Log Sys>_READ: Used for RFC calls from Solution Manager to the managed systems ABAP stack
- SOLMANDIAG: Used for RFC calls to Java-Stack of Solution Manager via JCO (Introscope)
- WEBADMIN: Used for RFC calls to Java-Stack of Solution Manager via JCO (EEM)
- NONE: Used for local RFC calls
Please note: If you adjust the number of SOLMANDIAG or WEBADMIN resources, be aware that you might also have to adjust the number of JCO listeners on the Java Stack of Solution Manager. This can be done in the basic configuration of SAP Solution Manager.
Changing the Logon-Group for EFWK
You can also specify logon groups for the SAP Solution Manager system. The Resource Manager then only runs extractors in the instances in the logon group. On the tab “Configuration” below the RFC Management you find the Logon Group maintenance for the EFWK. Here you can create and set logon and server groups.
Changing the Number of WLI per EFWK RM Run
In the last section you can change how many work list items (basically the number of extractors) are processed within one EFWK Resource Manager run. You can change this number in case there is a huge backlog of extractors that have to be processed. Changing this value might prolongs the runtime of the EFWK Resource Manager job.
Since each Resource Manager is enqueue protected this will not lead to problems, but the new instance of the RM will not be started. You will find a message like this in the spool output of the RM job instance.
The housekeeping in EFWK is performed by the job “E2E EFWK HOUSEKEEPING” on a daily basis. The job usually runs at 10pm.
The job performs the following activities:
- Landscape consistency check (Technical Systems, PPMS)
- EFWK internal consistency checks
- BI Destination
- SOURCE RFC Destination
- Check against E2E_RESOURCES
- Release banned extractors
- Check for Duplicates and LUW inconsistencies
- Cleanup the tables E2E_EFWK_STATUS and E2E_EFWK_LOG
- Housekeeping for DPC (Data Provider Connector)
- Generate Self-Monitoring Alert
The details for the house keeping job can be found in the application log (SLG1) under the Object E2E_ALERTING and the sub object EFWK.
The EFWK writes data in the BW of Solution Manager. To make sure this data doesn’t grow too much and too fast a BI housekeeping is implemented for the data from the EFWK.
Housekeeping for Standard BW Cubes
The housekeeping is done by the program E2E_HK_CONTROLLER. The program is scheduled in the daily job E2E BI HOUSEKEEPING.
Data in either aggregated and transferred from cubes with e.g. hourly resolution to cubes with daily resolution, or it is deleted, e.g. data from the higher volume cubes (H-Cubes) is deleted if it is older than the default of 91 days. SAP note 1480588 describes the housekeeping process in detail.
The status of the housekeeping can be checked in the application log (transaction SLG1). To find the log information for the housekeeping, enter the value E2E_HK_CONTROLLER in the field External ID.
The spool output of the job E2E BI HOUSEKEEPING contains information on the aggregation and deletions that have taken place.
The settings for the housekeeping are maintained in the tables E2E_BI_AGGREGATE for the aggregation and the table E2E_BI_DELETE for the data deletion. It is not recommended to change these settings!
The default aggregation and deletion settings are:
- Root Cause Analysis - The default values for aggregation are: Hourly data is stored for 90 days, daily data is stored forever. In addition, the daily data is finally aggregated to monthly values after one year for database related metrics.
- Monitoring and Alerting - The default values for aggregation are: Minutely data is stored for 8 days and then aggregated to hourly data. Hourly data is stored for 31 days, daily data is stored for 100 days, weekly data is stored for 52 weeks and monthly data is stored for 47 month. This can be adjusted in the infrastructure configuration for technical monitoring.
Housekeeping for Twin Cubes
To ease the housekeeping process the twin cube concept was introduced with Solution Manager 7.1.
In the twin cube concept the same cube exists two times. In the initial phase only cube 1 filled by the data loader. Once cube 1 is full, cube 2 is filled instead. After a certain time period the housekeeping kicks in and the data from cube 1 is dropped. The data loader changes its target and cube 1 is filled again until in the next housekeeping phase the data from cube 2 is dropped. This way of dropping the whole table instead of searching out single records by their age, speeds up the housekeeping process tremendously.
The following figure shows the data retention periods of the twin cubes: