Memory issues in APD processing
Many memory issues within the Advanced Process Design area can be avoided with simple changes to the design. In looking at similar issues arising in relation to the performance of the APD and, in a number of cases, memory and use of memory is a cause of APDs failing. This page might be helpful as a quick guide to simple features.
The details below should be helpful in reducing the datasets which are used in the APD processing. This should minimise the amount of memory short dumps being produced and should allow APDs to run better and more efficiently.
Tips for reducing memory consumption
Generally, memory errors and performance issues are caused by large datasets being processed in APD. The APD may also be inefficiently designed which causes the amount of data to be larger than the user expects. Below are some tips which can be used to reduce the amount of data processed and, therefore, reduce memory errors.
- Uncheck "Process Data in Memory":
- Start Transaction RSANWB
- Choose 'Goto' from menu bar
- Click on 'Performance Settings'
- Uncheck the flag 'Process Data in Memory
- Use APD Partitioning
- See 1901962 How Query Partitioning works in APD
- Check if the bXML interface for the MDX interface is already used in your system (release+ support package)
- See SAP note 1284239 RSCRM: Overcoming the 1 million row limit in RSCRM
- This will also overcome the 1 million cell limit in APDs (to be checked with available memory configuration)
- Check if an APD could be replaced by DTP with Query as InfoProvider
- Analyse if there is scope for improving the design of the APD:
- Check intermediate results at each node
- See at which node the APD is failing
- Conditions are ignored in MDX and cause extra overhead in APD, delete conditions and replace by a FILTER node in the APD (see KBA 2109881 APD: Bex Query with active Bex conditions causes memory and performance problem)
- Check if blank records are filtered out at joins (these can cause extra data)
- Reduce the records by using as many filters as possible in the early stages of APD
- Overall APD design:
- Filter data into smaller datasets
- Use multiple runs
- Aggregate all resulting data back into one DSO
- Multiprovider processing:
- Multiprovider works as UNION and join condition and this generates more records than expected
- Check the design of multiprovider/dimension and try to reduce the number of records
1790516 Memory issues in APD processing
1901962 How Query Partitioning works in APD
2109881 APD: Bex Query with active Bex conditions causes memory and performance problem