Registration

Dear SAP Community Member,
In order to fully benefit from what the SAP Community has to offer, please register at:
http://scn.sap.com
Thank you,
The SAP Community team.
Skip to end of metadata
Go to start of metadata

Problem

High CPU usage on one or more blades is observed and indexing or query performance is negatively impacted. Generally a high CPU utilization is not a problem. This can occur during heavy indexing or query execution activities. It is only critical when the high CPU usage on specific blade results in a “hang” situation or is associated with symptoms like high memory consumption.

Solution

Content of SAP note 1871088

1. Verify which process utilizes most CPU

Run the “top” command to get a list of processes utilizing the CPU. It may not necessarily be a BWA service which is the top CPU consumer.

If the BWA service is consuming high CPU, eg. TREXIndexServer or TREXNameServer, note down the PID of the process. Then go to Services -> Services tab, find the corresponding PID from the top command and confirm the blade name.

2. Check the CPU usage from TREXAdmin Tool ->  Services  -> Load

For the specific blade, get the timeframe when CPU usage was high and check the relevant IndexServer trace files.



You can backup the load history by exporting it.

3. Get a list of running threads

Go to the TREXAdmin tool, Services->Threads, get the screenshots of currently running threads. Select “Threads type”=<all>. This displays the activities the CPU is currently busy with.


Sort threads by “Duration” and check the longest running threads. Right click “Export table as CSV…” to save it to a file.



By checking the running threads, it is possible to see if the CPU becomes exhausted by an indexing job or query executions.

High CPU usage during indexing jobs
  1. Please verify whether too many indexing-jobs are running in parallel
  2. Please check the following parameters in TREXIndexServer.ini:

[indexing]
tmpx_threads
[sparse_check]
prepare_renumber_threads
set_sparse_threads

Check whether these parameters are set to too high. These parameters should be set to the number of CPU cores, which you can get from
Reorg -> Options -> Host Capacity (BWA 700) or Hosts -> Capacity (BWA 720).

 

High CPU usage when queries are executed
  1. Check these parameters in TREXIndexServer.ini:    
    [thread_limit]    
    threads_aggregate    
    threads_bw_merge_reftables    
    threads_aggregate_calls 

    These parameters should be set to number of CPU cores. 
    And parameters in TREXExecutor.ini:     
    [global]     
    use_old_trace_format = off   
    stop_net_fail = 1      
    [limits]     
    max_pop_threads = <number of cpu cores>     
    max_comm_threads = <2 times <number of cpu cores>>    

    Reduce the parameter values if they are set too high.
  2. Check the alerts in RSDDBIAMON2 and resolve them.

    Relevant alerts would be:     
    bia_indexes_size_perf_check: Index already bigger than the split threshold or incorrectly split.     
    check_too_many_parts: Index is not split properly across the blades.

    These will result in BWA performing more joins during query execution, leading to non-optimal memory and CPU usage.

  3. Check whether the issue is caused by non-evenly distributed indexes.
    Go to TREXAdmin Tool -> Reorg -> Usage by Service (I) to check the distribution of the indexes. Unevenly split indexes may cause high memory/CPU utilization on the blade which holds a large part of the index when index joins are performed.

    For more information on splitting, please refer to:     
    http://wiki.sdn.sap.com/wiki/display/TechTSG/Partitioning+a+fact+index

  4. If the high CPU scenario can be reproduced by running a specific query, it is useful to record a python trace to check the calls from BW. To activate the python trace, please refer to SAP Note 1533891. Alternatively you can follow the steps on the following WIKI-page: https://wiki.scn.sap.com/wiki/x/DbooHQ (→ point 3.)

  5. If the query that is causing the problem is not yet identified, please enable the statistics index to capture the query name during the timeframe when CPU usage was high. For more information regarding statistics index, please refer to SAP Note 1756353.

 

  • No labels