Skip to end of metadata
Go to start of metadata

Content

 

SAP Solution Manager 7.1 supports Apache Tomcat with End-to-End Root Cause Analysis and System Monitoring. Following Apache Tomcat products are supported: Apache Tomcat 5.5, Apache Tomcat 6.0, Apache Tomcat 7.0 (since SP8)

 

If the diagnostics templates for a particular Apache Tomcat version is missing in the SAP Solution Manager system, please download and import the template from following SAP notes:

  • SP07: 1783886
  • SP08: 1811700
  • SP09: 1882778
  • SP10: 1940337
  • SP11: 1989380

For more details please also refer to the following SAP notes:

  • SAP note 1508421 - SAP SLD Data Supplier for Apache Tomcat Server
  • SAP note 1438005 - Wily Introscope Agent for Apache Tomcat Server
  • SAP note 1633036 - SAP Solution Manager 7.1 E2E RCA Setup for Apache Tomcat

Overview E2E RCA setup for Apache Tomcat

The following table lists the steps necessary to set up E2E Root Cause Analysis for an Apache Tomcat Server system.

Setup Step

To be performed on

Remarks

Step1: Install SAP Host Agent

Managed system (host)

Once per Tomcat host

Step2:  Install Diagnostics Agent

Managed system (host)

Once per Tomcat host

Step3:  Install Introscope Java Agent* *

Managed system (Tomcat)

Once per Tomcat server

Step4:  Configure Tomcat Server* *

Managed system (Tomcat)

Once per Tomcat server

Step5: Install SLD Data Supplier

Managed system (Tomcat)

Once per Tomcat server

Step6:  Run Managed System Setup

SAP Solution Manager

Once per Tomcat system

Step7: Manual Managed System Setup Steps

SAP Solution Manager

Once per Tomcat system


 

1. Install SAP Host Agent

Current Diagnostics Agent installers install the SAP Host Agent automatically (see next section).

Alternatively, refer to SAP Note 1031096 Installing Package SAPHOSTAGENT.

2. Install Diagnostics Agent

Please refer to SAP Note 1365123 - Installation Strategy for Diagnostics Agents.

Please refer to SAP Note 1833501 - Diagnostics Agent Installer Versions.

3. Install Introscope Java Agent

Please refer to SAP Note 1438005 “Wily Introscope Agent for Apache Tomcat server” for details.

4. Configure Tomcat Server

To setup E2E RCA for Apache Tomcat please perform following two steps on the Tomcat server.

  1. Enable JMX remote connection to the Tomcat server.
  2. Enable Java GC logging.
  3. Enable Tomcat logging.

4.1 JMX remote connection configuration

Some Solution Manager applications require JMX remote connection to the Tomcat Server.  For example, Thread Dump Analyzer application connects to Tomcat Server through JMX to trigger thread dumps.

To configure JMX remote connection to the Tomcat Server set following JVM system properties in the Tomcat’s JVM startup configuration:

com.sun.management.jmxremote
com.sun.management.jmxremote.port = <jmx_port>

Password authentication for remote connection is enabled by default. To disable it, set the following JVM system property when you start the Tomcat’s JVM:

com.sun.management.jmxremote.authenticate=false

To protect Tomcat from unauthorized JMX access through JMX remote connection you should secure the connection with password and access files.  For that add following JVM parameters to the Tomcat’s JVM configuration

com.sun.management.jmxremote.ssl=false
com.sun.management.jmxremote.authenticate=true
com.sun.management.jmxremote.password.file=<path jmxremote.password>
com.sun.management.jmxremote.access.file=<path jmxremote.access>

There <path jmxremote.password> and <path jmxremote.access> are the paths to the password and access control files

Note: SSL is currently not supported.

The password and access files control security for remote monitoring and management. The password file $CATALINA_BASE/conf/jmxremote.password  defines the roles and their passwords. The access control file $CATALINA_BASE/conf/jmxremote.access defines the allowed access for each role.

Edit the jmxremote.access file. Add an appropriate role with the readwrite access in the jmxremote.access file, for example smdAgentRole like below

smdAgentRole readwrite

 

Edit the jmxremote.password file. Specify the password for the role you defined in the jmxremote.access file:

smdAgentRole abcd1234

 


You must specify the role and the password when you perform Managed System Setup for the Tomcat System in the Solution Manager.

Permissions to the password file

Ensure that only the owner has read and write permissions on jmxremote.password file, since it contains the passwords in clear text. For security reasons, the system checks that the file is only readable or writeable by the owner and exits with an error if not. This check may cause problems, for example, if Tomcat runs as Windows service as local user account because it might be quite tricky to restrict the access to the local system account.

Tomcat Service running as Local System account

Tomcat Service running as “user” account

 

The way you set up the password and access files depends on the platform and whether you are in a single-user environment or a multi-user environment.

On Windows for a (local) system account

Perform following steps to restrict the access on the jmxremote.password file to a built-in system account

  1. Right click password file and open Properties
  2. On the Security tab, go to “Advanced” dialog.
  3. In the “Advanced Security Settings” clear the “Include inheritable permissions from this object’s parent” checkbox. Remove all inherited permissions and apply the changes.

    before

    After

  4. On the Security tab, edit the permissions. Remove the permissions for all groups/user names except the SYSTEM account.

    before

    after

  5. Set the file owner to the “SYSTEM” account.

    before

    after

Please contact your Windows system administrator in case of problems. The solution may depend on your environment.

Note

Please note that after applying the steps you won’t be able to edit the file any longer because you user will have no permission on the file. To modify the file you will need temporary assign necessary permissions on the jmxremote.password file.

 

On Windows for a user account

To restrict the access on the jmxremote.password file to the Tomcat’s user account please perform the steps described above accordingly.

  1. Right click password file and open Properties
  2. On the Security tab, go to “Advanced” dialog.
  3. In the “Advanced Security Settings” clear the “Include inheritable permissions from this object’s parent” checkbox. Remove all inherited permissions and apply the changes.
  4. On the Security tab, edit the permissions. Remove the permissions for all groups/user names except the user account that Tomcat runs on.

On Linux:

Use the cmd chmod 600 to limit the access of file jmxremote.password

Further References

Please consult Apache Tomcat documentation for more details. E.g. for “The Apache Tomcat 6.0” ->”Monitoring and Managing Tomcat” ->”Enable JMX Remote” under following link http://tomcat.apache.org/tomcat-6.0-doc/monitoring.html#Enabling_JMX_Remote

Please refer also to Java documentation for JMX, e.g.  the “Monitoring and Management Using JMX” chapter in Java 1.5 documentation on http://download.oracle.com/javase/1.5.0/docs/guide/management/agent.html.

The chapter “How to Secure a Password File on Microsoft Windows Systems” on http://download.oracle.com/javase/6/docs/technotes/guides/management/security-windows.html

Examples setting up JMX parameters for Apache Tomcat server

a. If tomcat started via script file (e.g. on Linux), copy and paste following JXM system parameters to the Tomcat staring script (e.g. catalina.bat/catalina.sh), change the values if needed, especially for file's location.

set JAVA_OPTIONS=%JAVA_OPTIONS% -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9004 -Dcom.sun.management.jmxremote.authenticate=true -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.password.file=C:/APACHE/Tomcat/Tomcat6.0.33/conf/jmxremote.password -Dcom.sun.management.jmxremote.access.file=C:/APACHE/Tomcat/Tomcat6.0.33/conf/jmxremote.access

b. If tomcat started as windows service, open the “Configure Tomcat” application add the required JVM system properties on the tab “Java” into the field “Java Options”, change the values if needed, especially for file's location.

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9004
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=C:/APACHE/Tomcat/Tomcat6.0.33/conf/jmxremote.password
-Dcom.sun.management.jmxremote.access.file=C:/APACHE/Tomcat/Tomcat6.0.33/conf/jmxremote.access

 

Enable JMX collection in wily Agent

  1. Open the agent’s profile <AGENT_HOME>/IntroscopeAgent_tomcat.profile. Mase sure that the property jmx.enable is set to true (shall be enabled by default)
    introscope.agent.jmx.enable=true
  2. Copy the WebAppSupport.jar from the <Agent_Home>/wily root directory into the <AGENT_HOME>/wily/ext directory.

4.2 JVM GC log file configuration

Solution Manager may collect and analyze JVM GC logs for Apache Tomcat. However, by default no GC log is enabled on the Tomcat server.

To enable GC logging please set the following JVM system property when you start Tomcat’s JVM:

-Xloggc:<path to GC.log> - Enables logging for the JVM GC processes into the log file specified by the <path to GC.log> parameter. Please use an absolute path here.

-XX:+PrintGCDetails  - Gives details about the GCs, such as: Size of the young and old generation before and after GCs; Size of total heap Time it takes for a GC to happen in young and old generation; Size of objects promoted at every GC

-XX:+PrintGCTimeStamps - Prints the times at which the GCs happen relative to the start of the application.

Examples setting up GC parameters for Apache Tomcat server

a. If tomcat started via script file (e.g. on Linux) add following JXM system parameters to the Tomcat staring script (e.g. catalina.bat/catalina.sh):

set CATALINA_OPTS=%CATALINA_OPTS%  -Xloggc:C:/APACHE/Tomcat/Tomcat6.0.33/gcTomcat.gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

b. If tomcat started as windows service, open the “Configure Tomcat” application add the required JVM system properties on the tab “Java” into the field “Java Options”

4.3 Tomcat logging configuration

Please make sure, Tomcat logging is configured properly. Check whether the JVM system properties “java.util.logging.manager” and “java.util.logging.config.file” are set. For example,

-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager

-Djava.util.logging.config.file=C:\APACHE\Tomcat\Tomcat6.0.33\conf\logging.properties

5. Install and run SLD Data Supplier

SAP Solution Manager depends on the technical system definition in the Landscape Management (LMDB). The SLD-DS documentation describes how to install and run the SLD-DS for Apache Tomcat server. Please refer to SAP Note 1508421 SAP SLD Data Supplier for Apache Tomcat for details.

Please consult Solution Manager Documentation for general topics like for the steps needed to create or maintain your technical systems inside the Landscape Management of SAP Solution Manager.

Note:

Consider to assign an appropriate System ID to the Tomcat system describing your Tomcat server in a meaningful way. Use the SYSTEM_ID configuration property of the SLD-DS for that.  For example,

                SYSTEM_ID = ATMFR2

Otherwise the System ID identifier will be automatically assigned by the Solution Manager.

6. Run Managed System Configuration

Managed System Configuration makes your Tomcat system known to the SAP Solution Manager

The Managed System Configuration activity is performed in the SAP Solution Manager Configuration Work Center.

Start the “Managed System Setup” wizard for the Tomcat server system.

  1. Start the Open the Configuration Work Center -> Managed System Configuration -> Technical Systems.
  2. Select the Tomcat system in the list. It is possible to filter on the system type (Apache Tomcat Server) or on the System ID (ATTMFR2) or look for the entry by the Tomcat port/host name.
  3. Click on “Configure System. “Managed System Setup” wizard will be opened in a separate window.

Note:

The “Technical Systems” lists all technical systems known by the Landscape Management (LMDB) in the Solution Manager. If the Tomcat system cannot be found in the list, please check whether the SLD Data Supplier is installed and runs properly.

 

Perform “Managed System Configuration” wizard

The “Managed System Configuration” wizard consists of several steps you need to perform to complete Managed System Configuration for the Tomcat system.

  1. Select Product – to identify the Apache Tomcat product as “Diagnostics Relevant”
  2. Assign Diagnostics Agent – to connect the Diagnostics Agent on the Tomcat host to the Solution Manager.
  3. Enter System Parameters – to specify parameters like  what are the JMX port/user/password or which Introscope Enterprise Manager to be used
  4. Enter Landscape Parameters – to specify Tomcat’s installation path
  5. Configure Automatically – to run Introscope Host Adapter and Extractor setup.

Select Product

On the “Select Product” page you need to select diagnostics – relevant products installed on the Tomcat server.

 
The “TOMCAT BY APACHE” (of particular version) is the installed product for the Apache Tomcat software.  The “SBOP BI PLATFORM” is the example of another product that can be reported as installed on the Tomcat server (e.g. for BOE XI application).

In this step, please select the Diagnostics Relevant checkbox.  


Note:

In the Managed System Configuration you can only mark all the installed Software Products as Diagnostics Relevant. If you want to differentiate between them (to say, for example, SBOP BI PLATFORM is not Diagnostics Relevant) you can mark single Products in the Technical System Editor. For that click on the “Edit Technical System Software” button and set the “Diagnostics Relevant” check boxes accordingly.

Assign Diagnostics Agent

Connect the Diagnostics Agent to the Solution Manager and assign it to the Tomcat managed system.

If no Diagnostics Agent on Tomcat’s host is yet connected to the Solution Manager, go to the “SLD Agent Candidates” tab and connect the Diagnostics Agent to the Solution Manager.

The operation may take some time to complete.

Go to the “Agent Assignment” tab and assign the Diagnostics Agent to the Tomcat system.


 

 

Enter System Parameters

Specify or verify system parameters for the Tomcat.

Common parameters

  • “Introscope EM” - the Introscope Enterprise Manager’s host and port to which Wily Java Agent on the Tomcat server reports to.
  • “Tomcat Host” – the host name there the Tomcat server is installed
  • “Tomcat Http Port” – the Tomcat’s Http port
  • “HTTPs” checkbox – whether HTTPS protocol should be used (typically not)

     
    Tomcat parameters
  • “Setup User” – the role name for the JMX remote connection’s authorization (the value from the jmxremote.password) or blank, if no applicable.
  • “Password” – the password for the JMX remote connection’s authorization (the value from the jmxremote.password) or blank, if no applicable.
  • “JMX Port” – the port for the JMX remote connection.

Save the data pressing the “Save” button.

Enter Landscape Parameters

Please inspect the Landscape Objects in the hierarchy and check whether any user input is required.  

The status is set to “No Parameter”: no input required.

The status is set to “Default used”: verify the parameter.

The status is set to “Input missing”: enter the parameter.

SLD DS automatically reports the installation path of the Tomcat server. That value is marked as the “default” value for the “Installation path” parameter (“Default used” should appear).

Please check the parameters for the Tomcat system and update if appropriate.

IMPORTANT: for the first time, please save the data in that dialog in any case even if did change any values. Otherwise the landscape parameters will not be propagated to the configuration store of the Solution Manager. 

 
 

Configure Automatically

Perform the “Extractor Setup” and “Introscope Host Adapter” activities. It may take several minutes.

Please verify that both activities have been completed successfully. The status for both must be set to Green.

 
Check also the messages and other detailed information in the Log area at the end of the screen.

The Apache Tomcat is configured now for End-to-End Root Cause Analysis

7. Manual Setup Steps and Troubleshooting

Typically, no manual configuration is necessary for Apache Tomcat server.  The described setup steps should enable E2E Root Cause Analysis for Apache Tomcat system. However, for some E2E RCA application it might be necessary to perform addition configuration in Solution Manager/Tomcat Server. This chapter provides the details helpful for troubleshooting/manual configuration in problem cases.

Configuration for E2E Exception Analysis and Log Viewer

Solution Manager 7.1 can display Tomcat’s log files in the Log Viewer application and collect and analyze errors in the E2E Exception Analysis application. However, logging in Tomcat is very flexible. Tomcat logging can utilize different framework (Log4J, Java logging etc.) and can be configured very differently.  Logging depends also on the OS settings/user preferences.  

The Log Viewer/E2E EA for Tomcat functionality depends on correct configuration that matches the log settings of the particular Tomcat system.

E2E Exception Analysis configuration

Open “SAP Solution Manager Administration WorkCenter” -> “Framework” -> “Root Cause Analysis” -> Tab “Views”


Click on the “Display configuration” and inspect the configuration for Apache Tomcat (search for “catalina” in the config.xml file)

 

Following Tomcat log files are configured in E2E Exception Analysis for Apache Tomcat

stderr*.log
stdout*.log
catalina*.log
manager*.log
host-manager*.log
localhost*.log
admin*.log
jakarta_service*.log

The files are collected from the Tomcat default log folder %CATALINA_HOME%/logs. Please make sure, Tomcat logging is configured properly.  

Log Viewer configuration

Open “SAP Solution Manager Administration WorkCenter” -> “Framework” -> “Root Cause Analysis” -> Tab “Stores”
 
 
Navigate to the node “APACHE TOMCAT 5.5” and inspect the LOGS storec.

Following Tomcat log files are configured in LogViewer application for Apache Tomcat

stderr*.log
stdout*.log
 catalina*.log
manager*.log
host-manager*.log
localhost*.log
admin*.log
jakarta_service*.log

Log file parser configuration

Regular expressions are used to parse Tomcat’s log file. They are specified in the LV_config.xml configuration file of the Diagnostics Agent. To check the configuration open “Diagnostics Agent Administration” -> “Application Configuration” -> “com.sap.smd.agent.application.logfilecollector” application. Select the correct Diagnostics Agent and download the LV_config.xml file


 
Inspect the LV_config.xml file content. Find the configuration sections for Tomcat “Tomcat_FileSet” (to parse Tomcat logs) and “Tomcat_Jakarta_FileSet” (to parse Tomcat service log on Windows).

The regular expressions specified by the parameter “Regex” and the used time stamps “TimestampFormat” must be match Tomcat logs.

For example, to parse Tomcat Windows Service log Jakarta_service.log shown below

[2011-07-08 13:27:06] [info]  Commons Daemon procrun (1.0.5.0 32-bit) started
[2011-07-08 13:27:06] [info]  Running 'Tomcat6' Service...
[2011-07-08 13:27:06] [info]  Starting service...
[2011-07-08 13:27:07] [info]  Service started in 1249 ms.
[2011-07-08 13:27:22] [info]  Stopping service...
[2011-07-08 13:27:23] [info]  Service stopped.
[2011-07-08 13:27:23] [info]  Run service finished.
[2011-07-08 13:27:23] [info]  Commons Daemon procrun finished

The pattern for the regular expression in the Tomcat_Jakarta_FileSet must be

 [timestamp] (optional [location]) [severity] Message

Depending on the user preferences (language, time and date format), the applicable timestamp pattern are:

Format="MMMM dd, yyyy hh:mm:ss a" Country="US"
Format="dd.MM.yyyy hh:mm:ss a" Country="DE"
Format="dd-MMM-yyyy hh:mm:ss" Country="DE"
Format="dd-MMM-yyyy hh:mm:ss" Country="US"
Format="yyyy-MM-dd HH:mm:ss" Country="US"

Example: Tomcat service log in LogViewer

The Windows Service log Jakarta_service will be displayed in the LogViewer application:


 

Configuration for E2E Workload Analysis

E2E Workload Analysis for Apache Tomcat server uses performance data in Wily Introscope from two nodes:

  • Wily Java Agent node -  the data reported by the Wily Java Agent
  • Wily Host Adapter node – the data is reported by the Diagnostics Agent

The Wily Java Agent node has to match following structure

-<Host Name>
               - Tomcat
                               - <Host Name>_<Tomcat’s shutdown port>


The Wily Host Agent node has to match following structure

-<Host Name>
               - SAP HostAgent Process
                               - SAP HostAgent SMDA<Diagnostics Agent instance number>

The Wily Host Agent node must contain the “SAP GC” sub node with an item for the Tomcat server beneath it.


 

If no Host Agent node is available of if “SAP GC” node is missing, please verify that

  1. SAP Host Agent is installed and running properly.
  2. The “Introscope Host Adapter” activity in the “Managed System Configuration” has been successfully performed.
  3. The configuration file SapAgentConfig.xml for the Host Agent application contains correct entries for the Tomcat system.
  4. JVM GC log file is configured on the Tomcat server and its patch is available in the LMDB for the Tomcat system

Configuration of the Host Agent Application

The Wily Host Agent is a Diagnostics Agent application that analyzes and makes JVM GC performance data in Wily Introscope. Its configuration is stored in the SapAgentConfig.xml file.  To check the configuration open “Diagnostics Agent Administration” -> “Application Configuration” -> “com.sap.smd.agent.application.wilyhost” application. Select the correct Diagnostics Agent and download the SapAgentConfig.xml file

 

There should be “destination” and “action” elements for the Tomcat Server.

<destination>
 <property value="C:/APACHE/Tomcat/Tomcat6.0.33/gcTomcat.gc" />
</destination>
 <action prefix="SAP GC|wdfd00291023a_8005"
    name="SAP GC|wdfd00291023a_8005" destination="wdfd00291023a_8005"
    template="SapGC" />

GC log file path for Tomcat in Landscape Browser

The path to the JVM GC log file is reported automatically for Tomcat by the SLD Data Supplier. To check that the value is available in the LMDB, open the Landscape Browser and inspect the properties of JVM object.

Additional information

URL

Title

http://wiki.sdn.sap.com/wiki/display/SMSETUP

SAP Solution Manager Setup Wiki

http://wiki.sdn.sap.com/wiki/display/TechOps/RCA_Home

Root Cause Analysis Wiki

https://service.sap.com/instguides -> SAP Components -> SAP Solution Manager -> Release 7.1

Solution Manager Installation Guides

SAP Note 1508421

SAP SLD Data Supplier for Apache Tomcat

SAP Note 1438005

Wily Introscope Agent for Apache Tomcat server

  • No labels

1 Comment

  1. Former Member

    Thanks for the blog. I have 3 questions:

    1) I don't see the LV_config.xml in our solution manager system. We are on 7.1 SPS12.

    2) Can you post some screenshots on technical monitoring. I noticed that the reporting tab is blank. Is it normal?

    3) No Jakarta service found in the agent log viewer If that is what you refer to. I don't see the Jakarta service from the solman default trace and application trace.