Skip to end of metadata
Go to start of metadata

This page summarizes steps to enable full SSL/TLS communication for Introscope in combination with Solution Manager. The steps are mainly organized by the two communication ports opened by the Enterprise Manager:

  • HTTP port (default = 8081): This port is served by the embedded Jetty server. Converting this port from HTTP to HTTPS mainly involves adapting the file em-jetty-config.xml. This port is typically used when accessing Introscope Webview (e.g. dashboards), the UI5 applications sapdashboard and emergency monitoring, and also by Solution Manager when accessing Introscope. In the picture below the green arrows represent the http access.
  • RMI port (default = 6001): This port is used for incoming agent connections and for Workstation connections. The RMI communication can be done via multiple "channels" - more than one channel can be active at a time, thus opening multiple TCP ports. Adjusting the RMI ports requires changes on Enterprise Manager side and on agent side. As a result, SSL-enabling RMI is substantially more complex than the HTTP → HTTPS conversion.
    In the picture below the red arrows represent the RMI access. 
  • SSL Communication for Enterprise Managers in a cluster (MoM and Collectors).
    Communication between Enterprise Managers in a cluster (collectors and MoM) uses RMI. Introscope does not support using SSL communication between collectors and MoM. In the picture below this is represented by black arrows.

Strictly speaking even more combinations are possible: Agents and Workstations can also use HTTP and HTTPS. These options are not explained here.


Prerequisites

  • Solution Manager 7.2
  • Introscope Enterprise Manager version 9.7 or higher
  • Management module package 9.7 or higher

HTTPS Access to the Enterprise Manager (Webview, Webstart, sapdashboard etc)

The procedure consists of up to three parts which must be executed for standalone EM, MoM and the collectors:

  • Enable https port - this will switch to https, but still use the default certificates
  • (optional) update keystore and em-jetty-config.xml to use your own certificate
  • (optional) update https client in Solution Manager Java stack to check certificates

Procedure to Enable HTTPS Port for EM WebView

  1. Under the EM installation directory, open the file 'IntroscopeEnterpriseManager.properties' which is located under the folder '\config'.
  2. In this file, activate the property 'introscope.enterprisemanager.webserver.jetty.configurationFile' to enable customizing of the Jetty server. This is done by setting the following parameter:

    introscope.enterprisemanager.webserver.jetty.configurationFile=em-jetty-config.xml

    With this parameter set, the customization present in the file 'em-jetty-config.xml' will take action. 

  3. Open the file 'em-jetty-config.xml' and ensure the correct ports are set for  http and/or https. By default, this file will come with the port 8444 set for HTTPS and port 8081 for HTTP.

  4. Ensure also that the desired section is not commented. By default the HTTPS settings will come active and the HTTP settings will be commented.

  5. Again in the file 'IntroscopeEnterpriseManager.properties' set the following parameter with the HTTPS port:

    introscope.enterprisemanager.webserver.port=8444


    Even though this property is ignored by Introscope after enabling the Jetty customization, it will still be used by Solution Manager to generate URLs. To enable HTTPS after changing the port here the flag for "HTTPS" must be set in the Solution Manager Infrastructure Preparation Configuration afterwards.

  6. In the Solution Manager system, access the Infrastructure Preparation under section 'Define CA Introscope". Here, use the discover button to reload the EM information.
  7. Set the 'HTTPS' flag for the Enterprise Manager entry. Here the 'detail' section will show the new information:
  8. Restart the Enterprise Manager. This will ensure it is using the newly set port. Afterwards, the EM and SolMan will connect correctly through the new port.


    Also, from this point on, the WebView and Workstation will also use the HTTPS port.

Note – when using the HTTPS port 8444 for the EM WebView the complete SSL/TLS support in Introscope including the set of supported ciphers is standard Java functionality – it is not anything Introscope specific. This means the SSL/TLS Support depends on the underlying JRE/JDK being used.

If you encounter errors using a “weak” cipher please check that the JRE installed as part of the EM is fully equipped with the “unlimited strength jurisdiction policy”.

The policy files can be downloaded from the link:

http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html

In case of further issues always ensure that you are using a current and supported JRE for running with the EM. If necessary add the unlimited strength jurisdiction policy files again.

Create a server certificate for HTTPS usage in Introscope EM

Example sequence of commands to generate a server certificate for SSL.

Change the values used here:

  • myca is the certificate authority signing your keys.
  • fully qualified domain name of the EM host: myhost.my.domain
  • keystore password and key password: caapm9x


SSL Certificate
# generate a key pair
keytool -keystore emssl.jks -storepass caapm9x -alias emssl -genkeypair -keyalg RSA -keypass caapm9x -dname "CN=myhost.my.domain, OU=AGS, O=SAP AG, L=Walldorf , ST=Baden-Wuerttemberg, C=DE"
# generate a certificate signing request (into emssl.csr)
keytool -keystore emssl.jks -storepass caapm9x -alias emssl -certreq -file emssl.csr

Submit the certificate signing request (emssl.csr) to your certificate authority. We assume that the resulting signed certificate is stored in myhost.cer.

Copy the original key store to a new name:

SSL Certificate
# Copy the original key store to a new name
copy emssl.jks emssl2.jks
# import the ca certificate first (export from Internet Explorer e.g.)
keytool -keystore emssl2.jks -storepass caapm9x -importcert -file myca.cer -alias myca -noprompt
# import the signed certificate second
keytool -keystore emssl2.jks -storepass caapm9x -importcert -file myhost.cer -noprompt -alias emssl -trustcacerts
# check the key store
keytool -keystore emssl2.jks -storepass caapm9x -list
Keystore-Typ: JKS 
Keystore-Provider: SUN
Ihr Keystore enthõlt 2 Eintrõge.
myca, 03.04.2014, trustedCertEntry, 
Zertifikatsfingerabdruck (MD5): B5:5E:64:10:AE:09:6B:71:AC:77:A4:EF:61:99:21:71 
emssl, 03.04.2014, PrivateKeyEntry, 
Zertifikatsfingerabdruck (MD5): 14:AC:92:CD:E7:E6:B4:D4:5C:DF:3E:AF:53:78:61:D7
 

Edit em-jetty-config.xml to use this new trust store. Typically these properties must be updated:

  • keystore
  • password
  • keyPassword
  • certAlias

The last property (certAlias) is required in some cases only and should refer to the alias used for the EM certificate.

em-jetty-config.xml (up to 10.5)
<Call name="addConnector"> 
<Arg> 
<New class="com.wily.webserver.TrustingSslSocketConnector"> 
<Set name="validateCertificates">false</Set> 
<Set name="HeaderBufferSize">8192</Set> 
<Set name="RequestBufferSize">16384</Set> 
<Set name="verifyHostnames">false</Set> 
<Set name="port">8963</Set> 
<Set name="keystore"><SystemProperty name="introscope.config" default="./config" />/emssl2.jks</Set> 
<Set name="password">caapm9x</Set>
<Set name="keyPassword">caapm9x</Set>
<Set name="truststore"><SystemProperty name="introscope.config" default="./config" />/internal/server/keystore</Set> 
<Set name="trustPassword">OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v</Set> 
<Set name="needClientAuth">false</Set>
<Set name="certAlias">emssl</Set> 
</Arg>   
</Call> 

Introscope 10.7 warning

For Introscope 10.7 the format of the file em-jetty-config.xml has changed substantially due to the upgrade of the embedded Jetty to version 9.4.x. You cannot copy the file from Introscope version 10.5 or earlier! When transitioning from 10.5 or earlier to 10.7 you can map the properties as follows:

<= 10.510.7
keystore

KeyStorePath

password

KeyStorePassword

keyPasswordKeyManagerPassword

The properties to change for Introscope 10.7 are:

  • KeyStorePath
  • KeyStorePassword
  • KeyManagerPassword (only if different)
  • certAlias


em-jetty-config.xml fragments (10.7)
                    <Set name="KeyStorePath">
                      <SystemProperty name="introscope.config" default="./config" />/ssl/emssl2.jks
                    </Set>
                    <Set name="KeyStorePassword">caapm9x</Set>
                    <!-- Typically, key store password and private key password are identical. When private key
                         password is different than key store password, uncomment and set the following setting to
                         private key password. -->
                    <!-- Set name="KeyManagerPassword">OBF:1v2j1uum1xtv1zej1zer1xtn1uvk1v1v</Set -->

                    <Set name="certAlias">emssl</Set>

Check if the EM is Configured Properly for HTTPS

After saving the file em-jetty-config.xml restart the Enterprise Manager and check that the HTTPS configuration is active:

  1. The log file logs/IntroscopeEnterpriseManager.log should contain log entries as given in the example:
    1. ServerConnector should explicitly mention SSL and the HTTPS port (line 1)
    2. Directly below you should see details about the certificate that is selected via certAlias above. (line 2-4). If you do not see at least one or two of these lines this means your HTTPS configuration is NOT VALID!
  2. Access the Enterprise Manager via HTTPS: https://myhost.my.domain:8444/. If you decide to stay with the default certificate you may get certificate warnings. Using your own company-signed certificate no warnings should occur.


Log Example Showing HTTPS Configuration
[INFO] [main] [Manager.EMWebServer] EM[ServerConnector@3292d91a{SSL, (ssl, http/1.1)}{0.0.0.0:8444}] Request Header Size is :16384
[INFO] [main] [Manager.EMWebServer] Certificate 'emssl' subject: CN=myhost.my.domain,O=My company ltd,L=Frankfurt,C=DE
[INFO] [main] [Manager.EMWebServer] Certificate 'emssl' issuer: CN=Some CA,O=My company ltd
[INFO] [main] [Manager.EMWebServer] Certificate 'emssl' subject alternative DNS name: myhost.my.domain

Configure HTTPS client in Solution Manager

In Solution Manager Infrastructure Preparation, Step "Define CA Introscope", set the "https" flag for the EM.

You can configure the destinations towards Introscope in the Solution Manager Java stack to check certificates:

Launch Netweaver Administrator (http://host:port/nwa) on the Java stack of Solution Manager, go to System Management --> Configuration --> Destinations. There is one http destination entry for every Enterprise Manager following the naming scheme IntroscopeEM_emhost@port (port is the RMI port). The url specifies the http / https URL prefix that is used to access the Enterprise Manager via http. You can configure use of certificates here. Administration of the keystores etc. must be done via Visual Administrator (server -> services -> key storage) due to restricted functionality in NWA of Netweaver 7.02. Administration of destinations can also be done via Visual Admin (server -> services -> destinations).

Final Step for HTTPS: Check Status in Solution Manager

In Solution Manager Infrastructure Preparation, Step "Define CA Introscope": Make sure that all changes are saved and that all Enterprise Managers show status "green" after a refresh. 

HTTPS configuration is completed.



RMI via SSL: Communication between Enterprise Manager and Introscope Agents

Prerequisites

It may be necessary to explicitly enable TLSv1 on Enterprise Manager side since potentially not all components (SAP JVM 6, Introscope agent runtime in diagnostics agent) support TLSv1.2. In particular, Introscope agent versions 9.7 and lower do not support TLSv1.2. For this purpose put the following property into config/IntroscopeEnterpriseManager.properties:

introscope.enterprisemanager.protocols.channel2=TLSv1.2,TLSv1.1,TLSv1

Procedure

  1. Enable RMI port for SSL in the Enterprise Manager (see  section below)
  2. Edit the configuration of the Agents so the correct socket factory is used to connect to the Enterprise Manager(see sections below)

Enable RMI Port for SSL in the Enterprise Manager 

  1. Under the EM installation directory, open the file IntroscopeEnterpriseManager.properties which is located under the folder /config. Most of the properties mentioned below exist already in the file, but are commented out.
  2. In this file, edit the propertyintroscope.enterprisemanager.enabled.channels to define which channels will be enabled and accept connections for the EM. Each "channel" refers to a set of properties that configure a TCP port for incoming connections.
    By default, the 'channel1' is the default RMI port (6001) and the 'channel2' is the SSL RMI port (6443).
    It is possible to activate just one channel or both channels like the following:

    # SSL channel only
    introscope.enterprisemanager.enabled.channels=channel2
    
    # or activate both channels: default and SSL
    introscope.enterprisemanager.enabled.channels=channel1,channel2

    Recommendation

    When you activate only channel 2 for SSL and restart the EM all existing agents cannot connect anymore. To avoid this it is recommend to activate temporarily both channels and if required remove channel1 later when all agents are reconfigured.

    Collectors only

    For Collectors you must activate both channels. 

  3. Modify the property introscope.enterprisemanager.workstation.connection.channel to define which channel will be used by the Solution Manager. If you set this property with the value 'channel2' , when configuring agents in the future, the port set for channel2 will then be used by the agent to connect to the Enterprise Manager. The property will look like following:

    MoM and Standalone Only

    For Collectors the property introscope.enterprisemanager.workstation.connection.channel must not be changed. 

    # This property is used for Workstations launched via Java Web Start, to set
    # the communication port used for communicating with the Enterprise Manager.
    introscope.enterprisemanager.workstation.connection.channel=channel2
  4. Restart the Enterprise Manager so the changes take action
  5. In the Solution Manager system, access the Infrastructure Preparation under section 'Define CA Introscope". Here, use the discover button to reload the EM information.

  6. If you have set both channels as active, you should then see two Enterprise Manager entries here for MoM and Standalone EM.


    Collectors should appear only once with the 'old' port.
     

  7. If you have already performed the steps on section 'Procedure to Enable HTTPS Port for EM WebView', ensure to check the HTTPS flag here for the new EM entry.

Updating Agents Settings to Use Correct Socket Factory

Updating the Agent Profile Templates

With SolMan 7.2, it is possible to adjust the profile templates directly so the newly configured agents use the SSL Socket Factory and Port. You can upload templates for the agent profiles via diagnostics agent administration.

There is one template for wilyhost and one template for each byte code agent version. These templates can be customized on two different scopes:

  • Scope <Global>: If you switch the template on this scope to SSL then all managed system configurations for BCA and wilyhost will always use SSL!
  • Scope for individual hosts: The changes will affect only the hosts for which the templates are customized. This scope overrides customizing on scope <Global>

Agent profile template for Introscope Host Adapter (wilyhost)

  1. In Diagnostics Agent Administration select the tab 'Application Configuration' and navigate through 'com.sap.smd.agent.application.wilyhost / Application Resources / IntroscopeSapAgent.profile.template.'
  2. Download the default resource and save it locally as text file (.txt).
  3. Rename the downloaded file to IntroscopeSapAgent.profile.template (replace the underscore with dot and remove the file extension .txt)
  4. Edit the local file and change introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wily.isengard.postofficehub.link.net.SSLSocketFactory. Save it again.
  5. In agent administration, select the correct scope for the file(Global or specific host). Then browse to the modified file and press Upload.



    Here it is important to note that the template for wilyhost will only be considered if the setup has not yet been executed before( if the resource 'IntroscopeSapAgent.profile' is not yet customized in the relevant scope). To force the use of the newly uploaded template, remove the customizing of 'IntroscopeSapAgent.profile' in the relevant scope(host for which the configurations have been done in the past). This can be done by accessing the file 'IntroscopeSapAgent.profile' in the same path, selecting the relevant host in the scope and then removing the customized file.

  6. In the Managed System Configuration for the relevant hosts, ensure the correct EM entry is selected. This is important to ensure the correct port will be pushed into the agent configuration.

  7. Execute the step Introscope Host Adapter in Managed System configuration.

  8. If the Wilyhost was active before, a complete restart of the diagnostics agent is needed.

It is possible to check if the agent is connecting correctly to the Enterprise Manager by checking the file 'jvm_smdagent.out' under the work folder inside the agent installation path. Entries like the following will be present:

[INFO] [IntroscopeAgent.IsengardServerConnectionManager] Connected Agent to the Introscope Enterprise Manager at <HOST>:<SSL PORT 6443>,com.wily.isengard.postofficehub.link.net.SSLSocketFactory. Host = "<HOST>", Process = "SAP HostAgent Process", Agent Name = "SAP HostAgent SMDA98".


Agent profile template for Byte code adapter (wilybcaj5)

  1. In Diagnostics Agent Administration select the tab 'Application Configuration' and select the application relevant for your Introscope agent:
    • com.sap.smd.agent.application.wilybcaj5 for Introscope agent 9.x and higher
  2. Under the selected application node select Application Resources and the agent profile, e.g. 'WilyResources/ISAGENT.9.1.5.3-2014-10-22/IntroscopeAgent.profile'
  3. Download the default resource and save it locally.
  4. Edit the local file and change introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wily.isengard.postofficehub.link.net.SSLSocketFactory. Save it again.
  5. In agent administration, select the correct scope for the file(Global or specific host). Then browse to the modified file and press Upload.



  6. In the Managed System Configuration for the relevant hosts, ensure the correct EM entry is selected. This is important to ensure the correct port will be pushed into the agent configuration.



  7. Execute the step 'Byte Code Adapter Installation' in Managed System configuration.


     
  8. Restart the managed system to activate the changes.

Optional: Configuration of Certificates for Enterprise Manager and Agents

Default Behavior

By default the initial key stores are used and no additional configuration is needed. This means, however, that certificates are not validated for SSL communication.

Configure RMI via SSL for Certificates (EM side)

You can optionally configure the SSL port to allow only trusted agent connections. This is achieved by setting introscope.enterprisemanager.needclientauth.channel2=true. This requires the following:

  • Create and configure a trust store on EM side via the properties
    • introscope.enterprisemanager.trustpassword.channel2
    • introscope.enterprisemanager.truststore.channel2
    • introscope.enterprisemanager.trustpassword.channel2.plaintextpassword
  • Configure a key store on agent side (next section)

Edit the following properties in IntroscopeEnterpriseManager.properties. The effect is that all agents not providing a trusted certificate will be blocked from connecting to the Enterprise Manager.

# The truststore is optional.  It is needed only if client authentication is required.
# If no truststore is specified, the EM trusts all client certificates.
introscope.enterprisemanager.truststore.channel2=myTruststore
# To change the existing password, enter the new password and set this property to true. 
# Note: If this property is set to true and the password is not changed, the existing encrypted password will be encrypted again. 
# If password field for a new channel is configured, add the corresponding   
# plaintextpassword field and set it to true to enable encryption.  
introscope.enterprisemanager.trustpassword.channel2.plaintextpassword=true
# The password for the truststore 
introscope.enterprisemanager.trustpassword.channel2=mySecretPassword
# Set to true to require clients to authenticate. 
# If true, clients must be configured with a keystore containing a certificate trusted by the EM. 
# Default is false 
introscope.enterprisemanager.needclientauth.channel2=true

Configure RMI via SSL for Certificates (Agent side)

Procedure to configure a keystore for the RMI communication via SSL on agent side. Goal: Allow only trusted agents to connect. Agent authenticates via a certificate which is configured as trusted in the EM.

Note that there is no automated transfer of the keystore from Solution Manager or Enterprise Manager to the agent host. You have to explicitly take care for the transfer and specify a path that is available on agent side (d:\isagent\emssl2.jks in the example below).

Edit the following properties in IntroscopeAgent.profile:

introscope.agent.enterprisemanager.transport.tcp.socketfactory.DEFAULT=com.wily.isengard.postofficehub.link.net.SSLSocketFactory 
introscope.agent.enterprisemanager.transport.tcp.keystore.DEFAULT=d:\\isagent\\emssl2.jks 
introscope.agent.enterprisemanager.transport.tcp.keypassword.DEFAULT=caapm9x




Troubleshooting

Jetty Configuration Dump

For the https configuration you can activate a dump of the configuration: If em-jetty-config.xml is activated as described above add the property dumpAfterStart=true, immediately after the <Configure id="Server" ...> tag:

Jetty Dump
<Configure id="Server" class="org.eclipse.jetty.server.Server">

<Set name="dumpAfterStart">true</Set>

This will trigger a dump of the Jetty configuration to the stdout channel: EMService.log on Windows, em.log on Unix.

Http Access Log

To check if http requests arrive at the EM at all you can activate logging of all http requests. For this purpose add the following section to em-jetty-config.xml. All http requests will arrive in a file IntroscopeHttp-*.request.log.

Jetty Request Log
  <Call name="insertHandler">
    <Arg>
      <New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler">
        <Set name="requestLog">
          <New id="RequestLogImpl" class="org.eclipse.jetty.server.NCSARequestLog">
            <Set name="filename"><Property name="jetty.logs" default="./logs"/>/IntroscopeHttp-yyyy_mm_dd.request.log</Set>
            <Set name="filenameDateFormat">yyyy_MM_dd</Set>
            <Set name="LogTimeZone">GMT</Set>
            <Set name="retainDays">10</Set>
            <Set name="append">true</Set>
            <Set name="LogLatency">true</Set>
          </New>
        </Set>
      </New>
    </Arg>
  </Call>

SSL Trace

To get more details on TLS connection negotiation you can activate the standard Java SSL tracing. For this purpose add a Java VM parameter like -Djavax.net.debug=ssl:handshake:verbose:keymanager:trustmanager or -Djavax.net.debug=all:

On Windows add a new line to bin\EMService.conf:

wrapper.java.additional.8=-Djavax.net.debug=ssl:handshake:verbose:keymanager:trustmanager

.8 is the next free number for the parameter group wrapper.java.additional. Depending on your configuration you may have to choose a different number.

On Unix change the property lax.nl.java.option.additional in Introscope_Enterprise_Manager.lax.

References

Documentation can be found on every Enterprise Manager where the SAP management module package is deployed. Use the link https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/application-performance-management/10-7.html to get to the overview page of all guides.

  1. CA APM Configuration and Administration Guide, Chapter Configuring Enterprise Manager Communications
  2. CA APM Java Agent Implementation Guide, Chapter Installing and Configuring the Java Agent, Configuring the connection to the Enterprise Manager
  3. CA APM Security Guide
  4. Official Jetty documentation: https://www.eclipse.org/jetty/documentation/ (9.4.x is embedded in Introscope 10.7)

Reservations

  • Help links from SAP dashboards to the help content are currently always generates as HTTP links. Change to https manually to get the help content displayed.
  • Only one of the channels for each HTTP and RMI can be used in Solution Manager: either secure or non-secure
  • Currently no support for certificate and keystore handling in Solution Manager setup UIs. You can use the destination service of the J2EE server (via Visual Admin / NWA) to configure certificates.
  • Byte code injection agent and Introscope Host Adapter setup via Solution Manager do not support the protocol types "RMI via SSL", "HTTP", "HTTPS" right now. You have to manually adapt the profiles to switch the protocol.
  • No labels