Registration

Dear SAP Community Member,
In order to fully benefit from what the SAP Community has to offer, please register at:
http://scn.sap.com
Thank you,
The SAP Community team.
Skip to end of metadata
Go to start of metadata

Outside Discovery - Troubleshooting

For troubleshooting the Outside Discovery identify the issue and proceed accordingly:

IssueSolution
A logical hostname is alternately associated to two or more physical hosts, i.e. it is toggling between multiple physical hosts.See Hostname Switching.
The Outside Discovery is not sending any data at all.Aquire the Outside Discovery Logs and Analyse them.
The physical or a logical host name reported by the Host Outside Discovery is wrong.Aquire the Outside Discovery Logs and Analyse them.
The FQDN of a Logical Host is Missing.See The FQDN of a Logical Host is Missing.
The host name a database instance is associated to is wrong.Aquire the Outside Discovery Logs and see The Host Name Associated to a Database is Wrong.
Any other issue.Aquire the Outside Discovery Logs and Analyse them. If this did not help, check Outside Discovery - Solution Manager.

Acquiring the Log Files

For troubleshooting the Outside Discovery at the Diagnostics Agent, check the debug log of the Outside Discovery. The logs can be downloaded as described in the SAP Note 1076573. Compendium:

  1. In the Agent Administration application, tab Agent Log Viewer, link Log Configuration, set the levels of the log files e2edcc.log, e2edcc_host.log and e2edcc_db for the Diagnostics Agent on the physical host with the issue to Debug. In case of issues with an agent node (on a logical host), you need to identify the related physical host and change the log configuration of this host). Press Save in the Log Configuration dialog.
  2. Restart the Diagnostics Agent and the SAP Host Agent.
  3. Wait for 1/2 hour.
  4. Again under Agent Log Viewer, use the link Download logs for downloading the logs of the Diagnostics Agent. 
  5. Again under Agent Log Viewer, reset the log levels to their previous values.

For troubleshooting the Outside Discovery at the Solution Manager, do the following:

  1. In the NetWeaver Administrator, navigate to Troubleshooting → Logs and Traces → Log Configuration.
  2. At the ""Tracing Location" and search for "com.sap.sup.dataenrichment.impl.DataEnrichmentManager".

  3. Change the Severity to Debug and save the configuration.

  4. Restart the affected Diagnostics Agent to retrigger the Outside Discovery.
  5. Navigate to Troubleshooting → Logs and Traces → Log Viewer.
  6. Switch to View → Open View → Developer Traces and filter the location for DataEnrichmentManager.

Analysing the Log Files

As the Outside Discovery is only executed on Diagnostics Agent nodes associated to a physical host, the log files located at SMDAgent_<logicalHost>/log (with <logicalHost> being a logical hostname) can be ignored. Identify the most recent SMDAgent/log/e2edcc.*.log file; it should contain Debug log entries. Check the log for the following entries and proceed accordingly:

Log EntrySolution

java.net.SocketTimeoutException: Read timed out (local port 12345 to address 10.11.12.13, remote port 1128 to address 10.11.12.13)

See section The Diagnostics Agent Cannot Reach the Host Agent
Exception calling SAPHostControl Web Service
[DCCService.init] DCC DISABLEDSee section Outside Discovery is Disabled.
[PhysicalHostPushJob] Could not find hostname in 'Hostnames' list. Using Name as fallback full hostnameSee section SAP Host Agent Does not Provide FQDN.

Debug [PhysicalHostPushJob:getComputerSystem] resolveFQDNandIP failed - Using fallback method.
Debug [PhysicalHostPushJob:getComputerSystem] GetFirstValidFQDNandIP - Fallback Method. 

Error [DatabaseInfoPushJob] Exception calling DataEnrichment Manager Bean.
An exception occured during the execution of the function 'FM_DIAGLS_PUSH_DATABASE': The Host 'myHostName' does not existnullnullnull

See section The Host Does not Exist.

Switch to the SAP Host Agent Outside Discovery

In case the Solution Manager Version equals or is higher than 7.2 SP 6 it is recommended to switch to the Outside Discovery provided by SAP Host Agent. This will disable the Outside Discovery functionality of the Diagnostic Agents. See Note 2556432.

The SAP Host Agent will write a dedicated Outside Discovery log file (outsidediscovery.log) which can be found in the SAP Host Agent work directory (/usr/sap/hostctrl/work or C:\Program Files\SAP\hostctrl\work).

In case of problems with the SAP Host Agent Outside Discovery it is recommended to increase the SAP Host Agent trace level to 3 according to the Host Agent's troubleshooting guide. SAP Host Agent's troubleshooting guide. With the increased traces the Outside Discovery should be triggered manually by executing:

/usr/sap/hostctrl/exe/saphostctrl -function ExecuteOutsideDiscovery -sldreg

or

C:\Program Files\SAP\hostctrl\exe\saphostctrl.exe -function ExecuteOutsideDiscovery -sldreg

After the execution all of the SAP Host Agent's log files in the SAP Host Agent work directory should be attached to the incident and send to SAP.

Outside Discovery - Common Issues 

Hostname Switching 

An association of a logical host that keeps switching between multiple physical hosts in the LMDB is most likely caused by a wrong network setup (see SAP Note 2513388)The logical host is switching because it is defined on various physical hosts. Thus, this logical host is independently reported by the Outside Discovery of each physical host it is associated to. As this is not allowed according to SAP Note 962955, you need to fix the setup of the dedicated managed systems such that all logical hosts are unique.

The Outside Discovery only reports logical hosts that are associated to

  1. an SAP System Instance, or
  2. an Agent Node (Agents-on-the-Fly).

As a consequence:

  1. An SAP System Instances must not have be associated to a logical host name that is also associated to an SAP System Instance or an Agent Node on another physical host. This association is declared in the profiles of the SAP System Instances. 

  2. An Agent Node must not be associated to a logical host name that is also associated to an Agent Node or an SAP System Instance on another physical host.

Solution:

 Decide which physical host you want to have the logical host associated to. Remove it on the other physical host as described below:

  1. Ensure that SAP System Instances on different physical hosts have different logical host names as claimed in the NetWeaver installation guide [1] and do not setup Agent Nodes with the same logical host name on a different physical host.
    N.B.: The Outside Discovery asks the SAP Host Agent for the logical hosts of the SAP System Instances on this host and reports these to the LMDB as logical hosts, not checking whether there is actually such a logical host in the network configuration. The Solution Manager processing the Outside Discovery data then apparently stores these as logical hosts in the LMDB, even if the respective host name is not among the “alternative host names”. To change the logical host name of an SAP Instance follow the instructions of the guide "SM71SP5OutsideDiscMissingFQDN" → section "4.3 Step 3 – Checking SAP Profile" available at SAP Note 1611483.
  2. Remove the logical host name in the network configuration or exclude the logical host name in the Agents-on-the-fly setup for one of that physical host. If the two physical hosts are thought to be a high availability setup according to High Availability Installation Strategy (System Switch-Over) keep in mind that the fallback host must only be created once the other host crashed (by a switch over software) [1].

If non of the above mentioned solutions is feasible for you, upgrade your LM-SERVICE.SCA to SP 6 (SAP Note 2507007) or higher and switch to the Host Agent's Outside Discovery. The Outside Discovery of the SAP Host Agent works differently to the Diagnostics Agent, but here is no guarantee that this will fix your issue. The result is depending on your concrete scenario. In case the switch changed something contrary to your expectations, you can always switch back to the Outside Discovery of the Diagnostics Agent.

References:

[1] NetWeaver installation guide: http://service.sap.com/installnw75 (alternative link: https://websmp208.sap-ag.de/guidefinder) section 3.7 Planning the Switchover Cluster for High Availability

Outside Discovery is Disabled

If there is [DCCService.init] DCC DISABLED entries in the SMDAgent/log/e2edcc.*.log files, the Outside Discovery is disabled. To enable the Outside Discovery open the Agent Administration, go to the Application Configuration tab, click on com.sap.smd.agent.application.e2edcc and set e2edcc.enable to true.

SAP Host Agent Does not Provide FQDN of the Physical Host

If there is one of the following entries in the SMDAgent/log/e2edcc.*.log files, you need to review your network configuration, as described in SAP Note 962955

[PhysicalHostPushJob] Could not find hostname in 'Hostnames' list. Using Name as fallback full hostname
Debug [PhysicalHostPushJob:getComputerSystem] resolveFQDNandIP failed - Using fallback method.
Debug [PhysicalHostPushJob:getComputerSystem] GetFirstValidFQDNandIP - Fallback Method. 

 

The FQDN of a Logical Host is Incorrect or Missing

Potential Cause 1

Search the log files for the following:

[PhysicalHostPushJob] 3 SAP instance(s) found.
[PhysicalHostPushJob] SAP instance 1:
[PhysicalHostPushJob]   FQDN          = my-host-name.domain.de
[PhysicalHostPushJob]   hostname      = my-host-name
[PhysicalHostPushJob]   IP address    = 10.10.10.11
[PhysicalHostPushJob]   SID           = XYZ
[PhysicalHostPushJob]   System number = 00
...
[PhysicalHostPushJob] adding agent node: AgentHostname [mm_isAgentNode =true, mm_saplocalhost=my-host-name, mm_resolvedFQN=my-host-name.wrong-domain.de, mm_resolvedSQN=my-host-name, mm_resolvedIP=10.10.10.11, mm_sid=SMD, mm_pathInstance=/usr/sap/SMD/SMDA98, mm_instanceNumber=98]

Here the FQDN of the SAP instance on my-host-name is correct (my-host-name.domain.de), but the LMDB shows a different FQDN, namely my-host-name.wrong-domain.de, because the FQDN of the related Diagnostics Agent node resolves to my-host-name.wrong-domain.de. To fix this, ensure that the host name can be resolved correctly by adjusting your network settings. The Outside Discovery always prefers the FQDNs and IP addresses of the Diagnostics Agent nodes to the information provided by the SAP instances, i.e. the "adding agent node" entry needs to contain the correct data.

Potential Cause 2

If the FQDN stated in the Outside Discovery log does not contain the domain, the JVM might not be able to resolve the FQDN. The cause might be a misconfigured /etc/Hosts file. An evidence is that, the ping command does not return the correct FQDN, whereas nslookup does:

my-host-name:root> ping -c 1 my-host-name
PING my-host-name: (10.11.12.13): 56 data bytes
64 bytes from 10.11.12.13: icmp_seq=0 ttl=255 time=0 ms
 
my-host-name:root> nslookup my-host-name
Server: 10.11.12.13
Address: 10.11.12.13#42
Name: my-host-name.domain.de
Address: 10.11.12.13

Here, the system identifies the host as "my-host-name" and not with the FQDN. If the operating system can resolve a host name by the hosts file, The native OS functions (and that's what is called by Java) return the first host name that's configured in the hosts-file, namely "my-host-name". An nslookup is skipped then.

To fix this, change the /etc/hosts line:

10.11.12.13 my-host-name my-host-name.domain.de

to

10.11.12.13 my-host-name.domain.de my-host-name

In rare cases the Java parameter "-Dcom.sap.jvm.net.resolveLocalhost=true" might also help.

The Host Name Associated to a Database Does not Exist

Error [DatabaseInfoPushJob] Exception calling DataEnrichment Manager Bean.
[EXCEPTION]
java.rmi.RemoteException: RfcExecutionException; nested exception is: 
	com.sap.sup.admin.abap.rfc.exception.RfcExecutionException: An exception occured during the execution of the function 'FM_DIAGLS_PUSH_DATABASE': The Host 'myHostName' does not existnullnullnull
...

The log files of the Outside Discovery contain the error message stated above, if the Outside Discovery tries to report a database installation to the LMDB which is rejected because the host name that is associated to this database is not know to the LMDB. The database is reachable via myHostName and there is no mechanism to prohibit this. The Database Outside Discovery can only report a database if the "connect address" has been previously reported as a host by the Host Outside Discovery. The latter only reports host names that are either the physical host name the Diagnostics Agent is installed on, a logical host name associated to an SAP System Instance, or a logical host name associated to a Diagnostics Agent node. A host name that is neither associated to a Diagnostics Agent node nor an SAP System Instance is considered as irrelevant an will not be reported to the LMDB.

The following options will fix this issue:

Recommended Solution:

Switch to the Host Agent's Outside Discovery (LM-SERVICE SP 6 (SAP Note 2507007) or higher is required).  

Alternative Solutions:

  1. Reconfigure the managed system such that the host name is reported by the Host Outside Discovery.
  2. If you do not want to reach the database via myHostName, remove the host name from your network configuration.
  3.  Manually create the missing host in the LMDB.

The Host Name Associated to a Database is Wrong

If a database instance is associated to a unintended host name in the LMDB or if the association between a database and the desired host name gets deleted automatically every once in a while, access the Outside Discovery log files as described in section Aquire the Outside Discovery Logs, search for "DatabaseInfoPushJob.getDatabaseList " and detect the dedicated database instance. You will find log entries similar to the following:

[DatabaseInfoPushJob.getDatabaseList]  > Property: [Database/InstanceName]=[DBX].
[DatabaseInfoPushJob.getDatabaseList]  > Property: [Database/Host]=[my-host-name].
[DatabaseInfoPushJob.getDatabaseList]  > Property: [Database/Vendor]=[sap].
[DatabaseInfoPushJob.getDatabaseList]  > Property: [Database/Type]=[sap].
...
[DatabaseInfoPushJob] Get Database Properties for: Database[DBX1][DBX][SAP] .
[DatabaseInfoPushJob] > Property: [Database/DBRelease]=[12.3.4.5.6].
[DatabaseInfoPushJob] > Property: [Database/Capability/CopyMethods]=[Offline,Online].
[DatabaseInfoPushJob] > Property: [Database/ConnectAddress]=[Protocol=TCP;Host=my-connect-host-name;Port=1234 IPC:LISTENER].
[DatabaseInfoPushJob] > Property: [Database/ConnectAddress]=[Protocol=TCP;Host=my-second-connect-host-name;Port=5678 IPC:LISTENER].
[DatabaseInfoPushJob] > Property: [Database/InstanceList]=[Name=DBX1;Host=my-third-host-name;Name=DBX2;Host=my-forth-host-name].
...

The host name the Outside Discovery associates to the database is taken from "Database/Host", here "my-host-name". If this is not the desired host name, adjust the host name of the dedicated database accordingly. Additioally the last "Database/ConnectAddress" entry is also reported to the LMDB (here "my-second-connect-host-name"). 

Further References

 

 

  • No labels