Registration

Dear SAP Community Member,
In order to fully benefit from what the SAP Community has to offer, please register at:
http://scn.sap.com
Thank you,
The SAP Community team.
Skip to end of metadata
Go to start of metadata

Purpose

The purpose of this page is to clarify the understanding of the Auditing feature and detailed workflow.

Overview

Auditing allows you to keep a record of significant events on servers and applications, which helps give you a picture of what information is being accessed, how it's being accessed and changed, and who is performing these operations. This information is recorded in a database called the Auditing Data Store (ADS). Once the data is in the ADS, you can design custom reports to suit your needs.

Background of SIA and CMS Initialization.

When the SIA is started:

  1. It will first check all CMS’s in the cluster to see if they are running.
  2. If one is running, it will retrieve the list of servers and their command lines (including the local CMS if there is one) and start them.
  3. If no CMS is running, the SIA will start the local CMS.  When the CMS is initialized the SIA will retrieve the list of servers linked to it, along with their command lines and then start them (except for the CMS which is already running).

When CMS is Started:
  1. During CMS initialization, it will create multiple connections to the CMS database (14 by default)
  2. In contrast, it will only make one connection with the Audit Database.
  3. It will then bind to the port and request ports defined (or a random request port if it is not configured)

Auditing background operation:

1. Every time the CMS start and when it makes the connection to the audit database, it by default enters/checks for all the default tables and indexes.
2. The lookup tables are the standard tables which have generally static data in it. (e.g. Standard Client IDs for Webi, list of all the servers in the cluster, lookups to match the event type id with a string (e.g. 1006 = Delete)
3. It checks and the tables every time the CMS starts.
4. By default, this is how the tables look like for Auditing (All except for COMMENTARY_MASTER which is for the Commentary Service)

Audit Process

Only one CMS will write audit events to the Audit database. This is called the Auditor – the current auditor can be seen in CMC -> Auditing (see below)

    • A BO server that is generating the events is an auditee – all BO servers are auditees
    • A CMS which is an auditor is both the auditor and an auditee
    • If this CMS is shut down or loses connection to the Audit database, another CMS will take over the auditor role.
    • The Audit database connection information, retention period and types of event to be audited, plus extra required detail – such as SQL statements for report refresh – is defined in the CMC->Auditing page
    • Based on the events which have been selected, the servers responsible for that event capture the events in the form of .txt file in the Auditing directory on their local machine.

    • By default the path for Auditing directory, where intermediate auditing files are stored, would be: C:\Program Files (x86)\SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\Auditing.
    • The naming convention is the form of: audit<EncodedServerName><Event_ID> (in hexadecimal form) therefore all files created by the same BO server will always start with the same set of characters. Below is an example of file name generated by auditing (Note that the first 5 files all start with the same string so they are generated by the same BI server).

 To check which CMS in a BI cluster is an auditor, we can check the details from the CMC -> Auditing Page

Workflow in Detail:

  1. Each individual server will write its own Auditing information into a temporary file on its local machine (these are what we find in the Auditing folder on each machine.
  2. The Auditor CMS maintains a list of all the registered (running) servers.
  3. This Auditor CMS sends request to the first registered server in the list: Send me any events that are currently waiting to be written to the Auditing Database.
  4. The server acknowledges the request and then sends all the audit events and details which need to be written (the auditee will read back the data in the auditing files it previously wrote in the Auditing folder)
  5. The Servers for ex: Webi processing server, looks for each event from the .txt files and then sends the details to the CMS in a structured format. All servers will send any event to the CMS using the same ‘standard’ structure.
  6. Auditor CMS then processes each of the events returned in order.
    1. If the event already exists in the auditing DB, the event is skipped and the auditor moves onto the next event.
    2. The Auditor checks if the object related to the event (e.g. the refresh Webi report or the user that is logging in) still exists in the CMS repository (or it’s been deleted already).
    3. If the object if found in the CMS repository, the data is used to turn the CUID of the object (which is all that is stored when the audit event is written to the Auditing folder) into the real object name and folder path within BO.
    4. As there is always a delay between the audit event occurring and it being pushed to the audit database there is always the possibility that the referenced object no longer exists and this information connect be found.
    5. In the case the object doesn’t exist in the CMS repository and longer (it’s been deleted since this event was created) then either the delete event will have been inserted or it wont have been yet (but will shortly).
      1. The Auditor will check the audit DB to see if an delete event exists in the database – if it does it copies the name and folder path from the delete audit event in the DB and uses that.
      2. If it doesn’t (yet) exist in the audit DB it will write the event id and the CUID of the object in the INCOMPLETE_EVENT table in the audit DB
      3. When the delete event does come in it checks to see if any events are waiting of it (i.e. by looking at entries in the INCOMPLETE_EVENT table for any events with the same CUID) and will then fill in the missing events details before inserting the delete event.

For example:
1. If the report named (abc) has a View event. It will check if the report ABC (Object) is present in the CMS DB and then take actions accordingly.
2. Once it finds the object (report), it will write the details in the Audit DB with the associated event.
3. Once the details are written in the Audit DB, the CMS will request the servers to delete the .txt file it had audited.

Once every 3 minutes the CMS polls for the data from each servers in turn. This is one Polling Cycle.

The thread utilization shown in the CMC->Auditing page is: 

        • Time taken to perform the auditing cycle * 100 / Last Polling Cycle Duration = Auditing Thread Utilization (%)
        • ‘Last Polling Cycle Duration’ is 180 seconds or the time taken to perform the auditing cycle (whichever is greater). This means the Auditing Thread Utilization (%) can never be > 100%

Note that the Auditing Thread Utilization is NOT related to the CPU utilization of the Auditing thread – it is just related to the amount of TIME it takes

        • When we see 100 % thread utilization on the Auditing page, it means: that CMS has taken 3 minutes or more to complete the previous cycle. (The issue could be anywhere, getting details from Servers, sending the details to the Audit DB, or CMS waiting for any operations) – e.g. large number of events to import, slow Audit database access (either read or write), network performance etc.
        • If there is a problem process the auditing events from one of the servers the auditor will then move onto the next server (it will not delete the files in the Auditing folder event if it inserted some of them).
        • If there is no data to be written and no audit events triggered since the last polling cycle, it still prompts the server to share the data to written. And the server responds with no data to write.

Check below for the INCOMPLETE_EVENT:


        • Next time when the CMS gets a Delete Event.
        • CMS will check in INCOMPLETE_EVENT Table to see if there is any incomplete event been stored.
        • And then write it to the DB.
        • Later it deletes the entry for INCOMPLETE_EVENT

TRACING

  1. Set the Tracing level to Medium
  2. Ideally we can capture the details for CMS for auditing on the Medium priority
  3. Change the keep_num parameter to 50 or more. From <install_dir>SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\conf\BO_trace.ini
  4. Once we have the logs, change the tracing level back to unspecified and the keep_num back to default value.
  • No labels

2 Comments

  1. In your process you state that the audit files are deleted from the audit directory after they have been written to the Audit database.  At least as of BI4.1sp7, this doesn't happen.  The data is removed from the files but the files are left in the audit directory.  I assume this is a bug, but it's been this way for a while.  This results in a lot of empty text files being left in the Audit directory.

  2. Former Member

    Hi John,

    Once the contents from the files are pushed to the Audit Database, the file should be deleted from the Auditing directory.

    The files with 0kb size or no data in it, should not create any problem as the events are already pushed and the files play no role later.

    There might be some issue with network or with the server responsible to generate the .txt file in the Auditing Directory, causing the sequence to break.

    Regards,
    Rajdeep