Tips for Fine Tuning Performance of the Webi Applet Interface (Java Report Panel)

Introduction

SAP BusinessObject Web Intelligence 4.0 introduces a number of improvements over the older XI 3.1 version. One of the major changes was to the Java based designer/viewer found within the BI Launchpad application.

This viewer was redesigned from the ground up and was even branded with a new name Rich Internet Applet.  By the time BI 4.1 came around, the viewer was renamed yet again to the Web Intelligence Applet Interface. In this article, we will refer to it as the RIA as this was first written for the BI 4.0 version.  The RIA, previously referred to as the Java Report Panel in XI 3.1, has taken on a new architecture in BI 4.0 and instead of being 1 single jar file (ThinCadenza.jar) it is now deployed as a bunch of jar file. This architecture change has added some extra work for the JRE in some cases and may occasionally result in slower initial load times.

The good news is that with BI 4.1 SP03 and above, the architecture was revert back to a single JAR file that is friendlier to some of the security changes that have been made in the Java Runtime. Environment. 

This wiki page will go into detail on some tips, tricks and known issues that can affect the performance and load time of the RIA designer.  For General Performance Related Tips and Tricks on Webi, please see this SCN Doc

DOC - Tips for Optimizing the Performance of Web Intelligence Documents

Ensure you have the latest Support Package or Patch installed

codebase_lookup causing unnecessary delays in load time of the applet

Some of the older versions of the Rich Internet Applet (RIA) did unnecessary codebase lookups which could add additional load time as the applet attempted to search needlessly for some classes or resources.
The following versions include a new parameter;

<APPLET ...>
<PARAM name="codebase_lookup" value="false">
</APPLET>

That reduces the time spent trying to look for classes in the classpath.  The RIA contains all the classes it requires so doing this lookup is unnecessary.

This was logged internally at SAP using the following #s

• ADAPT01674619
• BOEWEBI-9247

VERSIONS:

• * SBOP BI 4.0 + SP04 + Patch 4.9+
• * SBOP BI 4.0 + SP05 + Patch 5.2+
• * SBOP BI 4.0 + SP06+
• * SBOP BI 4.1

This issue is described in more detail in the following Knowledge Base Article:

1887358 - Performance of the Rich Internet Applet for Web Intelligence is slower in some versions of BI 4.0

BI 4.x Rich Internet Applet split into multiple (60+) jar files (Prior to BI 4.1 SP03)

The Rich Internet Applet (RIA), often referred to as the Java Report Panel, use to be a single jar file in older versions of BusinessObjects Enterprise (XIR2, XI31).  This meant that each time the applet loaded for a client, only 1 check would be done to validate that the applet was, or was not, available in the local JRE cache.  This also meant that only 1 check would be done for security measures such as Online Certificate Revocation.   The result was a quick loading applet as the overhead was quite a bit less.

Fast forward to the BI 4.0 world and the RIA architecture was modified.  In BI 4.0 the applet is split into over 60 single jar files.  This has some benefits from a code development and maintenance perspective but can also cause some performance degradation when mixed with some of the new security features enforced by Java by default with newer versions of the JRE. 

Starting with JRE 1.7 update 25+, the default certificate revocation check settings changed.  The latest versions of the JRE now enable Online Certificate Status Protocol checks as well as CRL checks.  This means that the client side JRE has to connect to external Verisign servers to validate the certificates for each of the 60+ jar files.  On a slow connection, this can add minutes to the load time.

KBA 1904873 - Web Intelligence Rich Internet Applet loads slower after installing Java 7 Update 25 (JRE 7u25+) and above talks about this in more detail

The good news is that starting with BI 4.1 SP03 and above, the RIA has been consolidated back into a single jar file.  This means that the load time is greatly enhanced for customers experiencing delays using this Online Certificate Status Protocol check or during the local cache checks as well.

KBA 1975294 - In Business Intelligence 4.1, when using the webi Rich intelligent applet, it takes a long time to open talks about this change as well 

Check to see if Online Certificate Status Protocol is enabled by default 

Starting around Java Runtime Environment 7 Update 25 (JRE 7u25) a change to the default security options of the client side JRE was made by Oracle.  In the latest versions of the JRE, an option called Online Certificate Status Protocol is enabled by default.

This OCSP setting enables the JRE to validate the certificates of JAR files using online sources such as verisign.com.  In some environments, these checks can take anywhere from 1/10 of a second per check to 5+ seconds per check.  Multiply that by 60+ jar files and you see a noticable delay when loading our Rich Internet Applet. 

I talk about this above as well but its worth repeating.  I've seen customers cursing because of 1-2 minute load times on their system and just simply disabling this one option can often take that down to 5-10 seconds.  The below KBA, also referenced above, talks about this more and shows you where this option can be modified.

1904873 - Web Intelligence Rich Internet Applet loads slower after installing Java 7 Update 25 (JRE 7u25+) and above

NOTE:  It is also important to ensure you a changing the right "bit" Java options.  Loading the Java Control Panel (JCP) from the Windows Control Panel will often load the 64-bit version of the JCP which is not the version we want to change in most cases.  Most browsers are 32-bit and will load a 32-bit JRE so be sure to make this change on the right JRE.  The easiest way to load the 32-bit JCP is to launch it directly from this directory:  c:\Program Files (x86)\Java\jre7\bin\javacpl.exe

Utilize client side Java Applet caching

Every time the RIA loads, the local Java Runtime Environment (JRE) will check to see if it has the required resources cached locally. It does this by comparing the locally cached versions with the versions found on the server you are connecting to.   Resources are cached by the URL, and therefore hostname, so in order for cached resources to be used, you need to ensure the URL remains the same.

Here is a typical cache scenario to help you understand how this works:

1. UserA logs in to BI Launchpad using URL http://biserver/BOE/BI
2. UserA Launches the Web Intelligence application
3. The Browser loads the local JRE in order to load the RIA
4. The JRE queries the server (‘biserver’ in this case) for the jar file versions and sees if it has a locally cached version that matches both the hostname and the version
   1. If the JRE has a locally cached version, it will load it from the local cache (fast)
   2. If the JRE does not have a locally cached version, it will download it from the server (slower)
5. Once all the jar files are loaded, the RIA will be displayed

Notes on this workflow:

• The first time user tries to use the RIA, will be slower.  This is because the RIA resources are being downloaded locally (30mb+ over the network)
• Subsequent loads will be much quicker if the JRE is configured to use cache as the jar files will be loaded from the local storage instead of getting downloaded over the network
• Subsequent loads within the same browser session will be quicker as the JRE will already have the jar files in memory
• First loads in a new browser session will be slightly slower, even with cache enabled, as the JRE needs to load the jar files in memory.

How to enable/verify java cache settings

1. Open the Java Control Panel
2. In the General tab, click on Settings… in the Temporary Internet Files section
3. Ensure the Keep temporary files on my computer option is checked on and that the Disk Space to store the temp files is adequate.
   
   
   
   
4. Once this is verified, you can check that the webiApplet jar files are cached by clicking the View files button on the General tab.
   
   
   
   
5. This displays a listing of all of the cached resources that the JRE has in its local cache.  Be sure to select Resources from the drop down list.  Note that the webiapplet.jar is cached for host localhost in the bottom screenshot.  You can also see version here.  This is just 1 of many resources that make up the RIA.
   
   
   
   
6. The JRE will cache resources based on URL, Version, Modified Timestamps and of course Resource name.  This means that even though you may have already downloaded the RIA from one environment, it will not be utilized for other environments.  In the above screenshot, you can see the cache contains a webi_applet.jar from “localhost”  In the below screenshot, you’ll see the cache also contains a cached webi_applet.jar from “bipw08r2“  as well.
   
   
   
   
7. In this case, they are both the same version but since I used different URLs to access the BI Launchpad, they are cached separately.  The first time I opened the RIA from either URL, it would download and populate the cache file.

How to tell if cached files are being used or not

If you want to validate that cache is being used instead of downloading the resources each and every time, you can do so quite easily by utilizing Java Console logging.  Below are the steps to use when troubleshooting JRE caching issues.

1. Close all of your browsers to ensure that the JRE is not loaded
2. Open the Java Control Panel
   1. In cases where only 1 JRE is installed, you can do this from Control Panel/Java
   2. In cases where multiple JREs are installed, you will want to execute the javacpl.exe from the <Install dir>/bin/ folder of your JRE.
3. In the Java Control Panel, go to the “Advanced” tab.  Expand “Debugging” and select “Enable tracing” and “Enable logging” – This enables a trace and a log right away when the JRE is loaded
4. Expand “Java console” and select “Show console” – This launches the console when the JRE is loaded (for debugging only)
5. Expand “Miscellaneous” and enable “Place Java icon in system tray” – This makes it easier to load the control panel and console later on via right-clicking the icon
   
   
   
   
6. Start up your browser and launch the Web Intelligence application from BI Launchpad
7. You will now see three things happen
   1. The Java Console will pop up right away
   2. Tracing and Logging will be enabled and you should see a slew of entries right away
   3. The Java icon will be in the system tray
      
      
      
      
8. Once the console is popped, you can see the trace entries around the loading of resources.  Below is what a typical entry looks like if cache files ARE being loaded
   
   
   
   
9. And here is what it looks like if the cache entries are not found and it has to download the resources from the server
   
   
   
   As you may be able to tell, we download zipped resources to save on network traffic.  Since the cache entry for webi_applet.jar version 14.0.6.1036 was not found on this client machine, the JRE requested that the resource be sent down to the client (zipped).

Use the Java Next-Generation Plug-in

This is perhaps the most important thing you can verify on your client side.  Some companies may be unchecking this option in the JRE settings which can result in poor performance and a variety of client side errors that can plague your BI deployment.

The Next-Generation (NG) Plug-in is described in detail in the following articles:

http://www.oracle.com/technetwork/java/javase/plugin2-142482.html
http://www.java.com/en/download/help/new_plugin.xml

The bottom line is that it is extremely beneficial to use this plugin.  For Web Intelligence, this is particularly important for the following reasons:

1. The Webi Applet increases the Maximum Heap Size of the JRE to 512m when the NG is used.  Without this plugin, the default is just 96m which will be insufficient to load large reports. 
   1. Here are the system properties of the JRE without the Next Generation Plug-in running
      
      
      
      
   2. Here are the options with the Next Generation Plug-in running.
      
      
      
      
2. The Webi Applet increases the Thread Stack Size to 1024k when the NG is used.  This is required for multiple reasons and will remove the risk of “StackOverFlow” errors in the JRE.  Webi utilizes the stack to keep track of report changes, for things like undo/redo functionality, and to store the states needed for backup and recovery.   The default stack size is just 320k without the NG plugin.
3. The NG plugin allows Java.exe to run as a sub process of the browser session instead of as a DLL loaded in the browse.  It will spawn a separate java.exe for each Webi Report you open as well.  This means that if one of them crashes or has resource issues, it will not take down all of your sessions.  It also means that each Webi report loaded has its own pool of resources (Heap size and thread stack)
   1. Here is the Process Explorer view when the Next Generation Plug-in is utilized.  Note that a jp2launcher.exe and java.exe subprocess is spawned below the iexplore.exe.
      
      
      
      
   2. Here is what it looks like without using the NG Plug-in.  Note how no sub-processes are spawned.   This means that the entire browser session is sharing the same, lower, resource pool (96mb heap, 320k thread stack size)
      
      
      
      

How to enable the Java Next-Generation plugin

The steps below are outlined in detail on java.com here:  http://java.com/en/download/help/new_plugin.xml

1. Launch Windows Control Panel on the client machine
2. Click on the Java icon to launch the Java Control Panel
3. Go to the Advanced tab in the Java Control Panel and expand Java Plug-in

4. Make sure that Enable the next-generation Java Plugi-in option is selected
   
   
   
   This option may be greyed out or disabled.  In this scenario, you will need to toggle it on or off through the registry.  Follow these steps to change it from the registry:

   Modifying your registry can cause irreversible damage if done incorrectly.  Follow the below steps at your own risk and always make backup copies of registry keys before modifying them.

   
   
   

   1. Go to Start/Run and type in “regedit.exe” in the Run dialog box.  Click OK
   2. Navigate to the following area in the registry (32-bit Java keys)
      
      HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\JavaSoft\Java Plug-in<version>
      
      
   3. Find or create the registry key UseNewJavaPlugin (REG_DWORD)
   4. Ensure this key is set to value 1
   5. Close the registry and restart your browser sessions for the setting to take affect

Known Issues that can arise when not utilizing the JRE Next Generation plugin

This can cause a variety of issues.  Here are a few I was able to replicate in my testing:

Issue 1: java.lang.OutOfMemoryError: OutOfMemoryError

Issue 2: Exception in thread "AWT-EventQueue-2" java.lang.StackOverflowError_*

Known Performance issues with the Java Runtime Environment (JRE)

2260485 -  Web Intelligence applet takes much longer to launch with Java Runtime Environment 8 installed 

1904873 -  Web Intelligence Rich Internet Applet loads slower after installing Java 7 Update 25 (JRE 7u25+) and above

2134823 - Slow performance when launching the Web Intelligence JAVA Applet Interface 

General RIA/JRE Troubleshooting Tips

Below are some helpful tips that should assist in root cause analysis of RIA performance issues

Load the Web Intelligence application on its own to gauge the true applet load times. 

If you are experience slow load times only when modifying existing Web Intelligence documents, then you are likely seeing a performance issue with opening the document and not with loading the applet.

Try loading the Rich Internet Application on its own without modifying a report and see how long it takes.  Below are the steps I used to help gauge load time on my BI 4.0 system:
Workflow 1 – Determine the initial (1^st) load time

1. Close all Internet Explorer browsers
2. Launch a new IE browser and login to BI Launchpad
3. Go to the Applications drop down in the top right corner and select Web Intelligence to launch the Rich Internet Applet (assuming your preferences are set to do this by default)

4. Make note of the time it takes from the beginning of step 3 to when the designer loads

   The first time you load the RIA will always be a little bit slower.  Even if you are using cached jar files.  This is because the browser has to load the JRE as well as the applet and this adds time on to the launch.

   
   
   

Workflow 2 – Determine the subsequent load times for the same users

1. Without closing your browser, close the Rich Internet Applet sub-tab from BI Launchpad.  This should take you back to your home page or the last location you were in.

2. Launch the Web Intelligence application again and make note of the time it takes to load now.

   This should be quicker as the JRE will be loaded in the browser now and the jar files will almost certainly be loaded from cache

   
   Run through Workflow 2 a few times to get the average load time of the Rich Internet Applet itself.  In my experience, this should take between 10-20 seconds on average and can be as quick as 2-3 seconds on a fast network.   Keep in mind that the JRE still has to check the server versions of all of the required jar files to ensure that any cached versions are not out of date.  This means that slower networks and Wide Area Network (WAN) deployments will be slower.
   Once you have an average load time of the designer itself, test out modifying an existing report

Workflow 3 – Determining the load time of a report
Next we want to check the load time of a report itself.  Different reports will have different load times as there are many variables that can add on to the load time (# of objects, # of report tabs, visualizations, size of saved data, size of WID file, network bandwidth)
Here are the steps we can use to check the average load time of a report:

1. Login in to BI Launchpad
2. Navigate to the Webi report you wish to check the load time of
3. Right click on the Webi report and select “Modify”
4. Assuming you have the Rich Internet Applet configured as your default editor, this will launch the RIA.  Make note of the start time and the end time of the report and designer loading

5. We can then subtract the average load times from Workflow 2 to get an idea of the load time of this report.

   If this is the first time you are loading the RIA in this browser there will be additional time spent on loading the JRE and the jar files into memory.

6. Close the RIA and modify this report again.   Repeat this a few times to get the average load time for this report.   Subsequent load times will be faster as cache will be used, the JRE will be loaded and connections to required backend servers will be established

Test on the server itself or machines closer to the Application Server tier

Most customers deploy 1 or 2 application servers in the same location and will use a load balancer of some sort to distribute load across these.  If these servers are located in one site but are utilized for a Global BI landscape, there may be performance issue for regions with slower access to the application server tier.
A great example of this is a company that has Application Servers that are located in the United States and end users that access it overseas.   We often see that users that are co-located with the application servers will see better performance overall.
By testing with co-located user or on the actual Application Server itself (through remote desktop or something like this) we can see if the performance bottleneck is with the end-user, the network or the application server.

Tests to perform when trying to identify bottlenecks for the RIA

Here are some tests you can do to help pin point the source of performance bottlenecks for a global deployment:

1. Test on the application server itself
   1. This eliminates network as it is the quickest possible method of accessing the application server
2. Test from a client machine that is co-located with the application server
   1. This eliminates Wide Area Network (WAN) issues that might occur only for other sites
3. Bypass any load balancers, web servers, proxies, etc…  by hitting the <appserver>:<port>/BOE/BI directly
   1. This eliminates variables from the workflow and if you see a difference when bypassing these other tiers, it may mean that there is a configuration or deployment issue that needs addressing
4. Test different Web Browsers (Internet Explorer, Firefox, Chrome, Safari)
   1. This can tell us if it might be a browser setting or incompatibility that needs addressing and analysis
5. Try different versions of the Java Runtime Environment (32-bit)
   1. Some versions of Java are quicker than others.  Trying a different version of the JRE or comparing the versions between a client that is quick and one that is slow may help in finding root cause
6. Use Fiddler or other 3-rd party utilities to try and isolate the bottleneck
   1. Fiddler can help determine where the bottleneck may be occurring.  For example, we used it to discover that the RIA was trying to load resources from the entire classpath which was resulting in some 404 errors and additional load time for some users (fixed in versions mentioned at the top of this article)
7. Make sure logging/tracing is disabled when doing performance testing (BI 4.0, JRE, Browser, etc…)
   1. Tracing will reduce performance.  When doing benchmarking, it is best to disable tracing and logging to get your real times. 

Things to consider when evaluating performance

• There will be additional time added the first time the JRE loads in the Web Browser
• The network connectivity between the client machine and the application server will impact the speed of the load time of the RIA
• Accessing the RIA over a VPN can greatly increase the time it takes to load the RIA
• Accessing the RIA through a proxy server or load balancer can increase the time taken to load the RIA
• Some security settings for the JRE can increase the load time of the RIA (e.g. Enable online certificate validation)

In Closing.

Analyzing performance issues is a complex procedure that requires a deep understanding of the underlying technologies and workflows. The key to identifying and resolving a performance issue is to narrow it down to the source of the issue.

If you have any additional tips or tricks, please feel free to add to this page or send them to me and I will be happy to make the updates.