Disclaimer
According to the General Terms and Conditions for SAP Cloud Services it is prohibited to perform any action on the cloud service which circumvent or endanger its operation or security.
Please note, performance testing might have a serious affect on private instances as well as on shared instances where the SAP Analytics Cloud tenant resides with multiple customer tenants on same instance. This is important because services being used on a certain system are shared among other customers. Due to that reason performing stress/performance testing is only allowed on private SAP Analytics Cloud edition tenant.
- When in doubt, reach out to your Account Team (Account Executive) to clarify whether you are allowed to do the performance test or not.
- Understand that performance/specification of non production environments may not be the same as production environments. UAT performance testing cannot be performed on non production systems.
- As this is considered as a support operation please notify SAP (through your Account Team) at least 15 business days in advance the test so we can monitor the systems to avoid any outage.
In order to narrow down a certain performance issue to understand the root cause the below load test example might help to diagnose the Live Data Connection flows.
A general stress test might create a load, that could endanger the operation of the cloud service.
Disclaimer end
Foreword
I often get requests from customers for stressing SAP Analytics Cloud stories based on live data connections. For some of our customers, I had the opportunity to design simple stress tests to load network configuration and especially live connection performance.
Note: Use the new SAP Analytics Cloud Performance Benchmark tool to measure your client hardware score and network speed.
Such tests only intend to check if a lack of performance is due to network issues between a business user's workstation and backend datasources.
It has also been the opportunity to deeply understand how authentication and live connections work in SAP Analytics Cloud.
I could have used some other stress test tools to load and test functional behaviour and get measures (ie. Loadrunner). I only need to stress from a Business User Perspective and not in the perspective of large deployment which requires a real large scale test load.
I used jMeter which is an open source software, 100% pure Java application designed to test performance and simulate virtual users.
It was originally designed for testing web applications but has since been expanded to other test functions.
This software can be installed on any server or workstation to quickly design load test scripts.
Apache JMeter may be used to test performance both on static and dynamic resources, Web dynamic applications.
It can be used to simulate a heavy load on a server, group of servers, network or object to test its strength or to analyze overall performance under different load types.
JMeter does not run Javascript. So test script has to manage by itself all redirections based on SSO sequences and user scenario sequences.
We do not want to test workstation but network, backend and live connection performances.
This document intends to share my experience designing jMeter test script in this context.
Test Plan Scenario
My first activity has been to identify the best scenario and best stories to test. With customer we decided to test the following :
- User Authentication thru SAML2/SSO with custom Identity Provider (SAP Cloud Platform Identity Authentication Services)
- SSO with SAP Analytics Cloud
- SSO with SAP NetWeaver BW
- One SAP Analytics Cloud story with two models and two BW Live Connections Queries.
User Authentication Scenario (SAP Analytics Cloud and SAP NetWeaver BW through Live Connection)
The following diagram describes the Authentication Scenario based on SAML 2 SSO. It is composed by 4 main components :
- Browser: jMeter simulates Browser (Except Javascript execution which is not supported by jMeter).
- Identity Provider: support SAML2 Identity Federation and Authentication services. I use SAP Cloud Platform Authentication Services
- SAP Analytics Cloud as Service Provider
- SAP Analytics Cloud Tenant URL
- SAP Analytics Cloud SAML 2 endpoint
- SAP NetWeaver BW as Service Provider
- SAP NetWeaver INA protocole endpoint
- SAP NetWeaver SAML 2 endpoint
SAP NetWeaver BW Live Connection Scenario
- SAP NetWeaver BW Live Connection is based on CORS.
- Our story contains two charts with two different queries. In browser, Queries are asynchronous and runs in parallel.
Unfortunately, with jMeter queries run sequentially.
GetResponse request URL are dynamically composed by dynamic session ID. This dynamic session ID avoids any Denied of Service detection if your have a high number of GetResponse queries in the same story. So it is mandatory to extract sap-url-session-id from Response Header in step 2 above, to compose request URL of step 3,4,5 and 6 as below :
/${sap-url-session-id}/bw/ina/GetResponse?sap-client=001&sap-language=en&sap-sessionviaurl=XCommunication is also protected against Cross Forgery Attack with dynamic CSRF Token. So it is also mandatory to extract x-csrf-token from Response Header in step 2, to be injected in Header of step 4,5 and 6 above.
Obviously, do not forget to set your mysapsso2 string if required (we speak later about it in this document)
Architecture
Designing jMeter test plan script
Due to jMeter restriction, to design test script you have to understand each step of Identity Federation SAML 2 authentication process explained above.
You can start from scratch by opening a blank test plan or you can start by recording. I recommend recording even if jMeter recording tools does not catch all transactions, but 50% of the job is done.
Nevertheless, recording will give you most of Header description, Query definitions, but you will have to manage by yourself Javascript redirection.
General Settings of test plan
First, at the top of your script insert the following controlers:
- User Defined Variables to record general variables such as SAP Analytics Cloud URL & Path, Identity Provider Hostname and Path, SAP NetWeaver BW hostname and path, mysapsso2 string..., if you want to use your script for other occasions.
- HTTP Request Defaults to record general Header items such as (I am on Mac):
| User-Agent | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36 |
- HTTP Cookie Manager to enable automatic cookie recording and set Cookies policy (I chose Netscape)
SAP Analytics Cloud Authentication Procedure
| Initial SAP Analytics Cloud Tenant Get request Extract required information to redirect to SAML2 SCP tenantId, idpName, requestUrl, ... Use these extracted fields to feed parameters of next transactions:
Redirect to auth.hana.ondemand.com SAML2 SCP endpoint Extract SAMLRequest, RelayState Redirect to Identity Provider SAML2 endpoint (SAP CLoud Identity Authentication Service) Extract SAMLResponse, etc. Post User credential to Identity Provider Authentication Service Post SAML2 Response to SAML2 SCP endpoint Post SAML2 Response to SAP Analytics Cloud |
|---|
BW Live Connection Authentication Procedure
Prerequisite : jMeter cannot manage cookies in memory, so to implement test, you have two options to manage mysapsso2 cookie:
- Get mysapsso2 cookie string from Chrome Browser developer view (Recommended) and set a jMeter variable.
- Disable mysapsso2 cookie in Netweaver Instance Profile. To temporary disable mysapsso2 cookie, set login/accept_sso2_ticket to 0 in Instance profile of SAP Netweaver BW.
| Initial INA GetServerInfo request User is already authenticated thru SAC, so SAML Response is sent back and is extracted with Boundary Extract and Regex Extract tools. Post SAMLResponse to BW SAML2 endpoint Post SAMLResponse to BW INA endpoint |
|---|
Run BW Live Connection queries based on story
| Set Timestamp variable to be used in request GetServerInfo Preflight request (CORS) GetServerinfo Metadata request GestResponse Preflight request (CORS) Post GetResponse to get Metadata Post GetResponse for Query 1: Payload request contains JSON description of query
Post GetResponse for Query 2: Payload request contains JSON description of query. I used jMeter recording to get JSON descriptions in Payload request. You can also use Chrome/Developer view to extract JSON definition. |
|---|
Run you test plan
First you have to set Thread properties:
In this example, we simulate 100 users, rampup period will be every 10s and forever.
Sometime number of concurrent users has to be increased in SAP NetWeaver BW in transaction RZ11 on parameter rdisp/max_alt_modes.
and your summary report :
You can also run you test plan by using command line outside jMeter GUI as follow:
jmeter -n -t jmeter/StressTest.jmx -l gen-report.jtl
The -l option enables recording of all test information in JTL file to be later processed in jMeter/Tools/Generate HTML Report which produces great dashboards as follow:
