Page tree
Skip to end of metadata
Go to start of metadata

Blog

This article is introduced by a blog providing an overview and summary

Download

Preview or Download this presentation (version 1.0.2)

Video

Preview or Download this as a webinar



  • This article is suitable for

    • Developers or system architects that need to provision and/or manage users within SAP Analytics Cloud via an API

    • Project members who need to implement a solution that in turn uses the SAP Analytics Cloud SCIM API

  • This article provides

    • An understanding for how the API works, behaves and performs

  • Enabling you to

    • Dramatically reduce development time and effort

    • Adopt best practices and avoid known gotcha’s

    • Determine the right strategy for integration with existing systems

    • Avoid surprises, especially as user volumes increase

    • Understand aspects of how a Product or Service that uses the SAC SCIM API should be configured

Introduction to the SAP Analytics Cloud SCIM API





Common use-cases

  • Common use-cases for developing a solution for this API
  • Need to build integration with a Provisioning Solution or a custom solution to
  • Create users
  • Manage team membership
  • Delete users, for example ones that haven’t used the Service for a while
  • Using an API for mass user management can be more efficient and less error prone
  • Creating users can be automated, even when no provisioning solution is used by enabling ‘automatic user creation’ as part of the SAML Single-Sign-On. However there are times users need to be created before they login:
    • For example, to receive a publication, (though external recipients are supported, it may be more desirable to have them as registered users within the service)
  • Synchronising users’ team membership with a Provisioning Solution
    • as something caused the two systems to be out of sync with each other
  • Examples of solutions that pre-integration with SAP Analytics Cloud:

Key facts, official links and scope

Scope of this article

  • Focus to on Users and Groups (roles and teams)

Official links


Key facts

  • It is a non-blocking API
    • However, errors are expected if you attempt to update the same objects simultaneously
    • For example, updating a Team membership in the User Interface and via the API at the same time could result in:
      Response: {"status":500,"message":"Error executing the user/group operation."}
      Re-submit the request to resolve
    • Parallel operations are allowed
  • It is a synchronous API and has a 5 minute timeout

SCIM API Overview


SCIM ‘Groups’ are SAP Analytics Cloud ‘Teams’ and ‘Roles’, but you can only create, delete and update Teams, not Roles.

Roles hold: Teams and Users

Teams hold: Users

Session management

Theory

Tokens

  • accesstoken
    • Obtained using username/password of OAuth client
    • expiry will result in 401 response status ‘unauthorised’
  • x-csrf-token
    • To use any non-GET request, the x-csrf-token must be present in the header (if not a 403 will be returned)
    • It can only be obtained by performing a ‘fetch’ during a GET request
    • x-csrf-token is only valid inside its accesstoken session
    • Expiry will result in 403 response status ‘unauthorised’
  • Means
  • A new accesstoken will require a new x-csrf-token


Best practice

Tokens

  • accesstoken
    • Whilst a new session could be established for each ‘event’ (like creating a user), doing so will be costly when processing lots of ‘events’
      Re-use the session across all ‘events’ and only obtain a new session when the current one expires
  • x-csrf-token
    • Whilst a ‘fetch’ could be requested in every GET request, this delays the request by ~0.5 seconds
    • This could amount to a great deal of time when processing thousands of requests
    • Only ‘fetch’ the token once per session on the first GET request of the new session
    • If the next request isn’t a GET, then perform a separate GET to obtain the token
    • Avoid this extra GET whenever possible
  • Dependency issues
    • There is a dependency between the two endpoints
    • Failure of an endpoint could result in an endless loop (exceptionally rare!)


Practical examples

  • Example 1
    • GET request returns 401
    • Get a new access token
    • On first GET (which is a repeat of the request that previously failed) fetch the x-csrf-token
    • Continue requests of any method, re-presenting the x-csrf-token

  • Example 2
    • GET returns a valid 200, but the session expires just after the GET and before the following PUT
    • PUT request returns a 401 as the session has expired
    • Get a new access token
    • Need to issue an ‘extra’ GET to fetch the x-csrf-token
    • Re-submit the PUT that previously failed

Prerequisites

Prerequisites for using Teams

Must respect:

  • All teams must be created via the API *
  • Teams are created with a Team Folder
  • Teams can’t be created if the team folder already exists
  • Means:
    • For any team that has been created in the user interface, the API cannot work with it. It means if you’ve created any team through the user interface of SAP Analytics Cloud you can’t use the API to update them. This in turn, means you may need to delete all or some of those existing teams if you want to use the API to manage them.
    • Once a team has been created through the API the user interface can still be used to manage those teams in the same way as a team created manually via the user interface. You can mix the management of the team through both the API and the user interface as you please.
    • Need to manually delete the Team folder, assuming its not needed
    • Need to be careful not to transport a team via the Content Network, or when doing so understand two teams will exist of the same name. You’ll need to delete one of the teams leaving any team folder remaining

  • * applicable for non-SAP data centres. KBA 2857395 provides a workaround


Prerequisites for using Roles

Must respect:

  • Roles must pre-exist
  • API can’t yet create or update roles, only read who is a member of a role
  • Means:
    • if you want to add a user or a team to be a member of a role, those roles must pre-exist

Creating users


Recap of existing best practice for user to role/team membership

  • It’s best practice to
    • add users to be members of teams (green line)
    • where the teams are members of roles (green line)
    • rather than assign a user ‘directly’ as a member of roles (red line)
  • Because
    • access permissions on folders is specified by teams, not roles
      • (also possible on individual objects and also by individual users)
    • a single teams aggregates multiple roles
      • a single team thus forms a ‘Business Role’ definition like ‘Power User’
    • changing the ‘Business Role’ (a team) requires only updating the team, not all the individual users


POST request


SAML attribute mapping

  • userName and id
    • These two properties are the same except when using SAML attribute mapping
    • userName must be the SAML attribute used to map the user
      • For example, if you’re mapping on the email address, then userName must be the same as the emails value

Properties used only at user creation, then unchangeable

  • userName and id cannot be changed once the user has been created
  • givenName, familyName and displayName can only be updated by the identity provider

Roles and Teams

  • Roles (SAP Analytics Cloud Roles) can be specified, unlike any Teams
  • Team membership for a user is performed on the /Groups endpoint

Create users with the correct settings to avoid unnecessary updates later

  • Pay particular attention to:
    • a) Preferred language
    • b) Data access language
    • c) Date formatting
    • d) Time formatting
    • e) Number formatting
    • f) Clean-up notification number of days
    • g) System notification opt-in
    • h) Marketing email opt-in
  • Reference user parameters link


Multiple endpoints needed

  • May need multiple end-points to create a single user
    1. Create the user
    2. Add the user to Teams
  • If 1 user needs to be in 3 teams, then an update to each of the 3 teams is required
  • Each team, requires a GET/PUT pair
    • More on teams later

Workflow /Users


User creation will fail with a 409 conflict when

  • The userName or email is already in use by the same or different user

User creation will still succeed with a 201 when

  • The username is already in use, but has a different email
    • Result will be a ‘duplicate’ user with ‘_1’ added to the userName
    • Reading the responseBody.userName can test for this condition

Means

  • Creating a user and repeating the same request will return a 409. i.e. its safe to just ‘repeat’ a create user request unless you rely on the 409 response to determine future workflow (see later)
    Either
    • Only issue create user requests (POST /Users) when you’re sure the email hasn’t changed
    • Test for the duplicate user and delete it before updating the right user
  • Need to be careful about adapting this create workflow to include updating users’ email address


Workflow /Groups

Team updates require a pair of GET/PUT requests

  • GET is required to read all the team properties
  • PUT replaces all team properties
    • Therefore need to make necessary changes between the GET and the PUT to update the request body
  • Need a GET/PUT pair per team
  • To add 1 user into 3 teams, will require 3 GET/PUT request pairs

Best practice

Batch users together before updating teams

  • Team updates, although require a GET/PUT pair allow for multiple users to be specified
  • Batching requests together for Team updates will dramatically reduce # of requests and increase throughput


  • Examples 1 and 2 both create 2 users and each user is a member of Teams 1, 2 and 3
  • Example 2 ‘batches’ the request together, so there’s only an update per team,rather than an update per user per team
  • More users the greater the difference

Updating users

Basics


Most changing properties of a user:

  • Team membership

Next most changing properties are:

  • managerId
  • isConcurrent
  • email value

Other properties:

  • Are in the users control and can be changed by them via their profile
  • Roles assigned directly to a user isn’t encouraged, better to assign users to teams, and assign the teams to roles

Updating user:

  • GET/PUT request pair
  • Requires logic on the client-side to make changes as appropriate
  • The PUT contains the entire new user definition
  • The PUT response is similar to the GET response, in that it returns the user definition
  • Updating team membership cannot currently be performed on the /Users endpoint
    • Need to use the /Groups endpoint
  • The roles will return an array of length 1 with an empty value when no roles are assigned to the user

Multiple endpoints

  • Multiple end-points needed
  • Updating a user is a GET/PUT pair operation:
    1. Read a user
    2. Optionally update the user
  • Updating team membership
    • Reading the user does show which teams the user is a member of
    • Allows for optional updates to each team
    • Each team requires a GET/PUT pair operation

2 endpoints

  • /Users endpoint for updating non-team membership
  • Updates to a user is a GET/PUT pair on each user
  • The GET/PUT response does include the array of teams the users is a member of
    • Could be handy to determine if any team update is needed

  • /Groups endpoint for updating team membership
  • Updates to a team is a GET/PUT pair of the team

The PUT is optional

  • No need to perform the PUT if the users’ properties don’t need to be updated
  • Avoid the PUT if possible
  • Same applies for any team update
    • Team update may be unnecessary

Best practice

Batch users together before updating teams

  • Team updates, although require a GET/PUT pair allow for multiple users to be specified
  • Batching requests together for Team updates will dramatically reduce # of requests and increase throughput
  • Examples 1 and 2 both update 2 users and each user is removed from 3 teams and added into 3 different teams
  • Example 2 ‘batches’ the request together, so there’s only an update per team,rather than an update per user per team
  • More users and teams the greater the difference

Combining create and update workflows

Creating workflow or updating workflow?

Combining workflow options

Either start with the ‘Creating user’ workflow or the ‘Updating user’ workflow

Each have their pros and cons


Creating workflow part 1


Creating user workflow as the basis:

  • If a 409 (conflict) response is returned following a POST (to create the user), then:
    • The user either already exists (with the same email)
    • Or the email is in use by another user
  • Assuming, the email isn’t in use by another user, then just perform a GET/PUT on the user to update the user
  • If a 404 (not found) response is returned from the GET, then the email must be in use by another user
  • However, the duplicate user ‘_1’ would still be generated if the POST (to create the user) used a different, updated, email address for the user
    • This is addressed in part 2…

Creating workflow part 2

Creating user workflow as the basis:

  • When a duplicate user is detected, we can DELETE the user, only to then update the correct users’ email address
    • If the default Identity Provider (IdP) is used then a small drawback is the user will receive an email notification welcoming them to SAP Analytics Cloud, just because you’re updated their email address
    • The user will receive a welcome email to an account that will only exist for a brief moment
    • Email notifications are not sent if a custom IdP is used
  • This workflow is further enhanced in the ‘Managing errors’ section of this article
    • (The POST /Users can be problematic upon other errors)

Updating workflow

Updating user workflow as the basis:

  • If a 404 (not found) response is obtained from the GET, then the user doesn’t exist so we should create it
  • Given we’ve already read the user, the POST to create the user will only return a 409 (conflict) if the email is in use by another user
    • Probably, very unlikely!


Best practices


Updating user workflow as the basis:

  • Greatest throughput with the ‘Create workflow’ when at least 52% of the users are being created, otherwise use ‘Update workflow’
    • If updating email, then need at least 77% of the users are being created users to use the ‘create workflow’
  • Don’t use either if you’re just updating the users’ team membership!
    • Which is probably the most common property that needs to be updated on-mass


Combining workflows to obtain predictable User IDs

To obtain predicable userName when using SAML email

  • When creating the user the userName must match that of the email
  • The userName (and the path /api/v1/scim/Users/{userId}) allocated will be based on the email up to the ‘@’, and certain characters removed
    • For example first.last@myorg.org will be firstlast
    • A long email will truncate the username as userName has max 20 characters
  • But there’s often a need to have the userName to be more predicable and known by other systems
  • As a workaround (as suggested by an SAP community member):
  • A) create the user, but with the wrong email! userid@...
  • This will allow SAC to allocate the userName “MATTHEW1”
  • B) read the user to obtain a list of default roles added to the user after the user was created (we don’t want to drop them when we do C)
  • C) update the user with the ‘right’ email
    • Now the user has been created, we can change the email(unlike userName which we can’t change)
  • We had to create the user with the wrong email, just so the userName becomes what we want it to be, not what SAC tells us it should be! Then, we update the email, keeping the userName untouched
  • A drawback of using this workflow, to update a user, will always create a new user that will need to be deleted, so the ‘update-create’ workflow is then needed to ensure highest throughput
  • (Sample scripts 13x supports this workflow)

  • The ‘update-create’ workflow for this becomes as shown
  • Possible ‘gotcha’ with these workflows is any default role could change the ‘Business Intelligence License Type’ and the GET/PUT pair could undo this if you test or change that property
  • (Sample scripts 23x support workflow)

Team updates

Overview


Updating teams:

  • Always a GET/PUT request pair
    • GET reads the current membership
    • PUT writes back the new membership
  • There’s currently no API to perform a PATCH operation to add or remove individual users or roles (see roadmap!)


Updating team user membership:

  • GET/PUT request pair
  • Requires logic on the client-side to add/remove members to/from the members array as appropriate
  • The PUT contains ALL the members
  • The PUT response is similar to the GET response, in that it returns the team definition with all its members and roles
    • Can be handy to check all members have been added
  • Updating applies to both members and roles of the team

Best practice


Updating team membership:

  • Always update the team with all team members in one operation
    • rather than updating a team for each user
  • Same applies for roles a team is a member of

Updating teams:

  • Team (/Groups) endpoint is synchronous and subject to 5 minute timeout
  • The duration of the PUT call is dependent upon a number of factors:
    • Number of users (members) already in the team
    • Number of users being added and removed compared to the previous membership
    • Number of roles being added and removed compared to the previous membership
    • Number of users registered in the SAP Analytics Cloud service, not necessarily a member of any team
  • Means
    • When adding/removing members
      • the greater the number of members in the team, the longer the response time
      • the number of roles, has no impact on the response time
    • When adding/removing roles:
      • the greater the number of members in the team, the longer the response time
      • the number of roles, has no impact on the response time
  • In turn, this means as more users are added (into the team and also into the Service), the duration will extend
  • May need to ‘chunk’ updates made to the team for both members and roles
  • Only necessary when the team size > 4,500 users

Adding users into a team, chunk example

Adding 3,000 users into an empty team

  • Need the GET to read the current definition, metadata, displayname, roles


  • First ‘chunk’ PUT contains the first 1000 users
  • Second ‘chunk’ PUT adds in another 1000 users
  • Third ‘chunk’ PUT adds the remaining users


  • Chunking the update ensures that each request completes within the 5 minute window
    • (More on chunk size later)
    • This is a slightly silly example as you only need to chunk when the team size is > 4,500 members

Adding and removing users in/out of a team, chunk example

Chunk size is for both adding and removing

  • If the chunk size is 1000, then to remove 1000 members from a team (of 1000 members) and add 1000 different members would require 3 requests
  • The time to remove 1000 users is roughly the same as to add 1000 users
  • Its quicker to remove the users first before adding, since the duration is dependent on the number of members in the team
    • Always remove users (that need to be removed) before adding users to gain greatest throughput
  • If the chunk size was 2000, then it could be completed in 1 PUT request
    • The chunk would be ‘full’, as 1000 users are being removed and 1000 are being added

Chunk size for members of a team


Chunk size for members

  • Chunking in size of 1,000 members will not cause a timeout until the team size is over 30,000 users
  • However, changing membership of teams with over 1,000 members is slower than it could be, since multiple calls will be made unnecessarily
  • For the greatest throughput use the following formula to determine the initial chunk size:
  • Initial user chunk size = ( (42 - (42 - users_per_second)) x 210 ) - (number_of_users_in_the_team x 0.14)
  • Formula:
    • 210 is the target duration for the request in seconds
    • users_per_second is dependent upon the number of users registered in the Service
      • Maximum should be 41.1 for a service with a very limited user population
      • Reducing to 31.5 for a Service with a 40K user population
      • And reducing again to 21.85714 for a Service with a 80K user population
      • 21.85714 enables chunking for the largest team even at 32,500 users
      • The value ‘21.85714’ is hardcoded into the sample scripts – feel free to global search and replace this value

Adding users into a team, optimised chunk example

Adding 32,767 users into an empty team on a Service with 80,000 users

  • First ‘chunk’ PUT contains the first 4590 users, then 8262, then 11200 etc.
  • Each request was less than 5 minutes, whilst the entire duration to add 32767 users was ~50 minutes
  • As the team size increases, the request duration increases reflected by the ‘API users/sec’ observed throughput

Adding/removing users in/out of a team, dynamic chunk sizing

Adjusting chunk size

  • Maintaining the same chunk size for the duration of the operation when:
    • adding users into a team could cause a timeout
    • removing users from a team will reduce throughput
  • Need to adjust chunk size as team size changes over time
  • Suggested ‘configuration’ to make
    • When last PUT was:
    • > 4 mins 30 seconds, then reduce chunk size by 40%
    • > 3 mins 30 seconds, then reduce chunk size by 20%
    • < 1 min, then increase chunk size by 100%
    • < 2 mins, then increase chunk size by 75%
  • Only change the chunk size when the
    • chunk was ‘full’
    • request wasn’t a ‘repeated’ call due to a previous failure or re-attempt
  • as such requests tend to be quicker

Adding teams to roles

Updating roles the team is a member of

  • Identical to updating user membership
  • Except chunking of roles needs a different formula

Chunk size for roles a team is member of

Chunk size for roles

  • Chunking in size of 1 is suitable for all team sizes, however not optimal if adding the team to lots of roles
  • For the greatest throughput use the following formula to determine the initial chunk size:
  • Initial role chunk size = ( ( 210 / ( number_of_users_in_the_team x 46 / 210 ) ) x 10)
  • Formula:
    • Good, but conservative for small SAP Analytics Cloud Services and when team size is less than 5,000 members. Always ensure the value is at least 1
    • ‘210’ is the target duration in seconds
    • ‘46’ is the average rate of users/role/second on a SAP Analytics Cloud Service with 80,000 users registered. Maximum should be 68 for a service with a very limited user population
    • The value ’46/210’ is hardcoded into the sample scripts – feel free to global search and replace this value
  • API performance, for adding/removing roles, is dependent upon:
    • Number of users in the team
    • Number of roles being added or removed compared to the previous team definition
    • Number of users registered in SAP Analytics Cloud (regardless if they’re a member of the team or not)
  • When adding members into a team, it is necessary to reduce the chunk size. Unlike then, when adding/removing roles the response time is consistent regardless of the number of roles the team is already a member of
  • Therefore, there is no need to adjust the role chunk size as roles are added or removed

However, the formula (above) doesn’t allow for the greatest throughput for all sizes of SAP Analytics Cloud Services, a slight optimisation is possible

  • Suggested ‘configuration’ to make
    • When last PUT was:
      • > 4 mins 30 seconds, then reduce chunk size by 2
      • > 3 mins 30 seconds, then reduce chunk size by 1
      • < 20 seconds, then increase chunk size by 3
      • < 1 min, then increase chunk size by 2
      • < 1 min 30 seconds, then increase chunk size by 1
    • When adding/removing roles, the API response time isn’t entirely predicable, so alterations to the role chunk size should be made gradually hence the above only add or remove a few at a time


Order of operations for greatest throughput

  • Given performance of adding or removing roles is depended the number of members in the team:
    1. Remove any users from the team (that need to be removed)
    2. Then remove any roles from the team (that need to be removed)
    3. Then add any roles to the team (that need to be added)
    4. Then add any users to the team (that need to be added)
  • Changing roles and members at the same time is possible, but could be difficult to predict the duration and ensure the request keeps under 5 minutes


Managing errors

Sessions


Operations spanning multiple sessions

  • Multiple requests to complete an operation (like updating a team) can span sessions
  • Example:
    • Following successful PUT requests, one returns 401
      • (403 will be returned if the access token is valid, but the x-csrf-token is not)
    • Need to get a new access token
    • Obtain a new x-csrf-token by performing a GET
    • Re-submit previous PUT and continue


  • After obtaining a new accesstoken, catch any possible endless loop
  • Example:
    • Following new accesstoken request, upon first GET ‘fetch’ the x-csrf-token, but this also returns 401/403
    • Endless loops like this are extremely rare events and typically only occur when there’s an issue with the OAuth client in someway. Resolution may require a new OAuth client to be created and used
  • Remember that a new x-csrf-token must be
    • obtained for each new accesstoken
    • ‘fetched’ via a GET request before any non-GET request is made (or you’ll get a 403)
  • New sessions:
    • Only obtain a new session when one is really needed
    • Don’t obtain a new session thinking it will resolve any generic error, or you could end-up in an endless loop

Error leading to unknown state - Overview

500/502 responses

  • The API will typically return a 500 or 502 status if something went wrong inside SAP Analytics Cloud
    • Generally safe to re-attempt the request and typically this will resolve any ‘wobbles’
    • Commonly 502 is returned upon updating a team, when someone else is also updating it
  • However a POST /Users (create user) can be problematic and needs special care

401/403 responses - session expiry

  • A 401/403 response signifies ‘unauthorised’, the session has expired and a new accesstoken is needed
  • Typically, re-submitting a request is safe, once a new accesstoken has been obtained
  • However the session can terminate between requests or even ‘mid-request’ and this means the request may or may not have been successful
  • Means, any re-submitting of a POST /Users or a DELETE can be problematic and needs special care
  • Both types of errors (500/502 and 401/403) can be managed in a similar manner as they all lead to an unknown state

Managing errors

Introduction

Error on POST /Users triggering a repeat request

  • Request is complex because:
    • Adding a user to a team uses a different end-point
    • Your workflow may involve adding a user to a team, and the response of the request may impact that workflow, potentially preventing the user being added to a team
  • A duplicate user ‘_1’ is created if the unique combination of ‘userid’ and ‘email’ don’t match. A repeated request will result in a 409 conflict
    • This means you may not know if such a duplicate user was created or not. This is only ever a problem if you’re using a ‘create’ workflow to update a users’ email
  • Default roles are applied after the user is created
    • Your workflow may include a GET/PUT request pair, even though you’ve just created the user, and caution may be needed upon checking the ‘Roles’ assigned, since additional Roles may have been added. You wouldn’t want your workflow to accidently remove any of these roles. The same applies to the ‘Business Intelligence license type’, that can change when a default role is applied

Error on DELETE /Users /Groups triggering a repeat request

  • Request is complex because:
    • The object you are deleting may or may not have been deleted
      • Could mean any repeated call returns a ‘404 not found’ which could be unexpected


Managing errors with POST /Users

Errors following POST /Users (creating a user)

  • Very occasionally (less than 1 in 5000) a 502 response is returned
  • Almost always, the user will actually have been created ok
  • If you repeat the request AND you reply on the response code of 409 (conflict):
    • The 409 (conflict) response will be because you’re trying to create the user with another that shares the same email
    • This is because the request that returned the 502, was actually successful!
      • In other words, the user is in conflict with itself

  • What this means in practice when repeating the create user request
    • if you get a 409 (when the previous response was a 502) then the user exists and was probably created ok in the first request
      • You could follow your ‘user created successfully’ workflow to add the user to a team if needed
      • (sample scripts 10x have this logic, but it may not be safe to assume the username was created as requested, you need to read the user to check. The username will only be different if there’s a clash of email address. Samples 11x, 12x read the user to check)
    • if you get a 502 then simply read the user again (even if you’ve read it before to find the user didn’t exist which lead you to create it)
      • (sample scripts 20x have this logic)

Managing errors for a ‘create only’ workflow

502 following POST /Users (creating a user)

  • The create workflow, without an update aspect, shouldn’t be used if your logic assumes a 409 means the user wasn’t created and that logic is still needed (for example to add the user to a team)
    • (sample script 103 falls into this category as it has no update aspect, so it assumes the user was created if a 502 is followed by a 409)
  • For example it might mean your logic that adds them to a team is missed
  • ‘Create only’ workflow (without an update) is only ok when:
    • not using logic to add users to a team
    • the email hasn’t changed for any existing user
    • (sample scripts 101 and 102 fall into this category)
    • (sample script 103 mitigates this, by testing if a 502 is followed by a 409. Script 103 adds users to teams)

Managing errors for a ‘update-create’ workflow

  • The combined ‘update-create’ workflow that is based on the ‘update’, becomes as shown
  • The POST /Users with a 502 now re-reads the user to check the user was created
  • Put simply, you need to perform a GET request to test the status of the user following a 502 response


  • Any 401/403 error from a POST /User will also need to issue a GET request (once a new access token has been fetched)
  • Possible ‘gotcha’ with this workflow is any default role could change the ‘Business Intelligence License Type’ and the GET/PUT pair could undo this if you test or change that property

Managing errors for a ‘create-update’ workflow

The combined ‘create-update’ workflow that is based on the ‘create’, becomes as shown

  • Any 502 response following POST /Users (user creation) will simply re-attempt the request and follow the 409 update workflow
  • However, the original POST /Users request, although was most certainly successful, will have created a duplicate user ‘_1’ if the email didn’t match (because the email needs updating)
  • This means the subsequent GET would be on the original user, not the ‘duplicated’ one and the following PUT would return a 400 as you can’t update a users’ email to match that of another user (the duplicated user)!
  • Therefore, we need to delete the duplicate user and update the right users’ email
  • (A) We can search for the duplicate user and identify it by email
  • (B) Once we’ve found the user, we can check the UserID is different from the ‘right’ user (it will have a ‘_1’ at the end)
  • (C) We can then be sure to delete the correct duplicated user
  • The PUT request will then be successful as the email will no longer conflict
  • Same workflow applies for any repeated POST /Users, such as following a 401/403 because the session has expired
  • Sample scripts 12x and 13x follow this design
  • Default roles could have been granted to the user on the user creation
    • Important to allow these additional roles to remain and any GET/PUT doesn’t remove them
    • Possible ‘gotcha’ with this workflow is any default role could change the ‘Business Intelligence License Type’ and the GET/PUT pair could undo this if you test or change that property


Miscellaneous

Creating & Updating users Business Intelligence License type

isConcurrent user property

  • The request body setting is respected when:
    • There are NO roles directly assigned to the user (as compared to roles assigned indirectly, due to inheritance, via a team)
    • OR if any roles directly assigned to the user and these roles are Business Intelligence roles and/or Analytics Hub roles
    • There are concurrent licenses assigned to the SAC Service, and even when there are no concurrent licenses assigned to the SAC Service! It means the API allows you to set concurrent users, when you don’t even have a license for them!
  • The setting is not respected when:
    • Any roles directly assigned to the user (as compared to roles assigned indirectly, due to inheritance, via a team) are Planning roles
    • A Planning role can be either a ‘Planning Standard’ role, or a ‘Planning Professional’ role
  • The responseBody isConcurrent value will be the same as the requestBody isConcurrent value in all cases
  • Any isConcurrent setting is changed immediately after the request. It means a further GET /Users/UserID may be needed to validate the isConcurrent setting has been accepted
  • This applies for both the user creation (POST /Users) and updating a user (PUT /Users/UserID)
  • See related article via this blog for more on applying concurrent licenses

Other best practices - Schema versions

Avoid losing properties on schema version change

  • If you perform a GET followed by a PUT need to ensure the PUT request body contains the entire schema
    • User and Team schemas have a version and the User has additional extensions
    • If the schema version changes, or an extension is added, you could be PUTting back fewer properties than you read!
  • So, either check the schema version or read/write all the properties dynamically
  • If you omit a schema, then the existing properties are not respected
    • It means you must present all the schema’s and all the properties of each schema

Differences between NEO and CF

Known differences between NEO and Cloud Foundry

  • Basically identical, except for:

NEOCloud Foundry
Obtaining access token methodsGET or POSTPOST only
Access token path/oauth2/api/v1//oauth/
  • If you are using the NEO Platform, then a migration to CF may be an option for the future
  • Therefore it makes sense to:
    • Just use the POST method
    • Make the ‘oauth’ path a dynamic variable in your code

Integrating the SAP Analytics Cloud SCIM API

Strategy

  • A pure ‘user event based’ design may not always be suitable for scaling to a large user base
    • May need a mixture of models: ‘user event’ and ‘mass user update’
  • Ideal for maximum throughput:
    • ‘User event’ for user creation, without granting access rights to Teams
    • ‘Mass user update’ for regular granting of users to Team giving them access rights
  • Any ‘re-sync’ should not involve a ‘removal all – add all’ approach, but rather an intelligent ‘update team membership’ approach
    • Could even run this on a very regular basis

Considerations for provisioning solutions

  • Many provisioning solutions operate on a pure ‘push’ only model
    • Such solutions don’t perform a ‘GET/PUT’ pair but expect to issue a PATCH request
    • Means that updating a user or a team can ‘undo’ its configuration (users or roles can be dropped)
    • Given PATCH is not currently available (though planned) a slight adaptation is necessary
  • To resolve problems updating a user (PUT /Users/{id}):
    • Avoid the use of ‘default roles’
    • These change the roles and concurrent license type after the user is created
  • Instead
    • Create the users without any role
    • and with the right Business Intelligence user license
    • Assign the user to teams
    • Assign the teams to roles
    • (i.e. follow standard best practice)
    • All user POSTs and PUTs would need to set these
  • To resolve problems updating a team (PUT /Groups/{id}):
    • Hard code a mapping so a team, of a given name, is always assigned to certain roles
    • SAP Identity Provisioning has suggested code SAP Note 3027079

Sizing

  • For each request:
    • Duration for an ‘empty’ Service and the duration for a Service with 80,000 users registered
      • Where possible ‘lower end’ and ‘upper end’ averages are shown
    • Key information for how the endpoint behaves:
    • Influenced by
    • How it scales
    • Handy notes
  • Sample scripts throughput performance
    • For each sample, again for an ‘empty’ Service (or one with 500 users registered) and for a Service with 80,000 users registered
    • Throughput expressed as users/sec, secs/user etc.
    • Includes client-side overhead shown in (brackets) and expressed as a %
    • Please see the ‘samples’ article for details on the samples performance
  • Conduct your own ‘sizing’ exercise based upon the information provided
    • Endpoint performance
    • Throughput of the samples
    • % overhead of the samples
    • don’t underestimate the overhead processing of your client to SAP Analytics Cloud


Endpoint performance

Observed times by request

  • Times
    • Provided in seconds
    • Based on an average of at least 500 requests made during a ‘run’ of a sample script
    • The average times means many are quicker and slower
    • Lower end is the observed fastest average time (of multiple ‘runs’)
    • Upper end is the observed slowest average time (of multiple ‘runs’)
    • For both an almost empty Service (or up to 500 users) and one with 80,000 users registered in it
  • Observed times based upon
    • SAP Analytics Cloud waves 2021.1 to 2021.8
  • Times are guidance only
    • Times are provided only for guidance purposes and based upon observations taken at random
    • Should expect some requests to be considerably longer (and quicker!)
    • Should expect the performance to change and generally improve over time
    • Many factors influence performance: network, day of week, time of day, client-side configuration etc. etc.
    • Don’t reply too much on the times shown in this article, use them only for guidance!

POST /Users (create user)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
  • Scales
    • Linearly
  • Notes
    • API performs the same as the ‘Import Users’ manual workflow within SAP Analytics Cloud
    • Creating a user could require updating the /Groups endpoint to add the user to a Team
    • Isn’t influenced by number of roles the user is a member of
    • Any default roles defined are added after the user is created
    • It means the response body will not list default roles, unlike a GET once the user has been created
    • userid has a maximum length of 20 characters (of upper case, numbers or ‘_’)
    • * anomaly in the test results – just demonstrates there are many factors that impact performance and likely because the tests on the 500 user Service was at a very busy time

GET /Users/{id} (read user)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
  • Scales
    • Linearly
  • Notes
    • The ‘roles’ property will have length 1 with an empty string when no roles are assigned

PUT /Users/{id} (update user)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
  • Scales
    • Linearly
  • Notes
    • Must use with a GET request, so not to ignore the current properties
    • Isn’t influenced by number of roles the user is a member of

DELETE /Users/{id} (delete user)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
  • Scales
    • Linearly
  • Notes
    • You can’t delete a user that is a manager of other users

POST /Groups (create team)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
      • but only very slightly (0.1 seconds for every 80K users)
  • Scales
    • Linearly
  • Notes
    • API doesn’t involve adding any members or roles to the team

GET /Groups/{id} (read team)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
    • # of members and roles in the team (members and roles count the same)
    • Response of non-existing teams (404 ‘not found’) aren’t influenced by anything!
  • Scales
    • Linearly

PUT /Groups/{id} (update team members)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
    • # users already in the team
    • # users being added or removed compared to the previous team definition
      • The greater the ‘chunk size’ the greater the throughput
      • The largest ‘chunk size’ here was just 500 and a greater throughput is possible with a larger chunk size
      • With the maximum ‘chunk size’ (*) can be as great as ~10.5 users / sec for a Service with 80K users and a team of 20K users
  • Scales
    • For adding/removing members (users) – linearly
  • Notes
    • Removing members is about 15% to 20% quicker compared to adding members
    • Adding/removing members isn’t influenced by the number of roles the team is a member of
    • Maximum team size is 32767 members


  • Throughput of an SAP Analytics Cloud service with 80,000 users registered:
    • A linear throughput is observed as 32,767 users are added into an empty team in 14 ‘chunks’ (14 PUT’s)
    • Starts with over 20 users / second, reduces to 6.7 users / second
    • These are the maximum throughput times, since the largest chunk size was used on each request
    • Chunk size reduces as team size increases to ensure the request stays under the 5 minute timeout

PUT /Groups/{id} (update team roles)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
    • # users already in the team
    • # roles being added or removed compared to the previous team definition
    • The greater the ‘chunk size’ the greater the throughput – (1) has chunk size of 1
    • Throughput remains roughly constant as the members and roles increase, though it tends to fluctuate more compared to updating members
    • For an SAP Analytics Cloud Service with 80K users, the average maximum throughput is ~46 users/role/sec. For small populations the rate will be greater ~70 users/role/sec
  • Scales
    • For adding/removing roles – linearly
  • Notes
    • Removing roles is identical as adding roles
    • Adding/removing roles isn’t influenced by the number of roles the team is already a member of (unlike adding members)

DELETE /Groups/{id} (delete team)

  • Influenced by
    • # users registered in the SAP Analytics Cloud Service
    • # of members (users) in the team
  • Scales
    • Linearly
  • Notes
    • This deletes only the team, not the members (users) of the team
    • It means the users still exist after the team has been deleted
    • The team will be removed from any roles it’s a member of



In general, please post your comments to the blog post that introduces this wiki page rather than directly here. Thank you





  • No labels