Page tree
Skip to end of metadata
Go to start of metadata

Blog

This article is introduced by a blog providing an overview and summary

Download

Preview or Download this presentation (version 1.0.1)

Video

Preview or Download this as a webinar



This article is suitable for either of the following:

  • System administrators needing to use the SAP Analytics Cloud SCIM API, but has no interest in how the API works, nor desires to undertake any development effort
    • i.e. you just want to use it to fulfil a function
  • An API developer who wants to
    • dramatically accelerate their development
    • use the samples as a basis to incorporate best practices in their solution and avoid surprises, especially as user volumes increase
    • Developers will also need to refer to accompanied ‘SCIM API Best Practice and Performance’ article

Use-cases

  • Creating users before they login
    • (Since even once ‘automatic user creation’ is enabled for SAML SSO, the user isn’t created until they login)
    • To receive a publication via email
    • To apply security directly on the user
      • generally not recommended to apply security on the user directly, generally better to use a team to benefit from inheritance
      • More typical with a planning use-case
  • Deleting users

  • Updating users:
    • license type, active status
    • manager
    • default language, datetime/number format etc.


  • Team management
    • Creating teams
    • Adding/removing users to/from a team
      • (scripts are not subject to the 4000 user limit per team)
    • Team on team operations, such as
      • Adding members of one team to another team
      • Removing members of one team from another team
    • Deleting teams

  • Transporting users and teams from one SAP Analytics Cloud Service to another
    • Doesn’t really transport a user since their personal content is not transported!
    • Transports users, users of a team* and teams
    • Includes their roles and team assignment of the user and the primary team
      • i.e. all the green lines in the diagram
  • Supports both NEO and Cloud Foundry Platforms (and any combination)
  • SAP Analytics Cloud supports transporting:
    • Teams (without their membership)
    • Team to Roles relationship
    • Roles (without their user membership)
    • But the API does allow us to do this!

Sample scripts

Postman collections

Demo

Preview or download


Guidelines used for development of these samples

Easy to consume

  • Simplified data file as much as possible
  • Most scripts support .csv data files, others need to use a .json structure
    • .json files needed for arrays that can’t be expressed in a .csv, e.g. an array of users to add to a team
  • Sample for almost every need
  • Minimum code changes needed, code is commented thoroughly

Maximum throughput

  • of the SAP API
    • Highly tuned and intelligent logic
    • For example, adding a list of users to a team, only the users that need to be added will be added
    • Automatic self-tuning, chunking and batching of team updates
  • of Postman
    • Optimised code, Java heap optimisation, use of Postman variables etc. to minimum script overhead

Known result

  • Although highly optimised, its critical the sample returns a known result


Sample scripts description

For each group of samples, two main pieces of information are provided:

  • Overview
  • Throughput performance
    • Throughput expressed as users/sec, secs/user etc.
    • For an ‘empty’ Service (or one with 500 users registered) and for a Service with 80,000 users registered
    • Includes client-side overhead shown in (brackets) and expressed as a %

Sample scripts for creating and updating users

Overview

  • 18 samples that create and update users
    • Some samples read .csv, others read .json
    • Some are optimised for creating, others for updating
  • Scripts here that update only update:
    • Business Intelligence license type (concurrent/named)
    • Manager
    • Email
    • Roles and/or Teams
      • (depending upon which script you pick)
  • If you’re unsure, your likely ‘go to’ scripts are highlighted

Data files

  • Scripts xx1 can read a data file of type .csv and .json
  • Scripts xx2, xx3 can only read a data file of type .json


Sample scripts for creating and updating users

  • Scripts 10x only create users and perform no update function
  • Scripts 11x, 12x, 13x perform both perform create and update functions
  • Scripts 1xx will create a duplicate user ‘_1’ if the email doesn’t match the userid
    • Scripts 12x will manage this by deleting this duplicate user before updating the correct users’ email!
  • Scripts 13x and 23x are only for when
    • using SAML SSO
    • and you’re mapping the user on their email
    • and you need to obtain a predictable user ID, defined by you, rather than one derived from the email address and defined by SAP Analytics Cloud
  • For greatest throughout
    • Use scripts 1xx when at least 52% of your users in the data file are to be created, otherwise use 2xx
      • If you’re updating emails, then need at least 77% of the users to be created
    • Do NOT use these scripts if you are just updating the team assignment, use scripts 6xx which will provide a greater throughput(0.79 users/sec verse 38.17 users/sec!)
    • Although script 13x and 23x can be used when not using SAML, and not mapping on the email, they are much slower as an update to the user will always be performed after creating a user (this is to update the users’ email, unlike all other scripts that do this at the time of user creation)
  • Team updates are performed every 500 users and at the end of the data file

Updating users’ roles and teams

Flexible options for role and team assignment

  • Scripts that perform update functions on users’ roles and teams have an ‘action’ function, similar to the SCIM ‘operation’ function
    • Possible actions: add, remove, replace, keep
  • Role and team actions operate independently of each other
  • An empty array [] can be used with ‘replace’ to remove all roles, or remove all teams from a user

Throughput performance



Sample scripts for updating users

Overview

  • All scripts are ‘user centric’, even script 403 which given a user you can assign teams to that user
    • Unlike script 6xx which given a team you can assign users to that team
  • Scripts 406 and 407 are only needed if you created users incorrectly in the first place!
    • Avoid needing to use these by defining the users correctly at time of creation


Throughput performance


Sample scripts for updating users

  • Do NOT use these scripts if you are just updating the team assignment, use scripts 6xx which will provide a greater throughput
  • * % overhead is greater with more teams

Throughput performance for user and team centric scripts

Updating team membership

  • Scripts 403 and 6xx both update team membership
  • 403 is ‘user centric’, and 6xx are ‘team centric’
  • Scripts 6xx will provide a greater throughput compared with any user centric script
    • 0.79 users/sec verse 38.17 users/sec!
    • Always use script 6xx for updating team membership whenever possible

Updating users by team

Overview

Sample scripts for updating users by team

  • Like the 40x samples, these 45x samples remain ‘user centric’ but unlike the 40x samples, you specify a team name to obtain the list of users that are to be updated
    • Means the data file is very simple, its just a list of teams, instead of a list of users
  • Scripts 456 and 457 are only needed if you created users incorrectly in the first place!
    • Avoid needing to use these by defining the users correctly at time of creation

Throughput performance

Sample scripts for updating users

  • Postman isn’t really designed for thousands of requests per iteration
    • Means the maximum team size for all these scripts is about 1000 users until stability issues should be expected

Deleting users and teams

Overview


Sample scripts for deleting users and teams

  • Script 852 deletes all users of a team as well as the team itself
  • Script 801 deletes the team and might timeout if the team is very large
    • it will still delete the team!
    • 802 just removes the users before deleting the team, so not really much point to it!
  • Both 801 and 802 will not delete the users of the team, they just delete the team
  • 852 is quite versatile, enables you to delete either just the users of a given team, the team without deleting the users, or both!
  • You can’t delete a user that is a manager of other users
    • Need to either
      • Delete any manager after having deleted all the users of that manager
      • Update the other users to have a different manager, or no manager at all!

Throughput performance

  • Don’t compare 301 with 80x as these scripts delete entirely different objects, one is for users, one is for teams
  • If 852 is deleting the users and deleting the team, the performance will be:
  • = (time to delete all users) + (time to delete an empty team)
  • Postman isn’t really designed for thousands of requests per iteration
    • Means the maximum team size when deleting users of a team is about 1000 users until stability issues should be expected

Creating teams and updating their display name

Overview

Sample scripts for updating and creating teams

  • A versatile script that can do one or both of the following:
    • Create a team, if it doesn’t already exist
    • Update the displayname of a team
  • All of these sample scripts will create a team if the team doesn’t exist, but the displayname will be (team name)+“ with team folder”
    • e.g “Team1 with Team Folder”
  • So this sample script enables you to give the team displayname a more suitable title
  • Whenever a team is created, a team folder is always created with it
    • You may need to manually delete this folder if its unwanted
    • (The option, within the user interface, to not create a team folder was provided after the API was developed)

Throughput performance

Team Management

Overview

Sample scripts to manage teams

  • Allows you to perform ‘actions’ on teams, such as:
    • ‘add’ or ‘remove’ users to or from a team
    • ‘add’ or ‘remove’ roles a team is a member of
    • Add team members of one team to another
      • You can’t add a team to another, this adds team members to be members of another team
  • All scripts are team centric and provides far superior throughput compared to any 1xx, 2xx, 4xx scripts


  • Sample 602 and 612 reads an array of users in field file_JSON_users, and an array of roles file_JSON_roles
  • Whilst 653 uses another team to obtain the list of users and roles
  • An empty array [] can be used with ‘replace’ to remove all roles, or remove all users from a team


Batching of teams

  • Functionally 602 and 612 are identical, however:
    • 602 will not perform any updates until the end of the file, when it will then update the ‘net’ result of all previous entries
      • It ‘batches’ the requests and intelligently determines the net result
      • Will also update the teams when (teams read >30, or (teams read>4) and (users >40,000))
    • 612 will not ‘batch’ the requests together, it will update each team immediately after its read the team
  • 612 is recommended as it will have a higher throughput and will generally be more stable, however 602 is useful when you have lots of entries for the same team, rather than 1 entry per team
    • Sample 653 also behaves like 612, it also doesn’t batch updates to a team
    • 602 has the overhead of holding all the teams’ definitions is memory, but has the advantage of only processing ‘net’ difference
    • (though all scripts only process the ‘net’ difference when updating a team, 602 determines the net difference across multiple entries of the data file)



Operations

  • All 6xx scripts provide actions: add, remove, replace and keep
    • Means a simple ‘team copy’ can be performed with ‘replace’
    • ‘keep’ is useful as it does nothing, handy if you just want to update only users, or only roles
  • Script 653 have additional actions: intersect and exclude
    • Enables ‘set’ logic on team members
  • All 6xx scripts operate actions on both team members and roles (the team is a member of)
    • Team and roles actions are independent of each other

Throughput performance for users (members)

  • Adding users into an empty team, cumulative time (on a Service with 80,000 users registered):
  • By 11 minutes 20 seconds: 11,200 users where added into the team
  • By 40 minutes: 28,450 users where added into the team


  • In another test, adding 10,000 users into 8 empty teams took 1 hour 28 minutes
    • (on a Service with 80,000 users registered)


Throughput performance for roles


Transporting teams and users from one SAP Analytics Cloud Service to another

Overview

Sample scripts for transporting users and teams from one SAP Analytics Cloud Service to another

  • Highly versatile scripts that ‘transport’ users, teams, teams of users and their respective relationship to roles
  • Source and target can be on different ‘Platforms’ (NEO or Cloud Foundry) or on the same
  • If you are using SAML SSO and mapping the user on the email then use samples 933 and 983
  • Limitations apply which are explained later

Sample scripts 903, 933 ‘transport’ just users

  • Users aren’t really transported as such, as their personal content is not transported
  • Teams aren’t really transported as such, as any team folder content is also not transported
  • When a user is transported:
    • It includes these user properties:
    • Userid, userName, preferredLanguage, active, email address, photos, dataAccessLanguage, dateFormatting, timeFormatting, numberFormatting, cleanUpNotificationsNumberOfDays, systemNotificationsEmailOptIn, marketingEmailOptIn, isConcurrent, manager
    • The user in the target will always be updated with these properties (and only these properties) of the source user
    • The target user will either be updated or created as needed
  • Scripts provide actions for Teams and Roles: add, remove, replace and keep
    • The target users’ team and role assignment is as per the diagrams
    • Team and role actions operate independently of each other
    • Allows for flexible options and supports a variety of life-cycle scenarios


Sample scripts 953, 983 transporting teams

  • You provide a ‘primary’ team which is simple the team of users you’d like to transport
  • The relationships highlighted in green can be transported
    • A user can be in multiple teams and any non-primary teams are ‘secondary’
    • Both the user and the primary team role membership can be transported too
    • The secondary team to role membership can not be transported. This means you may need to re-run the script for other teams to be ‘primary’
  • Flexible options enable different relationships to be transported, with or without the actual users being transported themselves


Sample scripts 953, 983: Example 1: Transporting just the users, of a given team

  • When ‘file_transport_users_of_primary_team’ is true the users of the team are transported
  • The ‘file_transport_primary_team_roles_action’ is keep, so the target relationship is respected
    • All the other actions are available (add, remove, replace)
    • If you need to update the roles of other teams, you’ll need to run this script for each team
  • Postman isn’t really designed for hundreds of requests per iteration
  • Means the maximum team size, when transporting users, is about 100


Sample scripts 953, 983: Example 2: Transport users and their team relationship


This configuration will transport the users’ team membership


Sample scripts 953, 983: Example 3: Transporting primary team with display name only

  • The users are not being transported, nor the members of the team
  • Because the ‘users_of_primary_team’ is false, the users_role_action and users_team_action has no effect
  • There is no limit on the number of users that make up this team membership


Sample scripts 953, 983: Example 4: Transport primary team membership without transporting the actual users

  • This configuration transports the team membership and the teams’ display name
  • The users in the source are expected to be in the target
    • If the users are missing in the target the console log will list each user not found
  • The ‘users_of_primary_team’ is false so the users_role_action and users_team_action has no effect


Throughput performance

  • Data for the other scripts is not provided
  • Script 933 is comparable with 903 and 233
  • Script 993 is comparable with 953 and 233


Getting started overview


Setup

  • User Guide provides step-by-step instructions
  • Takes about 40 minutes, longer if you need to make changes to the code in steps F and G


  • Step C includes creating OAuth client in SAP Analytics Cloud
    • need SAP Analytics Cloud admin rights to complete this task
  • Step F is needed if you have multiple SAP Analytics Cloud Services and you’d like to ‘transport’ users and/or teams from one to the other
  • Step G is needed if you’re creating users with scripts 1xx
    • ‘System wide’ user settings (like datetime format, language etc.)
    • Set any SAML mapping property

User Guide provides for each sample:

  • Full data file syntax specification
    • sample .csv and .json file are provided making it easier to get started
  • Detailed description for how each sample works
  • When its suitable and not suitable
  • Known limitations, throughput and overhead %


Creating users hardcoded settings

  1. Preferred language
  2. Data access language
  3. Date formatting
  4. Time formatting
  5. Number formatting
  6. Clean-up notification number of days
  7. System notification opt-in
  8. Marketing email opt-in

Hardcoded settings for creating users

  • To keep the data file as simple as possible, many user properties are hardcoded into the scripts
  • Before creating any users, you must edit the scripts to use the settings for your environment
    • Only necessary for scripts that create users, except scripts 9xx, which ‘transports’ a user with all their properties
    • Most likely all these setting will be the same for all your users
    • The User Guide has more details


SAML Authentication – mapping to anything other than userid

  • If you’re using the default Identity Provider, then this part isn’t relevant
    • All you need to know is that userName and id (or userid) are all one and the same thing


  • If you’re using your own Identity Provider AND you’re mapping on the users’ email AND you’d like to assign a userName of your choice, rather than one based on the email and determined by SAP
    • Then you don’t need to update any scripts, simply use samples 13x, 23x and 953 and 983


  • If you’re using your own Identity Provider AND you’re NOT mapping the user based on the userid
    • Then you must also update the 1xx create user scripts
    • For example, you’re mapping on the email (and you want to allow SAP to determine the userName based on the email)
    • The User Guide has more details

Recap of existing best practice for user to role/team membership

  • It’s best practice to
    • add users to be members of teams (green line)
    • where the teams are members of roles (green line)
    • rather than assign a user ‘directly’ as a member of roles (red line)
  • Because
    • access permissions on folders is specified by teams, not roles
      • (also possible on individual objects and also by individual users)
    • a single teams aggregates multiple roles
      • a single team thus forms a ‘Business Role’ definition like ‘Power User’
    • changing the ‘Business Role’ (a team) requires only updating the team, not all the individual users


Prerequisites for using Teams


Must respect:

  • All teams must be created via the API *
  • Teams are created with a Team Folder
  • Teams can’t be created if the team folder already exists


Means:

  • For any team that has been created in the user interface, the API cannot work with it. It means if you’ve created any team through the user interface of SAP Analytics Cloud you can’t use the API to update them. This in turn, means you may need to delete all or some of those existing teams if you want to
  • use the API to manage them.
  • Once a team has been created through the API the user interface can still be used to manage those teams in the same way as a team created manually via the user interface. You can mix the management of the team through both the API and the user interface as you please.
  • Need to manually delete the Team folder, assuming its not needed
  • Need to be careful not to transport a team via the Content Network or the legacy Export/Import method


  • * applicable for non-SAP data centres. KBA 2857395 provides a workaround


Prerequisites for using Roles

Must respect:

  • Roles must pre-exist
    • API can’t yet create or update roles, only read who is a member of a role
  • Means:
    • if you want to add a user or a team to be a member of a role, those roles must pre-exist


Content namespace

Namespace aware

  • The samples are ‘Content Namespace’ aware
  • It means when a role is provided as a value in the data file if the role shares the same content namespace as the Postman environment there’s no need to specify the PROFILE:namespace: as part of its name
    • For example, if a Role called ‘Role1’ exists in the SAP Analytics Cloud Service and the Postman environment ContentNamespace variable is set to the same namespace as the Service the role was created in, then only ‘Role1’ needs to be specified
  • For any role that doesn’t match the ContentNamespace then PROFILE:namespace: must be specified with namespace set accordingly
    • For example, the provided ‘BI_Admin’ role would be PROFILE:sap.epm:BI_Admin


How to determine the ‘Namespace’ of a role

  • Make an API call to GET a users definition
  • Easiest with your browser
    • http://SERVICENAME.XX.sapanalytics.cloud/api/v1/scim/Users/userID
  • The namespace will be shown as part of the roles
    • The namespace for the default Admin role is ‘sap.epm’
    • But you need the full text in the JSON entry:
    • PROFILE:sap.epm:Admin
    • More info in the documentation

Common design

  • If the team doesn’t exist, it will be created for you
    • Remember you may need to delete the team folder
  • Full session, error management and intelligence built-in:
    • The samples will re-establish any sessions as needed
    • All collections, within the same Postman Environment, will share the same session
    • Errors from the API are handled and automatic re-attempts will be made
    • This includes errors that are exceptionally rare
    • Updates to users and teams are only performed if any changes are necessary
  • Data file field names
    • All share common field names (file_userid for example), except for scripts that delete, so to avoid accidental deletion!
  • The Postman console shows ‘errors’ ‘warning’ and ‘info’ logs that are typically handy to read

Managing errors Non-API errors

  • There can be many non-API errors that need to be managed by the client application
  • Postman, for example, may report an error such as:
    • Error: Socket hang up
  • In these conditions, Postman, will quit that request and move to the next entry in the data file
  • This is when ‘batching’ requests could be problematic
    • For example, if Postman quits a request (and it’s the last request in the data file) but the script has ‘batched’ together requests into a single call, then all those batched requests will fail
    • May need to re-run the entire script
  • Postman recommend you use the command-line interface rather than the Graphical User Interface for performance and stability reasons

Developer Notes

Refer to the ‘Best Practices’ article for more details on the workflows the scripts support
To get started with Postman, Postman recommends https://learning.postman.com/docs/getting-started/introduction/

Postman code


Postman script optimisation included

  • Variable definitions: ‘let’ preferred over ‘var’, and ‘const’ preferred over ‘let’
  • Local variables preferred over collection and environment variables
  • Kept variable use and assignment to minimum

Result was

  • More stable collection
  • Collections where able to run for longer
  • Reduction in script overhead

Postman ‘tests’

  • Scripts will show a ‘pass’ if the script manages and handles the response
    • For example, a status 404 ‘not found’ to read a team, won’t show an error, as the team will be created
    • May need to adapt the script tests as needed for your purposes

Updating teams

  • Updates to all teams, regardless of the script, follow the same design
  • An initial ‘chunk’ size is determined
    • Initial user chunk size = ( (42 - (42 - 21.85714)) x 210 ) - (number_of_users_in_the_team x 0.14)
    • Initial role chunk size = ( ( 210 / ( number_of_users_in_the_team x 46/210 ) ) x 10)
  • Once the first update has been made, the chunk size will automatically and dynamically adjust to optimise the throughout based upon the performance of SAP Analytics Cloud
    • Please refer to the best practices for expected throughput figures
    • Feel free to adjust these figures accordingly (global search and replace on the text “21.85714” and “*46/210”)
    • Only need to do this, if any team is over 4,590 users and total users < 80,000
  • Updating a team is optimised to this order of actions
    1. Remove any users
    2. Remove any roles
    3. Add any roles
    4. Add any users
  • Only users and roles that need to be added or removed are actually made, i.e. the scripts are highly intelligent and don’t perform a blind update
    • This means an update to a very large team could take just a few seconds
  • Throughput (users/sec users/role/sec) is shown in the console as the team is updated


Frequently Asked Questions

Q What happens when a user is created (not necessarily with any roles) and default role(s) are also defined
  • The default role(s) are added after the user is created
  • The default roles may change the Business Intelligence License Type at the same time as the default role is assigned
Q Can I create a user without any roles
  • Yes, and this is generally considered best practice as it generally easier to add uses into team(s) and then add the team(s) into roles. There are many advantages, one is when you need to change the role assignment you only need to update the teams assignment to roles, rather than all the individual users.
Q Does the users’ active status have an impact on the license consumption
  • No. It means a value of true and false both count towards license consumption
Q What happens when the users’ active status is changed from true to false
  • If the user is already logged in, their session will be terminated immediately
  • They will receive an error when they attempt to login
Q Is the performance greater when ‘importing’ a list of users to be created via menu-security-users compared to the API
  • No, the performance is the same
Q When I import a list of users to be created via menu-security-users what can’t I do
  • You can’t create users with a data access and language settings of your choice
  • You can’t set which teams the user should be a member of
  • If you export your list of users, you can see which fields are available, or not!
  • It means using the API (and these sample scripts) is typically a better option
Q Is it possible to create users without an email notification being sent
  • If the non-default IdP is used, the email notifications are not sent KBA 3043764
  • For those using the default IdP, then email notifications are always sent, however you could create users with the ‘wrong’ email address and then update the email after.
  • Samples 13x, 23x, 933 and 983 may fulfil your requirement entirely!
  • These scripts will create a user with the ‘wrong’ email and immediately update the user with the right email
Q When deleting a user via the API will it delete personal content
  • Yes, all personal content is deleted and there’s currently no option to transfer the ownership (unlike using the user interface of SAP Analytics Cloud)
Q When deleting a user via the API, what happens to their public content
  • Their public content remains and the ownership is ‘lost’, however re-creating the user with the same user ID will restore the ownership of public content
Q Can I see what actions have been performed via the API
  • Yes, the Activity Log will show all the events that are captured
  • The username will contain the OAuth client username
Q Can I use the API and update Teams that where created via the Content Network or legacy Export/Import
  • No (unless you are using an SAP Data Centre)
  • Such teams will be created as though its through the user interface KBA 2857395
  • It means that you need to be careful about transporting teams if you want to use the API to update them
  • These samples enable you to transport teams
Q How can I remove all teams a users is a member of when I don’t necessarily know the teams they’re already a member of

Sample script 403 will fulfil this requirement. Just set the ‘teams action’ to ‘replace’ and provide an empty array for the list of teams:

Q I have questions about assigning Business Intelligence concurrent licenses, what is the best resource to answer these types of questions

An article introduced by this blog will answer all questions about the technical assignment of licenses when assigning roles and teams to users 

Q How do I update all members of a team with a different Business Intelligence license type

Script 451 will do this for you

In general, please post your comments to the blog post that introduces this wiki page rather than directly here. Thank you

  • No labels