Skip to end of metadata
Go to start of metadata

Metadata Management Questions

What native SAP Metadata Integrators does SAP Information Steward support?

SAP Information Steward supports native metadata integrators in support of SAP PowerDesigner, SAP Data Federator, SAP Data Services, SAP HANA, SAP NetWeaver Business Warehouse and SAP BusinessObjects Enterprise.

For additional details, check out the SCN blog, “Metadata Management in a SAP Landscape.”

In addition to these native SAP metadata integrators, there are additional native metadata integrators in support of Common Warehouse Modeling (CWM) and popular relational databases, including Teradata, Sybase SQL Anywhere, MySQL, Oracle, MS SQL Server, IBM DB2

See the Information Steward Product Availability Matrix on SAP Support Portal for an up-to-date listing and details on the product versions supported.

How is metadata discovery supported with the MIMB (Metadata Integration Model Bridge) Integrator Source?

Information Steward will extract both metadata objects and their relationships for non-native integrator sources like IBM Data Stage, Netezza, etc. with the Metadata Integration Model Bridge (MIMB) Integrator Source.  See http://www.metaintegration.net/Products/MIMB/SupportedTools.html for a complete listing of sources supported.  There are a few exceptions in terms what is supported through Information Steward.  See the Information Steward Product Availability Matrix on SAP Support Portal for exceptions.

Note that the user needs to have the correct license/key code to leverage MIMB through Meta Integration (MITI).

Is there any way to define a filter around what metadata is crawled within a source? 

For the SAP BusinessObjects Enterprise Metadata Integrator Source, run-time parameters can be used to collect a subset of objects.  You can specify the names of the folders that you want to include in the collection as well as select what metadata objects you want to collect (e.g. choose from Universes, Web Intelligence documents and source Universes, Desktop Intelligence documents and source Universes, Crystal Reports and associated Universes, Dashboard Objects and associated Universes.  The SAP BusinessObjects Enterprise Metadata Integrator also provides the runtime parameter, “collectRptFldVar,” to enable or disable the collection of field level metadata for Crystal Reports and Web Intelligence documents.  Field level metadata includes report fields, formula fields, SQL expression fields running total fields and variables.

The SAP NetWeaver Business Warehouse Metadata Integrator Source provides runtime parameters to filter the queries or workbooks to collect.  Options include Query Name Expression and Workbook Name Expression.

When configuring the SAP HANA Integrator Source, you can limit the metadata collection to a named schema or a named package(s). 

  • Table Schema: The software collects all tables and views that belong to the specified schema. If a view in the specified schema is a Relational view or Information Model view that references a different table schema, the metadata from the referenced table will also be collected.
  • Packages:  Limits the collection of HANA Integrator source configuration to the named package or packages and any child packages.  To enter multiple package names, separate each name with a comma.

See the Information Steward Administrator Guide for full details, available on the SAP Help Portal.

Is it possible to extract only the changed / newly updated metadata into the Information Steward Metadata Repository?

For the SAP BusinessObjects Enterprise and SAP PowerDesigner Metadata Integrators, it is possible to incrementally collect CMS and physical data model metadata using selective collection runtime parameters.  Use the “Update existing objects and add newly selected objects” option to collect a subset of CMS and physical data model metadata based on the changes made since the last run.  Using this option may reduce processing time.

See the Information Steward Administrator Guide for full details, available on the SAP Help Portal.

Is it possible to view multiple versions of metadata and each versions respective impact and lineage diagrams in SAP Information Steward?

No.  Metadata will get updated with get new, subsequent run of Information Steward’s Metadata Integrators, but there are not currently versioning capabilities or the ability to manage multiple versions within Metadata Management.

Can the SAP BW system to be crawled by on any platform (e.g. Windows, Linux, UNIX, …)?

Only the Information Steward Metadata Integrator component needs to be installed on Windows, SAP NetWeaver BW can be on any platform.

See the Information Steward Product Availability Matrix on SAP Support Portal for details.

Does Metadata Management support automatic extraction of metadata when metadata changes in its respective sources?

Metadata Integrators need to be scheduled to crawl the source system(s) in order to keep the central metadata repository within Information Steward in synch with source systems.  Note that it is possible to schedule based on system events.

Will database stored procedures extraction include metadata relationships (e.g. within a stored procedure if a number of tables are called, will these table relationships be identified)?

Stored procedures are captured as metadata objects. The stored procedures are NOT parsed.  Therefore, no relationships are derived from the implementation of stored procedures.  

I have received some errors when leveraging the Teradata Metadata Integrator.  What do they mean and how can I fix them?

Error in [processJoin]. Left Table : [UD464_BOBJ_USER_ARCV2]. Right Table : [UD464_BOBJ_USER_GRP_XREF_ARCV2].  (CMM-10117)

  • How it limits what we collect: We didn’t collect any details about the particular join.  The join won’t be listed and there is no relationship established between the columns in the join.
  • How you can fix it: The join is not defined correctly in the Universe, you need to open the universe in Designer and check the joins between the tables that are reported in the warning message, fix that and export the universe back to CMS and run the integrator again

Unable to parse SQL for derived Table [derivedTableName]. Error: Encountered …Base tables will not be associated with the derived table. (CMM-10008) (CMM-10117)

  • How it limits what we collect: No relationship will be established between the derived table and base tables.  Impact/Lineage won’t go beyond the derived table because of this.
  • How you can fix it: The syntax in the cases in the log were Teradata-specific syntax.  There isn’t anything they can do to fix that if Teradata syntax is needed.  We’re limited to ANSI standard SQL.  We have supported a couple of common database-specific syntaxes, but it isn’t our standard practice. 

Parsing failure for derived Table [derivedTableName]. Unable to find tables associated with columns [columnList]. (CMM-10019) (CMM-10117))

  • How it limits what we collect: The derived table was parsed and most relationships were established.  If it can’t find the named column in any of the tables being used, this error is listed.  No relationship will be established between the column and the base table column.  Impact/Lineage won’t go beyond the particular column reference.
  • How you can fix it: In this instance, there is no issue.  We interpreted some Teradata keywords as columns (DATE, CURRENT_DATE).  Since they aren’t columns, there is nothing to fix.

Does Information Platform Services (IPS or mini-BOE) support Metadata Management and Metapedia integration with BI Launch Pad/Info View reports?

Yes, with the 4.2.1 release the integration between Information Steward Metadata Management and Metapedia and BI Launch Pad is available via the standalone Information Platform Services (IPS) deployment as well as the full BusinessObjects BI Platform.

The metadata collection is incomplete or incorrect.  How do I proceed?

The Metadata Integrator ran successfully; however, within Information Steward the metadata seems to be missing (e.g. a specific universe object was not collected).  Typically, this is a problem associated with the source system (e.g. for a universe that CMS Integrator fails to collect, Information Designer Tool will fail to open it due to same error).  To troubleshoot this issue, verify the log file.  Even if the integrator ran successfully, it may have run into errors while collecting specific objects. The log file will contain the details.  Check the source system for the objects that integrator failed to collect. Troubleshoot the source system to ensure stable state of the source objects.

How do I access the Metadata Integrator log files?

Open the Central Management Console (CMC) and navigate to the Information Steward area and choose Metadata Management | Integrator Sources.  Select the appropriate Metadata Integrator.  Click the Action | History (upper left hand corner).  The Integrator History dialog will open with scheduling information.  To view the database log file, select Actions | Database Log in the menu bar of the Integrator History dialog.  Or, click the download link in the Log File column.

By default, Metadata Management writes high-level messages (e.g. number of universes processed, number of reports processed, and so on) to the log.  You can change the message level on the configuration page for the integrator source.

The impact/lineage is missing for the Data Services Metadata Integrator.  How do I proceed?

The Data Services Metadata Integrator ran successfully; however, the impact and lineage is not available within Information Steward.  This problem is typically associated with the fact that the Data Services Integrator did not collect column mapping, because the column mappings are not available in Data Services Repository.  To resolve this issue, calculate column mapping within the Data Services Designer.  To do so, go to the Object Library and right-click.  Select Repository | Calculate Column Mappings. 

Then, rerun the Data Service Metadata Integrator.

I am receiving a connection error, “Integrator not deployed.”  How do I resolve this?

When attempting to execute a metadata integrator, the log states, “Connection Error: Integrator not deployed.”  The cause is that the integrators are not installed or there is no IS job server set within the server groups.  If the integrators are missing, you need to do a repair install and install the integrators.  If the integrators are already available, then make sure that:

  • IS Job Server is enabled and running
  • Integrator Scheduling Service is selected in IS Job Server configuration
  • IS Job Server is member of each Integrator Server Group

What do I do if the Metadata Integrator runs out of memory?

If the metadata integrator run fails because the integrator runs out of memory, additional memory needs to be allocated in the Java Virtual Machine (JVM) setting.  To increase the memory allocation for the Java process, go to the Central Management Console.  Select Information Steward and from the Actions menu, choose Schedule.  Select Schedule | Parameters and modify the JVM Arguments parameter.  To set the maximum memory limit to 2GB, set the parameter “–Xmx” as “–Xmx2048m”.  Note that 64-bit integrators can leverage more than 2GB of memory.

I can’t see lineage in the lineage reports.  What could be the issue?

It is possible that the report lineage was not calculated and the MMT_Alternate_Relationship table is empty.  To resolve this issue, run the “Compute Report Lineage” utility.  To do so, go to the Central Management Console and select Applications.  Click on Information Steward Application and select Manage | Configure Application.  Select Manage Utilities, highlight “Compute Report Lineage” and select Actions | Schedule or Run Now.

I cannot create new Erwin Metadata Integrator, seems that the file is too large.  Help?

You need to update the parameter ‘uploadMaxFileSize’ in web.xml, see SAP Note 1694907.

The metadata integrators are stuck in a "Pending" status, how do I get it to start?

The java variables are required to be properly set for the user that runs the SIA, see SAP Note 1566736.

I am having issues with the Meta Integration Model Bridge Integrator Source (e.g. MITI or MIMB Integrator), any tips?

Some things to consider:

  • One of the common reasons for Meta Integration Model Bridge Integrator Source failure is incorrect bridge configuration parameters.

--  Read the description of the bridge parameters and makes sure they are correct.

--  Look in the log file to see the same parameters are not getting corrupted.

--  Some bridges ask for home directory (e.g. OWB / ODI bridges). Please make sure you give the correct home directory for the product (e.g. the home directory for an Oracle database is not same as OWB).

  • If the MIMB integrator run fails with an error associated with source connectivity, this could be due to the fact that the appropriate vendor specific client is not installed on the same machine where the MIMB integrator is running.

--  Refer to MITI website to find out whether you require a client.

--  Check whether the client is installed on the same machine as the MIMB integrator.

--  Use the client tool on the machine to logon to the source application using the same connection parameter that you used to configure the integrator.

  • For SAP, MITI has customized ETL bridges that give out only a summary view, e.g. Source-to-Target mappings at a column or object level. The transformation details are not available.
  • For most of the Microsoft technologies - like SSIS and SSAS – the MIMB integrator uses Windows Authentication. Even if you give the user name or password in the configuration screens, they are not honored if the integrator service is launched from a service which is running in the system account.  Make sure SIA is running under the appropriate account that has permissions to these source systems or databases.
  • Some of the MIMB bridges have two flavors:  metadata collection from an application (application repository) or metadata collection via an exported file.  If you have problems with one flavor, try the other one.  When getting in touch with MITI’s support team it is better to give them an exported file, so they can easily reproduce the issue.
  • If MITI metadata integrator ran successfully but Information Steward doesn’t display the expected metadata, it is typically a MITI issue (that requires MITI support).

When troubleshooting MITI metadata integration issues, you can take a look at the integrator log files to get a better idea where the issue stems from.  Conceptually, there are three sections in the log:

  1. Pre-processing performed by Information Steward (core IS)
  2. Metadata collection by the MIMB integrator source (MITI)
  3. Post-processing by Information Steward (core IS)

If you see “Running MIMB...” this means that you have entered section #2.  If you see “MM Post processing...” this means you have entered the section #3.  If the integrator failed in section #1 or #3, it is an Information Steward issue. If the integrator failed in section #2, most likely it is MIMB integrator error.

Here are some sample MIMB integrator issues and there associated notes:

  • The MIMB integrator run against an Informatica source fails with error “Failed to spawn the process: 'The system cannot find the path specified.’”

--  MITI is expecting encrypted password. Encrypt the password appropriately or avoid using password encryption.(SAP Note 1694907)

The MIMB integrator against an Informatica PowerCenter repository source does not work in a Japanese environment

--  MIMB only supports pmrep.exe English version. Informatica PowerCenter (File) bridge can be used to import metadata in other environments. (SAP Note 1417525)

  • The MIMB integrator run against an Informatica PowerCenter repository failed with error code ERROR_IMPORT.

--  The MMT_Configuration table in the Information Steward Repository needs to be cleaned up. (SAP Note 1409399)

  • Unable to run MITI integrator to retrieve the metadata exchange workflow from Oracle Warehouse Builder

--  Make sure valid ‘Repository Subset’ parameters have been defined. The only valid parameters allowed in this field are 'source', 'target', 'mapping' or 'workflow'. (SAP Note 1572004)

Metapedia Questions

Is Metapedia a suitable solution to document data standards?

Yes, Metapedia intended to be a business glossary, which could very well be data standard definitions.

Can we add new columns to Metapedia for additional documentation?

Yes.  This is accomplished by adding custom attributes.  To do so, click Manage | Custom Attributes in the main top menu bar of Information Steward.  Note that the logged on user must belong to either the Data Insight Administrator or Metadata Management Administrator group to have permissions to add custom attributes.

When adding custom attributes to Metapedia, can we configure drop-down values for some columns to restrict the allowable data entry?

Yes.  When adding a custom attribution, the Valid values (attribute name) dialog box supports the following options:

  • Any value – Enter any value for the attribute (default)
  • Range - Enter minimum and maximum values to limit the value to a range. For decimal types, enter a zero before the decimal point.
  • List of values - Enter a value and click Add to add the value to the list. Repeat for additional values that the attribute will accept.

Is it possible to activate an approval workflow as part of governance process when there is a need to change these standards?

Yes.  After a term has been created (manually or through an Excel file import), it then goes through an approval process, where a person with approval rights either rejects or approves the final terms.

Can we create and manage versions in Metapedia?

For example, when a change needs to be made to a standard, a new version should be created so that data stewards can track how standards have evolved over time using version history.

Yes.  The term can be reverted back to the last approved version. For example, if a term was changed and approved three times, it can only be reverted back to the third approved version. It cannot be reverted to the first or second approved versions.

The following steps will revert a term to the previously approved version:

  1. From the Metapedia tab, highlight the term you want to revert, and then Edit.
  2. Optional) Click View History to review previously approved definitions and the dates of term progress and states.
  3. Click Revert, and then enter comments. Click OK.
  4. Review the term to ensure that it is the version you want. To make additional changes to the reverted definition, click Modify. Otherwise, click Close.

How does search work?  Can users search on a string and can the tool search through all the content?

Yes. Metapedia searches all terms and its various attributes such as description, synonyms, keywords, custom attributes, etc.  The advanced search feature allows you to add multiple conditions to narrow your search results. You can define each condition by object type, attribute, and match method.

Data Insight Questions

How do I validate for allowable characters in a field?

You can use match_regex or a string function to check for valid or invalid characters in a data column.  For example:

  • Not allowed: match_regex($parameter0, '[^#$!%?+:;|_~\-\\\[\\\{^*.]', NULL).  Note that double escapes are required on the brackets [], {}.
  • Allowable: match_regex($parameter0, '[a-zA-Z]’, NULL)

How should we setup projects and scorecards within Data Insight?

How projects are setup within Information Steward is up to you and your organization’s goals for data quality and information governance.  Note that each project may have multiple scorecards.  Some examples of project and scorecard setup include:

  • A single Information Governance project with scorecards per data domain (e.g. customer, supplier, product, material, plant, etc.)
  • Multiple projects per data domain (e.g. customer, supplier, product, material, plant, etc.) with scorecards within that project organized by sub-domains (e.g. for customer, scorecards for existing customers and prospective customers and for material, finished goods, semi-finished and raw materials)
  • A single Information Governance project with scorecards per business process
  • A single Information Governance project with scorecards per data source
  • Multiple projects per data source with a single scorecards for that data source
  • Etc.

Again, it is really dependent on the scope and maturity of your data quality initiative or information governance program as well as the visibility requirements for the initiative or program.  If you are just starting up an information governance program and your customer data domain comes first, you might want to create a “Customer” project with scorecards per source of customer data or with a single scorecard for your customer master data repository or data warehouse (or both).  If you are performing a data migration, you might want to create a “Migration” project with scorecards for each source being migration as well as the target data store (or maybe just the target).  If you have a more mature information governance organization working across multiple lines of business, you may want to organize scorecards so that LOBs can gain insight into the quality of the business processes that impact them the most.

In other words, no straight answer to this question; however, Information Steward allows flexibility to organize projects and scorecards in a way that meets your business needs.

How are data quality scorecards built?   Are they based on projects?

Yes, data quality scorecards are based on projects.  Note that each project may have multiple scorecards. 

Can a rule be shared between projects?

With the 4.2.1 release of Information Steward, rules can be shared between projects by promoting an approved rule to “Public”.

Note: In previous versions, all approved rules became public or visible in all projects.

Platform / General Questions

What are the minimum number of CPUs/cores required to support Information Steward?

Does Information Steward support this database or that application source?

What operating systems is Information Steward supported on?

Check out the latest Information Steward Platform Availability Matrix for information on:

  • Product infrastructure dependency
  • (Minimum) Hardware requirements
  • Operating system support
  • Language and web browser support by OS
  • Java application server support by OS
  • Application connectivity support by OS
  • DBMS connectivity support by OS

See the Information Steward Product Availability Matrix on SAP Support Portal for details.

What is the recommended deployment of Information Steward, Data Services and Information Platform Services (IPS) / BusinessObjects BI Platform?

With the Information Steward 4.2 release comes the recommendation to install both Data Services and Information Steward with a dedicated (for DS and IS only) installation of the Information Platform Services (IPS), version 4.1 SP2.  For more information, see the SCN article “Upgrading your SAP BI system? Installing Data Services/Information Steward? Here are some things you’ll want to know” http://scn.sap.com/community/enterprise-information-management/blog/2013/09/05/upgrading-your-sap-bi-system-installing-data-servicesinformation-steward-here-are-some-things-you-ll-want-to-know.

Do you have any tips for installing Information Steward?

Check out the Information Steward Installation Tips and Tricks wiki page. 

Check out SAP Knowledge Base Article 1570523 on Information Steward Installation Troubleshooting Tips.

In addition, here is a video on the basic steps for installing Information Steward 4.x that can serve as a useful resource.

Can Information Steward and Data Services share a single database repository?

Yes, they can share a single database server instance, but differentiated by database or database schema.

Does Data Services and Information Steward have different release and patch schedule?

No.  SAP releases Data Services and Information Steward together and maintains compatibility between these products and supports some cross-product use cases, so it’s recommended to keep them in sync and upgrade them together. 

Do you recommend deploying Information Steward completely separate from Data Services?

Architecturally, when it comes to scaling up Data Services and Information Steward for its target volumes, job frequencies, etc., Information Steward has a dependency on Data Services Job servers. From a hardware resource allocation perspective, you should ideally have a dedicated (group of) Data Services Job Server(s) assigned to Information Steward jobs and keep this group of job servers separate from the group of Data Services Job servers that would perform standard ETL jobs. 

From a release and shared functionality perspective, we recommend that you keep Data Services and Information Steward releases in sync. 

Do you recommend leveraging a shared BI Platform or a standalone Information Platform Services (IPS) when deploying Information Steward?

See http://wiki.scn.sap.com/wiki/display/EIM/Information+Steward+Deployment+-+Guidelines+on+sharing+BI+Platform+Vs.+Standalone+Install for recommendations.

Can Information Steward be used in a virtualized environment?

Yes, many customers have installed Information Steward on Virtual Machines (VMs).

Does Information Steward support SAP Authentication?

Yes.  If your user profiles for users are already defined in SAP, Information Steward will be able to reuse your SAP Authentication.

How can we setup roles for Information Steward?

The Central Management Console (CMC) manages security information, such as user accounts, group memberships, and object rights that define user and group privileges. When a user attempts an action on an Information Steward object, the CMC authorizes the action only after it verifies that the user's account or group membership has sufficient privileges.

Information Steward provides pre-defined user groups that have specific rights on objects unique to each module. These user groups enable you to grant rights to multiple users by adding the users to a group instead of modifying the rights for each user account individually. You also have the ability to create your own user groups.

Use SAP BusinessObjects Business Intelligence Platform or Information Platform Services (IPS or mini-BOE) security to create users and authorize user access to the objects and actions within the Information Steward modules:

  • Data Insight
  • Metadata Management
  • Metapedia (under Metadata Management in the Central Management Console)
  • Cleansing Package Builder
  • Match Review

See the Information Steward Administrator Guide (chapter on Users and Group Management) for more information.

How do I schedule an Information Steward task to run after a Data Services job is complete?

Check out the following wiki page:  How to schedule Information Steward task to run after Data Services job is completed?

Note that as article explains, there is limitation with Information Platform Services (IPS).  Since EventServer is not part of IPS package, system events cannot be used.  

What type of recommendations do you have for development, test and production environment setup?  What is the common setup to be able test out validation rules before implementing them?

Development, quality assurance (QA) and production environments are very important for Data Insight and the development of validation rules and scorecards.  Metadata Management and Metapedia are typically run in production environments only.  In terms of Cleansing Package Builder and Match Review, there is a Data Services dependency to consider.  Cleansing Package Builder is a development environment (for the development of custom cleansing packages and making updates to the SAP-supplied person and firm cleansing package).  Cleansing Packages are then published for usage with Data Services (either dev, test or production environments), so would follow the Data Services lifecycle.  Match Review is dependent on Data Services Match transform output, so whether the review is happening in a dev, test or production environment is really dependent on the Data Services deployment. 

Deployment varies from organization to organization, depending on size of the organization, complexity of their landscape, maturity of their information governance program, IT policies/processes in place, etc.  Some have a Dev, QA and Prod.  Some have only a Dev and Prod.  And others have a sandbox, Dev, QA, benchmark and Prod.

When migrating SAP Information Steward between landscapes, ensure that same version of each product is installed and configured in all landscapes.  However, the landscapes themselves do not need to be identical.  For example, in your source landscape you might have an Oracle database for your Information Steward repository and a WebSphere web application server. Your target landscape might have a SQL Server database for your Information Steward repository and a Tomcat web application server.  You can migrate many objects at the same time using the Promotion Management tool in the Central Management Console (CMC).  When you use Promotion Management for migration, all the information associated with the object, including user security is retained.  Otherwise, you can migrate smaller objects through the Information Steward import and export feature.  You can export projects and objects from that system and import them into the production system to make the projects and objects available in the production system.  For more information, see the Information Steward Administrator Guide’s chapter on “Migration between landscapes.”

My Information Steward repository failed during installation, what might be the problem?

There may be an incompatibility between your Information Steward installer and BI Platform (BIP)/Information Platform Services (IPS) version.  Check out SAP Knowledge Base Article 1767164.

Note:  Knowledge Base Article 1740516 contains more specific information about compatibility between Data Services and Information Steward and the SAP BusinessObjects BI Platform and Information Platform Services.

How do I migrate my Information Steward repository (4.1 and up)?

See SAP Knowledge Base Article 1773657 on how to migrate your Information Steward repository to a new DB server.

How do I configure my Data Services job server for Data Insight?

If you are getting an error that "No JobServers are configured for ISJobServerGroup," you need to configure the Data Services job server.  See SAP Knowledge Base Article 1648053 on how to configure a Data Services job server for Data Insight.

 

  • No labels

3 Comments

  1. Former Member

    Hi Experts,

    I have two questions on the Information Steward tool

    1. Does it Store Data as in Master Data for reporting?
    2. Does the tool provide the ability to send the data back to the source in order to correct it?

     

    Thank in advance,

     

    Cheers,

     

    Rabie 

     

  2. Best Practice re Metadata Management Administrator Role Assignment:

    What are best practices re what group is given the administrator role in Information Steward’s Metadata Manager toolset? Is this role typically granted only the SAP BASIS team or are Metadata Manager administrator rights also granted to individuals in the IT organization owning and administering the toolset?

  3. Former Member

    Hi Rabie,

    1. Does it Store Data as in Master Data for reporting-- Yes it store data in the database with tables names starting from MMT like below, you can create your own Views on top of these tables and then make use of it for reporting.
    2. Does the tool provide the ability to send the data back to the source in order to correct it? No. You can anyways create the report on Failed data and then publish or broadcast the data back to the users so that they can correct it.