SAP 3D Visual Enterprise Generator provides several ways of producing visual data enriched with semantic information out of engineering data that originated from CAD systems. The Visual Data Integration (VDI) workflow provides a platform for storing this data into SAP ERP/DMS.
The Visual Data Integration (VDI) workflow processing pipeline consists of a fixed sequence of data processing stages that are implemented as 3D Visual Enterprise Generator Workflow Controller steps. Each processing stage transforms some form of input data into some form of output data.
Input into the processing pipeline is usually in the form of engineering CAD data and a set of attributes, or a recipe/ set of instructions on how to retrieve engineering CAD data and the corresponding set of attributes from an external system. Output data produced as the output of the final processing stage of the pipeline is typically a set of viewable files that contains the correct set of attributes stored in SAP ERP/DMS document BOM.
Each 3D Visual Enterprise Generator Workflow Controller step representing a data processing stage has a corresponding 3D Visual Enterprise Generator process that executes a sequence of operations on the supplied input data in order to produce output data expected by the next processing stage. Since 3D Visual Enterprise Generator processes are represented in the form of xml scripts that can be modified using the 3D Visual Enterprise Generator Process Designer, each data processing stage can be further customized by modifying its corresponding script
Core Concepts and Terms
What Outputs are created as a Result of the VDI Workflow?
The Visual Data Integration workflow converts source CAD and PLM or PDM engineering data into a collection of tagged viewables, storing those viewables and accompanying metadata into hierarchically organized SAP DMS document structures and accompanying SAP ERP objects. In most cases, the resulting viewables and structures are used for many different purposes in various manufacturing processes.
VDI supports DMU (Digital Mockup)
The VDI workflow creates a single static viewable file (.rh or .vds) for each DIR. This is required to support DMU. DMU determines the viewable to display by locating documents with the specified material master object link. The assumption is there will be only one viewable file on the resultant DIR.
Mapping a PDM CAD Object to a DIR
The most common way in which engineering CAD and PLM or PDM systems store data is as sets of separate files; part files that are traditionally used to represent individual simple components, and assembly files that are used to represent assemblies of simpler components that can be parts or assemblies. These sets of files can reside on disk or in a database, stored against a type of document object that can store additional semantic PLM/PDM data. This method of storing CAD engineering data can lead to ambiguities in cases where CAD parts or assemblies have multiple discrete configurations, or if their appearance and properties can change depending on the context in which they are used within an assembly – occurrence. This means that a single CAD part or assembly file on disk or in the database can have different geometrical representations depending on its occurrence in the assembly. The information used to modify the representation and give the occurrence specific appearance is typically stored in the assembly file that includes the context dependent component. As CAD files store geometrical data in form of parametric curves and surfaces, CAD packages can always apply context dependent information to individual components, so for a CAD package it is not necessary to store different occurrences in separate files. A CAD assembly structure also contains 3D positional information typically at every level, relative to the parent occurrence of the component. This way, positions in space of individual occurrences of parts and assemblies can be effectively manipulated by changing just the position of their parent occurrence. In most cases, positional information is stored in CAD files, and for subassemblies it can be context dependent if the subassembly itself is context dependent.
Viewables created by the VDI Workflow
Producing a viewable using the Visual Data Integration workflow from CAD data involves translating the parametric surfaces and curves for each individual occurrence in the assembly into an equivalent set of lightweight triangular meshes while preserving the exact same positions in world 3D space as in the source CAD assembly. If there are context dependent occurrences in the assembly that are coming from the same CAD file on disk, these are represented by different viewables. A single viewable cannot store more than one triangular mesh representation for an assembly or a part, so a single CAD file can correspond to multiple viewables if its geometrical appearance is occurrence context specific.
For viewables that contain triangular meshes and their assembly structures, we say that they are fully resolved because the file and the document that corresponds to a particular viewable can have only a single lightweight triangular mesh representation.
The Visual Data Integration workflow cannot produce viewables and SAP DMS document info record structures that are NOT fully resolved.
Determining the DIRs that will be Created
For source CAD files that can contain context specific assembly or part occurrences whose geometric representations can differ and that are represented by a single CAD file we say that they are NOT fully resolved. A consequence of this is that the Visual Data Integration workflow maintains viewable and SAP DMS document info record structures that correspond to different geometrical representation of CAD files depending on their occurrences – this structure may differ from the original set of CAD files structure on disk.
Customizing the Workflow using Extension Points
The workflow has been designed in the knowledge that there are many scenarios and a single out of the box solution will not suffice for many installations.
Triggering the VDI Workflow
There are several methods in which a VDI workflow controller job can be triggered:
SAP 3D Visual Enterprise Generator
SAP 3D Visual Enterprise Generator can
Whenever a file of a specific extension
The hot-folder approach can be used in
as the input message into the VDI workflow.
It is more limiting than the web service
|web service API|| |
SAP 3D Visual Enterprise Generator web
Submitting jobs using the web service
|VDI Upload Client tool|| |
Uses the Upload Client to manually load
The VDI Upload Client approach is easy
Visual Data Integration should be regarded as a platform providing a set of APIs to achieve transformation of engineering CAD data with attributes into viewable data with semantic information stored in SAP ERP DMS. Typically, business processes vary to a great degree between different customers so almost every implementation requires individual customization.
Due to the inherent complexity of the topic, you must perform detailed investigation of customer data, business processes and system landscape. Based on the results of this investigation, you must undertake careful planning prior to beginning the implementation, in order to ensure that the correct approach is applied. Conceptually, there are four phases in the process of VDI workflow deployment:
|Step||Action||Remarks / Subsequent Steps|
|1||Blueprint - Data Study Phase||Requirements Gathering|
|2||Blueprint - Planning Phase|
|4||Testing and Correction Phase|
Blueprint - the Data Study Phase
The purposes of the data study are as follows:
● Gathering requirements in terms of the supported inputs and outputs of the Visual Data Integration implementation
○ The set of CAD features that will be used as input for visualization
○ Attributes that can be used to maintain mapping between the documents of the external system that provides the source data and SAP DMS Document Info Records
○ How the external system will provide associations between the SAP materials and the documents of the source external system that are being visualized
● Understanding the customer’s landscape and business processes
The information gathered in this phase is used in the Planning phase to make decisions about the optimum configuration of the VDI workflow. These decisions will have a major impact on the overall success of the VDI implementation.
The Data Study phase involves gathering information on the following:
● Source PLM/PDM/CAD/SAP data
● Downstream usages of visualization data and SAP ERP business data produced by the VDI workflow
● Security, authentication, involvement, and accessibility of various systems in the landscape
● Criteria for successful implementation
Each of these topics is discussed separately, in the following sections.
Typically, engineering data is managed in a file/document management system. This system will be further referred to as External PLM or PDM System. Usually, each record corresponding to a file contains additional information relevant for engineering and business processes in the context of the PDM external system, and not just the CAD modeling that is typically stored in the file.
You must answer the following questions:
- What non-visual business or engineering process data from the External System needs to be present in viewables and SAP DMS?
- Is all the data listed in the answer for question 1 present and up-to-date in the CAD files that are used for creating viewables and SAP DMS document structures?
- If the data listed in the answer for question 1 is not present and up-to-date in the CAD files, from where can it be retrieved?
- What data can be used to uniquely and unambiguously identify an individual geometric representation of the CAD file across multiple conversions? (This will be further referred to as External PDM Identifier)
- What data can be used to uniquely and unambiguously identify between different versions of an individual geometric representation of the CAD file across multiple conversions? (This will be further referred to as External PDM version).
- The target SAP DMS document structure and the positions in space of relevant instances of subassemblies and parts should be taken from:
- The top level CAD file supplied to the conversion OR
- A file independent from CAD files that holds the assembly structure and position information extracted from the External System
- For every CAD assembly sent for conversion to the VDI workflow, it can be said that for each individual CAD file that is part or subassembly of that assembly:
- For every CAD file within the CAD assembly only a single geometric representation exists that is the same for every occurrence of that part or subassembly within the top level assembly OR
- There can be a CAD file within the CAD assembly structure with a finite, predetermined number of fixed geometric representations (configurations and family instances; for example, a standard stock bolt with different set of lengths), and the choice of a particular geometric representation depends on the particular occurrence of that part or subassembly within the top level assembly OR
- There can be a CAD file within the CAD assembly structure with an infinite number of geometric representations (deformable parts; for example, hoses, tubes and wires), where each geometric representation is computed based on parameters specific to individual occurrence of that part or subassembly within the top level assembly (flexible components)
- Is PMI information from the source CAD files required to be present in the resulting viewables?
- Are model-views or captures from the source CAD files required to be present in the resulting viewables?
- Are geometric items that are hidden in the CAD files required to be present in the viewables (if the answer is yes then do they need to be initially hidden or initially visible in the viewable)
- Is any of the construction geometry used to model CAD files required to be present in the viewables?
- Do exact CAD feature trees for each individual CAD part need to be present in the viewable scene?
- Are there any portions of the CAD/external system assembly hierarchy that do not need to be represented as individual VDI documents in SAP ERP?
- Are the viewables stored for child documents of the document corresponding to the top-level CAD assembly file showing the viewable:
- as if it was observed independently and outside of the context of the top level CAD assembly file?
- as if it was the first occurrence of that file observed in the context of the top level CAD assembly file?
- How is change managed by the External PDM system?
- Is in work data to be sent to conversion?
- How does the release lifecycle of the engineering data appear? (in terms of statuses)
- Are there situations when viewables or SAP DMS information should not be updated?
- How can the material master data be retrieved and associated with External System PDM Identifiers?
- Is there an expectation that the viewable files that already exist in SAP DMS are reused and that CAD files that were previously converted to viewables are not going to be accessed by the VDI workflow?
- The input data comes from SAP DMS using which of the following:
- CIDEON SolidWorks CAD partner integration
- DSC NX CAD partner integration
- .riess CREO CAD partner integration
- SolidEdge CAD partner integration
- CATIA CAD partner integration
- Inventor CAD partner integration
Downstream Usages of Viewables produced by the VDI Workflow
There are further applicable questions on the information that must be present in the viewables that will be delivered to the end user and what kinds of viewables are to be produced. The answers to these questions are usually driven by the downstream viewable content requirements.
You must answer the following questions:
- What ERP business data (SAP Materials, Document Attributes, Classifications and other SAP objects) should be present in the viewables?
- Is the material master information required to be present in viewables?
- Is there an existing material master, and if so is it the same SAP system that will store the viewables?
- If the material masters do not exist should they be created by VDI workflow?
- Is there a need to populate SAP document attributes on DMS document structures based on some input data? If so, you must specify the source data and the attributes that must be created.
- Is there a need to populate SAP document characteristics on DMS document structures based on some input data? If so, you must, specify the source data and the attributes that must be created.
- Are there any other SAP objects that need to be populated based on specific input data? If so, you must specify the source data and the objects to be created or populated.
- If the answers to any of the above three questions was yes, should this data be visible in the created viewables?
- What CAD or external system data needs to be present in the viewables?
- Under which metadata category or key names should the CAD/external system/SAP ERP data appear in the produced viewables?
- What 3D Visual Enterprise file format should be used to store the produced viewable files?
- What additional viewables and viewable derivatives should be stored in the document structures created by the VDI workflow?
Information about the System Landscape
This step involves gathering overall information about the system landscape. This includes the following, relating to various systems in the landscape:
You must answer the following questions:
- To which SAP ERP system and client will the VDI workflow be connected?
- What is the EHP version of the SAP ERP system to which the VDI workflow will be connected?
- How is the external system PDM data going to be sent to the VDI workflow?
- How will VDI workflow access the CAD files required to build viewables?
- How will the VDI workflow conversion job be triggered?
- Where the VDI workflow conversion job is triggered by a hot-folder or web service, what is the SAP ERP technical user that will be used by the VDI workflow to store data in ERP?
- Does any third party system require a status report about conversion results?
- Which SAP DMS Document Type will the VDI workflow use to create document structures?
- Which storage category will be used to store viewable files and derivatives?
- Is any third party system (other than external PDM system) involved in the process of retrieving the input data, creating or storing business data to SAP ERP?
- Is there an expectation that single-sign on will be used for authentication on the SAP ERP?
- Is access to ERP RFCs available to SAP 3D Visual Enterprise Generator?
Criteria for Successful Implementation
In a complex environment that involves multiple systems communicating with each other, it is important to define criteria upon which the implemented solution will be evaluated. The results of this evaluation can then be used to classify the success of the implementation. More importantly, if some of the results of evaluation cause the implementation to be unsuccessful based on the above described criteria, a specific set of actions can be undertaken in order to correct the problematic behaviors.
These criteria can be used as an acceptance test to demonstrate that the implementation works according to expectations – this is usually done together with a customer.
List of Common Criteria
The most common set of criteria for a successful implementation (test cases) of the VDI workflow involves the following:
- Identify relevant input data sets – ideally, there would be a set of representative documents in the external PDM system that exhibit most of the features and behaviors that VDI integration is expected to handle.
- Identify the expected resulting output for relevant documents in the input data set. This expected output can be a number of things; for example, document structure, contents of the viewable, various SAP DMS objects, web service calls to various external systems, or a user message.
- Carefully select the input data sets and expected resulting output. Preferably, these would be to be quick to process, simple enough to allow easy inspection of results, and complex enough to represent a large portion of the set of features the system is expected to handle.
- Identify the edge cases that the system cannot process correctly – these cases should be handled by the system with an adequate error or warning message.
- Selection of the input data sets should be driven by answers to the questions posed in the preceding sections
In a more complex solution, it is difficult to come up with all the criteria before performing the actual implementation. Further criteria can then be devised through an iterative process – an initial set of test cases that can be reasonably planned beforehand, and as the implementation progresses, the list can be progressively improved.
Blueprint - Planning Phase
The purpose of the Planning phase is to make correct decisions on how the VDI workflow will be implemented based upon information gathered in the Data Study phase.
The planning phase involves making further decisions, based upon the results of the Data Study phase:
Determining what VEG Workflow is the Correct Choice
It is very important to consider other available SAP 3D Visual Enterprise Generator workflows within the context of the problem the implementation is to solve prior to making a decision to provide a solution using the VDI workflow.
|Source PDM/PLM||CAD Format||VEG Workflow||Customizations||Where are the viewables stored|
|SAP PLM||PTC Creo (.riess)||DMS Assembly Build||Not necessary||Source CAD documents|
|CATIA5||DMS Assembly Build||Not necessary||Source CAD documents|
|SolidEdge||DMS Assembly Build||Not necessary||Source CAD documents|
|Autodesk Inventor||DMS Assembly Build||Not necessary||Source CAD documents|
|SolidWorks(CIDEON)||Visual Data Integration||Not necessary||Separate document structure|
|Siemens NX(DSC)||Visual Data Integration||Not necessary||Separate document structure|
|None of the above||Visual Data Integration||Likely required||Depends on the source|
|Non SAP PLM||Any format supported by VEG||Visual Data Integration||Likely required||Separate document structure|
Determining Input Data Structures
Ideally, the input data provided to the VDI workflow should contain all the necessary information required by the VDI workflow to create viewables, SAP DMS document structures and other selected ERP objects. There are three major types of data source. These are the following:
- CAD files
- External metadata accompanying the CAD files
- External metadata available for retrieval through enterprise services
Information that must be extracted from input data by the VDI workflow in order to produce viewable and document structure outputs is usually the following:
- Assembly structure and absolute positions of elements in 3D coordinate space
- Content of viewable files and presentation of 3D tessellated geometry in relation to the assembly structure
- Metadata associated to elements of the assembly structure
- SAP DMS document structure and associations of viewable files with SAP DMS document info records
- SAP DMS document info record attributes and classifications
- Associations of the output SAP DMS document info records and other SAP objects
Assembly Structure and Absolute Positions of Elements in 3D Coordinate Space
As described in the Core Concepts and Terms section, the VDI workflow creates the fully resolved structure of geometrical representations of the external PDM documents. You must make a choice at the beginning of the planning process as to which of the following two modes will be used to create assembly structures.
External PDM system supplies fully resolved assembly structure mode
If the external PDM system has knowledge of the internals of CAD files, and if it can guarantee that every single CAD file will contain only a single up-to-date geometrical representation (if the CAD assembly is fully resolved in terms defined in the Core Concepts and Terms section), it can supply the full assembly structure and positional information in a form of xml file that accompanies the CAD files. This xml file would then be used as a recipe to create viewables from CAD files using hierarchical structure and positional information specified in the xml file supplied by the external system. The same supplied assembly structure would be used to create or update SAP DMS document info record structures. This way of supplying the input data will further be referred to as “fully resolved structure supplied mode”.
Fully resolved assembly structure needs to be inferred from the CAD files that external PDM system supplies mode
In this case, the external PDM system has no knowledge of the internals of the CAD files and it cannot guarantee that each CAD file will contain only a single up-to-date geometrical representation. The VDI workflow infers the fully resolved assembly structure and positions of components in 3D coordinate space from the information stored in the supplied CAD files, using a file labeled as top-level assembly file by the external PDM system as the starting point. This way of supplying the input data will further be referred to as “fully resolved structure extracted from CAD files mode”.
Content of Viewable files and Presentation of 3D Tessellated Geometry in relation to Assembly Structure
In a case of “fully resolved structure supplied” mode, the content of viewable files is determined by the assembly structure supplied in the xml file and the CAD files. The assumption is that the CAD files do not contain multiple different geometrical representations.
In a case of “fully resolved structure extracted from CAD files” mode, the VDI workflow must perform the following:
- Analyze the CAD file that was labeled as the top-level assembly file by the external system to establish whether there is more than one geometrical representation present in the file
- Use input values and workflow configuration to decide the top-level CAD component occurrences that will be converted to viewables
- Based on the selected top-level CAD component occurrences, determine all the child occurrences that will be converted to viewables
- For each different CAD file occurrence that is selected for conversion to viewables, create a unique occurrence identifier that can be used to distinguish between this and other occurrences of the same CAD file in different contexts across the top level assembly file
Selection of 3D Visual Enterprise Generator CAD import options for the CAD format that is being converted will always have an impact on the contents of the viewable.
Metadata associated to elements of the Assembly Structure
In both “fully resolved structure supplied” and “fully resolved structure extracted from CAD files” modes there is often a need to associate metadata that is not read directly from the CAD files with the contents of the viewables. This metadata can accompany the CAD files, or it can be retrieved using various enterprise services. In the case of “fully resolved structure extracted from CAD files”, there can be additional complications involved related to determining how to correctly associate the companion metadata with the right occurrence of the CAD component especially if there are multiple different occurrences relating to the same CAD file.
SAP DMS Document Structure and associations of Viewable Files with SAP DMS Document Info Records
The resolution of viewable assembly structure in both modes should take into account the lifecycle of the supplied components in the source external PDM system. This means that the version of the component needs to be taken into account when determining the viewable assembly structure – you must ensure that each individual occurrence of a different version of a component gets a separate viewable and a separate SAP DMS document info record. Sometimes external PLM/PDM system documents can have special statuses that can limit what can be done with them; for example, if documents are released they cannot be modified. You can customize the VDI workflow to take into consideration these special statuses and to choose whether to update elements of fully resolved SAP DMS document structure. Once the fully resolved viewable assembly structure has been determined by using any of the above methods, the VDI workflow will persist it in an internal document. At that point, you can perform further workflow customizations in order to manage associations of the elements of viewable scene hierarchy with SAP DMS document info records. By default, the viewable assembly structure maps 1-1 to the SAP DMS document info record structure. You can further customize the elements of the viewable assembly structure that will be mapped to independent SAP DMS document info records.
In the case of some CAD features like “virtual components” or “phantom assemblies”, it is desirable that even if this component has its own substructure, this substructure is not reflected in the SAP DMS document structure and it is present only as content of viewables.
SAP DMS Document Info Record Attributes and Classifications
Depending on the requirements, the SAP DMS document info records corresponding to the fully resolved assembly structure might need to be populated with specific attributes and classifications. You can customize VDI workflow to infer this data from the input CAD files, accompanying metadata or metadata available through enterprise services so that the SAP DMS document info records corresponding to the elements of the assembly structure have the right attributes and classifications. The assignment of attributes and characteristics happens usually during or just after the fully resolved assembly structure has been determined from the input data.
Associations of the output SAP DMS document info records and other SAP objects
Depending on the requirements, you might need to associate the SAP DMS document info records corresponding to the fully resolved assembly structure with SAP materials and other SAP ERP objects. This is typically done by establishing DMS object links between the SAP document info records corresponding to the fully resolved assembly structure and other SAP ERP objects. The association of the elements of fully resolved structure that have corresponding SAP DMS document info records and other SAP ERP objects is typically done during or just after the fully resolved assembly structure has been determined from the input data.
A common scenario is associating fully resolved assembly structure SAP DMS document info records with SAP materials using material object links.
Determining the way in which the VDI job is Triggered
|3D Visual Enterprise Generator standard hot-folder|
SAP 3D Visual Enterprise Generator is
The hot-folder approach can be used in
A SAP 3D Visual Enterprise Generator
Submitting jobs using the web service
|VDI Upload Client tool|
The Upload Client is used to manually
The VDI Upload Client approach is easy
Choosing the Correct Input Message for the VDI Workflow
The VDI workflow internally uses the VDI input xml format to handle input data. This xml format supports either of the fully resolved structure modes.
VDI input xml format is publicly available, and is the default method of supplying input to the VDI workflow. However, if the external PLM/PDM system is supplying as input a file that is not in this format, you can convert the input file into the VDI input xml format in the VDI workflow job. This can be done through a standard extension point. If the input data is “fully resolved” as described in the Core Concepts and Terms section, the VDI workflow manifest will ultimately have a different content and structure than if the input data is not “fully resolved”. Each of these cases will be processed in a different way.
If there is a requirement to reuse viewables that have been built previously, and not to send all the CAD files every time, input data must be “fully resolved”. It is not possible to reuse previously built viewables if data is not “fully resolved” and if the VDI workflow needs to perform inference of the assembly structure from CAD files. In a reuse scenario, you can send batches of part files to be pre-built for later reuse.
Choosing the Correct VDI Workflow Configuration in VEG 8.0 GA and VEG 8.0 SP1
SAP 3D Visual Enterprise Generator is delivered with a few pre-installed VDI workflow instances. Each of these instances corresponds to a particular type of workflow. Some will work out-of-the-box, and some are provided as starting point and require further customization:
|Workflow Instance||Functional Description|
|Visual Data Integration (SAP PLM SolidWorks Integration)|
Supports SAP PLM SolidWorks integration. This instance
|Visual Data Integration (Resolved Structure)|
A starting point for “fully resolved structure supplied”mode –
|Visual Data Integration (Structure Extracted from CAD)|
A starting point for “fully resolved structure extracted from
|Visual Data Integration (PLMXML)|
A starting point for integration with PLMXML referencing JT
|Visual Data Integration (Upload Client)|
This workflow instance has been configured to work out-of-the
Choosing the Correct Set of CAD Import Options
Depending on the CAD file format, and requirements for content of the downstream viewables, the SAP 3D Visual Enterprise Generator CAD file format import options will affect the assembly hierarchy, content and appearance of viewable files. These import options must be chosen with great care, because changing them at a later point in time can cause modifications of the SAP DMS document structures even if there were no changes of the source CAD file.
Also, choosing to import CAD features that are not going to be used will have unnecessary negative impact on viewable file size and both conversion and viewing performance.
Determining Mapping between Source and Target Identifiers
This is the most important part of the VDI workflow, and it involves determining how the elements of the fully resolved assembly hierarchy map to SAP DMS document info records.
A mapping table exists in SAP ERP that stores a mapping between external PDM system identifiers and SAP DMS document info record document numbers. Based on the contents of this table, the VDI workflow knows if there is an existing SAP DMS document info record that needs to be updated corresponding to that element of fully resolved assembly hierarchy, or a new SAP DMS document info record needs to be created. When a new SAP DMS document info record is created, its external system PDM identifier is associated with the newly created document info record number in the mapping table.
An External PDM system identifier can be constructed from multiple data inputs corresponding to an element of the fully resolved assembly hierarchy in any way that respects the following guidelines. It is a three part key that consists of the items in the following table:
|Data Input||Functional Description|
|PDM Identifier||Uniquely identifies a document within the external system|
Uniquely identifies a particular version of a document within
Uniquely identifies an occurrence specific geometric
The external PDM system identifier must be chosen carefully so that:
- Different CAD files within the external system always have different document identifiers
- Different versions of the same CAD file always have the same document identifier but different version
- If a single CAD file contains multiple different occurrence specific geometric representations each of these should have a different Configuration identifier
- It must be possible to correlate the configuration identifier a specific occurrence within a CAD file (a good example is configuration name as in CAD file)
- If a single CAD file contains multiple different occurrence specific geometric representations and the Configuration identifier has not been specified, VDI workflow will provide one by generating a random string. A downside of this is that new SAP DMS document info records will be generated for these occurrences in each separate conversion process
- Document identifier and version for the same CAD file in separate conversion jobs at different points in time are the same
- There must exist at least one external system PDM identifier for a fully resolved assembly hierarchy
- An element of fully resolved assembly hierarchy must have an external system PDM identifier in order to appear as a part of SAP DMS document structure
- If an element of fully resolved assembly hierarchy does not have a PDM identifier it will only appear as content of the viewable if viewables that contain that element exist
An example of a valid external system PDM identifier would be an identifier obtained via an enterprise web service by passing several metadata values read from the CAD file, as long as it conforms to the requirements listed above.
Typically, configuring a VDI workflow will involve modifying the PDM identifier extraction extension point to return a PDM id based on input element of the fully resolved assembly structure.
Handling SAP Material Associations
SAP materials are key identifiers used for multiple purposes in SAP ERP. It is very likely that the VDI workflow will need to associate produced viewables and SAP DMS document info records with SAP materials. There are two key aspects to the material association process:
1. Determining the association between SAP materials and elements of fully resolved assembly hierarchy – this is a process similar to extraction of external system PDM identifiers, and it involves customizing an extension point within the VDI workflow that will return an SAP material id for a given element of fully resolved assembly structure. The association to SAP material is not mandatory.
2. Creating SAP materials if they don’t already exist in SAP ERP for elements of fully resolved assembly hierarchy – if the requirements are such that new SAP materials must be created for a given element of fully resolved assembly hierarchy, and an extension point exists that can be customized in order to create new SAP materials. This method should be used with care, as material management is often a very complex process that requires careful planning and has impacts on the functioning of the whole ERP system. Ideally, there would be an existing mechanism responsible for material management. Materials would never be created and the existing materials would always be used.
It is also possible to assign materials to the SAP DMS document structures created by the VDI workflow. In this case, materials do not have to be supplied directly to the VDI workflow – they will be retrieved during processing and associated with elements of fully resolved assembly structure that correspond to the SAP DMS document info records to which those materials were linked.
Determining the Business Data to be represented in Viewables
Viewable scene elements can have associated metadata. This metadata typically comes from CAD files, and can be controlled to a degree through the CAD importer’s import options.
By default, the VDI workflow adds the following metadata relevant for the processing session:
- External system’s PDM identifiers
- SAP DMS document info record numbers of the documents used to store viewables
- SAP material numbers associated with documents that are represented by the viewables
You can add additional metadata into viewables through an extension point. This metadata is not limited to the data currently available in the processing session (such as attributes and characteristics) – any data retrievable through accessible enterprise services from SAP ERP or other systems can potentially be added to the viewables. Metadata can also be filtered/prevented from becoming a part of the viewable.
Determining Business Data from Source PLM/PDM to be represented in ERP
By default, the VDI workflow stores SAP DMS document info record attributes, characteristics and SAP material object links against the SAP DMS document structure used to store the viewables.
Any SAP ERP object that can be managed through an RFC or a web service accessible to the VDI workflow can, in theory, be maintained through extension points that allow calling of arbitrary SAP ERP RFC or web service.
Systems other than SAP ERP can potentially be called to maintain their specific objects if these systems have enterprise services accessible to VDI workflow.
Customization typically involves the following:
- the objects to be created
- how exactly to create these objects
- relationships with other objects that need to be reflected
- Extracting this information from the data available to the VDI workflow through an extension point
- Using the extracted information to manage the actual objects through another extension point
Determining how to Retrieve the Input Data from the Source
You must determine the way in which SAP 3D Visual Enterprise Generator retrieves the input data from the source PLM/PDM.
The design of the VDI workflow allows for extension points to retrieve the CAD data and other metadata to be used as a source for viewables and fully resolved assembly structure. There are two conceptually different approaches for making this data available to the VDI workflow, described in the following table:
Data is “pushed” to the VDI workflow. In practice, this means
Data is “pulled” from external sources on demand by the VDI
The Implementation phase involves installation of SAP 3D Visual Enterprise Generator, configuration of the VDI workflow and all the necessary customizations required for successful end to end execution of the conversion process.
VDI Workflow Step sequence
A typical VDI workflow implementation performs the following sequence of actions:
- Obtaining adequate VDI input manifest xml and accompanying CAD files (usually performed in Prepare for Processing step, download (if required) is performed in Download Controller child job).
- Creating the VDI ERP Storage XML file from VDI input xml and accompanying CAD files (usually performed in Document Structure Creation step).
- Synchronizing VDI ERP Storage XML with the existing business data stored in ERP already associated with the external PDM identifiers present in the xml file (done in Create new DIRs and read existing DIR data step).
- Building viewables based on the VDI ERP storage XML file (viewables are built by Build Viewables Controller in a child job based on a Build Viewables Request document created typically in the Prepare for building Viewables request step).
- Synchronizing ERP business data with information from the VDI ERP storage XML file (usually done in Update DIR structures and properties data step).
- Storing produced viewables against associated viewable documents (usually done in the KPRO store child job).
- Optionally, a foreign system can request a machine readable job report from SAP 3D Visual Enterprise Generator listing the outcome and details of the performed job.
Installation of SAP 3D Visual Enterprise Generator
SAP 3D Visual Enterprise Generator 8.0 or higher must be installed on a machine that can access SAP ERP. For installation of SAP 3D Visual Enterprise Generator, see the SAP 3D Visual Enterprise Generator 8.0 Installation Guide.
Setting up the Connection to ERP
The VDI workflow relies on being able to communicate with SAP ERP
For this connection to work correctly, you must ensure the following:
- SAP GUI is installed on the machine on which SAP 3D Visual Enterprise Generator is running
- SAP GUI has all the connections that will be used to connect to ERP configured for user account on which SAP 3D Visual Enterprise Generator is running
After the above two conditions are fulfilled, the SAP systems that were configured in SAP GUI can be used in the SAP Integration/Connections pane and a new connection to that SAP system can be successfully created.
Please refer to: Setup VE Generator to SAP ERP system connection for more details.
The VDI workflow uses SAP ERP as a data storage system. In order for this to function properly, there are a few prerequisites that need to be present on ERP side.
Please refer to Setup SAP ERP system for storing viewables for more details.
Choosing the Correct VDI Workflow Instance in VEG 8.0 and VEG 8.0 SP1
There are several workflow instance types created by default. Some can be used with minimal or no additional configuration, and others are provided only as a starting point for further implementation, requiring additional customization in order to be used in production.
The workflow instance types created by default are as follows:
- Visual Data Integration (SAP PLM SolidWorks Integration)
- Visual Data Integration (Resolved Structure)
- Visual Data Integration (Structure Extracted from CAD)
- Visual Data Integration (PLMXML)
- Visual Data Integration (Upload Client)
The default instances should be duplicated and kept for reference, while the copies can be customized and used to do the actual processing. This is especially recommended for PLMXML, Resolved Structure and Structure Extracted from CAD. These are provided as examples rather than production ready implementations.
Basic VDI Workflow Settings in 3D Visual Enterprise Generator
Please refer to General VDI workflow settings for more details.
Providing the Correct VDI Input Manifest File
The VDI input manifest xml file is used to specify the input data for the VDI workflow job. It is usually treated as a starting point by the VDI workflow in the process of determining the fully resolved assembly structure and accompanying metadata; for example, external system PDM IDs and SAP material IDs. In some cases, the VDI input manifest is provided directly to the VDI workflow, and in others, another type of file is provided to VDI workflow and through an extension points it gets converted into the VDI input manifest file. The VDI input manifest xml file format has a well-defined xml schema that should be used to validate whether the file is correct.
VDI Input Manifest Structure
The VDI input manifest xml consists of the elements listed in the following table:
This is the root element of the file. It must have one child from the set of following elements:
The choice of this element determines the type of the input:
The PDMStructure element may have a single PdmDocumentsFromFile child element and a single Materials child element.
ViewUp and ViewRight: Optional attributes that allow you to specify the direction of the viewable scene’s up and right axes respectively. The value of these attributes should follow the convention “name of the axis (X or Y or Z)” followed by “POS” for positive direction or “NEG” for negative direction; for example, XPOS and YNEG. If these attributes are not provided, their values will be “ZPOS”for ViewUp and “YPOS” for ViewRight.
UnitOfMeasure: A required attribute, and its value is always “Millimeter”. This is to make it obvious that all the positional data supplied through this xml format is assumed to be in millimeters. Currently other units of measure are not supported.
NumberFormatCulture: A required attribute, and its value is always “Invariant”. This is to make it obvious that all the number data supplied through this xml format is assumed to be in en-US culture.
|BuildResolvedStructure||Passes a fully resolved assembly structure as described in section Determining Input Data Structures based on Investigation Results. It has to have a single child element PdmDocumentInstances that is used to specify top level instances for the fully resolved assembly structure. It may have at most one PdmDocumentsWithChildren element that is used to describe fully resolved assemblies in the fully resolved assembly structure.||N/A|
|BuildCADStructure||Passes information about all the necessary CAD files required to perform extraction of fully resolved assembly structure corresponding to the “fully resolved assembly structure inferred from CAD” mode described in section Determining Input Data Structures based on Investigation Results||TopLevelDocumentRefs: A required attribute, used to specify one or more references to ids of PdmDocumentFromFile elements. Specifies the top level CAD files to be used to extract the fully resolved assembly structure. This way, it is possible to specify multiple top level assem|
|BuildAllPdmDocuments||Marks that in the VDI input manifest file where it is present, all PdmDocumentFromFile elements are treated as fully resolved parts and processed as such. This mode is intended to be used in conjunction with BuildResolvedStructure element.||N/A|
|PdmDocumentsFromFile||Represents a collection of PDM documents that are associated with CAD files. This element can have any number of PdmDocumentFromFile child elements.||N/A|
|Materials||Represents a collection of Material elements – it can have any number of Material child elements.|
|Material||Represents an SAP material. Its parent is the Materials element|
Id: A required attribute used to uniquely identify each Material element within the scope of the VDI input manifest document. The value of this attribute can be used to reference any Material element.
MaterialId: A required attribute used to store the actual value of SAP Material identifier. Maximum length is 18 characters.
|PdmDocumentInstances||A child element of the BuildResolvedStructure element, used to specify instances of PdmDocumentWithChildren or PdmDocumentFromFile to be created as the top-level elements of the fully resolved assembly structure. It can have any number of Child child elements.||N/A|
|PdmDocumentsWithChildren||A child element of the BuildResolvedStructure element, used to represent a collection of assembly PDM documents that have child documents. This element can have any number of Child child elements, and it corresponds to “fully resolved structure” mode.||N/A|
|PdmDocumentFromFile||A child element of the PdmDocumentsFromFile element. It represents a CAD file to be used in the conversion process. Additional metadata associated to that CAD file can be represented by using any of the child elements: SapCharacteristics, SapDocumentAttributes, Metadata and MaterialRefs. Additional information on how to process this file can be supplied using the ProcessingFlags child element. This element cannot have children of type PdmDocumentFromFile – this is intentional because PdmDocumentFromFile assumes that the information from the CAD file should be used to resolve viewable assembly structure for that PDM document. If child PDM documents need to be specified for a PDM document explicitly, the PdmDocumentWithChildren element should be used, and this is only possible in BuildResolvedStructure mode. PdmDocumentFromFile can have either AbsolutePath child element if the CAD file they are pointing to can be found on a network or local file system location accessible using a specified path, or CustomPath child element if the CAD file they are pointing to needs to be located and retrieved using an extension point.|
Id: A required attribute of the standard xml xs:Id type. It is used to uniquely identify a PdmDocumentFromFile element within the scope of VDI input manifest xml. PdmDocumentFromFile and PdmDocumentWithChildren share the same id space, so it is not possible to have PdmDocumentFromFile and PdmDocumentWithChildren elements with the same value for the id attribute. The value of this attribute is used to reference documents within a VDI manifest xml document.
Name: A required attribute of type xs:string used to specify a name for a PDM Document. Maximum length of this attribute is 255 characters. PdmIdentifier: An optional attribute used to specify the external PDM system document identifier for a particular PdmDocumentFromFile element as described in Determining Mapping between Source and Target Identifiers.
PdmVersion: An optional attribute used to specify the external PDM system document version for a particular PdmDocumentFromFile element.
PdmConfiguration: An optional attribute used to specify the external PDM system configuration identifier for a particular PdmDocumentFromFile element.
Status: An optional attribute that can be used to supply additional information about the particular status of the document in the source PDM system; for example, “In work” or “Released”. The value of this attribute can be handled later in various extension points.
|Child||Represents a parent relative instance of a PdmDocumentWithChildren or a PdmDocumentFromFile. The Child element can be a child of PdmDocumentInstances or PdmDocumentWithChildren elements, and this means it can be used only in BuildResolvedStructure mode.|
ID: A required attribute of the standard xs:Id type. This attribute is used to uniquely identify any Child element within the scope of VDI input manifest xml document. The value of this attribute should not overlap with values of identifiers of other element types. The value of this attribute can be used to refer to a specific Child element.
DocumentRef: A required attribute of the standard xs:IDREF type. This attribute is used to reference an existing PdmDocumentFromFile or PdmDocumentWithChildren by specifying the value of its Id attribute within the scope of a VDI input manifest xml document.
TMatrix: An optional attribute used to specify parent-relative 3D transformation matrix for that Child element. This transformation matrix will be associated with this particular instance of the PdmDocumentFromFile or PdmDocumentWithChildren. In order to calculate the world position of a particular instance of a PdmDocumentFromFile or PdmDocumentWithChildren, all parent Child element TMatrix attributes beginning at the topmost Child element must be multiplied. TMatrix is a list of 12 double space delimited values stored in a single string that corresponds to a standard 4x4 3D transformation matrix where the column that represents perspective is omitted. If TMatrix attribute is omitted, the “identity” transformation matrix is assumed. If another transformation matrix is multiplied by Identity transformation matrix, the result is the value of the first transformation matrix
Represents a collection of SAP characteristics associated with a PdmDocumentFromFile or a PdmDocumentWithChildren elements. This element can have any number of Characteristic child elements.
For use in Python script, the type of SAPCharacteristics is SAP.VE.Generator.FileFormats.ERPIntegrationManifest.ViewModel.FromFileCharacteristics.
Represents the actual SAP ERP characteristics
SAP characteristics must exist in SAP ERP, and they must be associated with the SAP DMS document type that is used to store viewables. By default, this is VED.
For use in Python script, the type of Characteristic is SAP.VE.Generator.FileFormats.ERPIntegrationManifest.ViewModel.FromFileCharacteristic.
ClassType: This required attribute represents the SAP ERP class type. Maximum length is 3 characters. ClassName: This required attribute represents the SAP ERP class name. Maximum length is 18 characters.
CharacteristicName: This required attribute represents the SAP ERP characteristics name. Maximum length is 30 characters.
CharacteristicValue: This required attribute represents the SAP ERP characteristics value. Maximum length is 30 characters.
|SAPDocumentAttributes||Represents a collection of SAP DMS document info record attributes associated with a PdmDocumentFromFile or a PdmDocumentWithChildren elements. This element can have any number of Attribute child elements||N/A|
Contains information about SAP DMS document info record attributes
Only document attributes that have aCategory attribute equal to “DRAW” are supported by the RFC middleware used to manage SAP DMS document structures
Category: A required attribute used to represent internal category of SAP DMS document info record attributes. Currently, only “DRAW” is supported. Maximum length is 10 characters.
Name: A required attribute used to represent name of SAP DMS document info record attribute. Maximum length is 30 characters.
Value: A required attribute used to represent value of SAP DMS document info record attribute. Maximum length is 255 characters.
|Metadata||Represents any other metadata that needs to be associated with a PdmDocumentFromFile or PdmDocumentWithChildren. At a later point in processing, this metadata can be used to enrich the viewables, or it can be associated with some other data in SAP ERP through extension points. Metadata can have one or more MetadataGroup child elements.||N/A|
|MetadataGroup||Represents a collection of MetadataKeyValues – it can have one or more child elements of that type||Name: A required attribute used to represent metadata group or category name. All MetadataKeyValue elements under a MetadataGroup element will have the category name specified by this attribute. Maximum length 250 characters.|
|MetadataKeyValues||Represents a collection of MetadataKeyValue elements – it can have one or more child elements of that type||N/A|
|MetadataKeyValue||Represents a metadata key value pair|
Key: A required attribute used to represent a metadata key name. Maximum length is 250 characters.
Value: A required attribute used to represent a metadata value. Type is string, and maximum length is 4000 characters.
|MaterialRefs||Represents a collection of references to SAP materials for a PdmDocumentFromFile or PdmDocumentWithChildren elements. It has multiple MaterialRef elements||N/A|
|MaterialRef||References a specific material by using the value of the Material element’s ID attribute. It is a child element of MaterialRefs element.||N/A|
|ProcessingFlags||References a collection of ProcessingFlag elements, and used to specify additional behaviors for PdmDocumentFromFile and PdmDocumentWithChildren elements. It can have multiple ProcessingFlag child elements.||N/A|
|ProcessingFlag||Specifies additional processing behaviors for a specific PdmDocumentFromFile of PdmDocumentWithChildren. Processing flag is much like a property that a document either has or does not have|
Name: A required attribute that specifies the name of the processing flag
Value: An optional String attribute that specifies whether the value of the flag is true or false.
|AbsolutePath||Represents a network or local file system location where a file can be found from the SAP 3D Visual Enterprise Generator machine. This element allows for “any” attribute so that users can add additional custom data to it if required||Path: A required attribute that stores the file absolute path|
|CustomPath||Represents references to files that need to be retrieved using an extension point. The CustomPath element allows for a sequence of “any” elements.||N/A|
Example: BuildCADStructure with AbsolutePath Elements
Example: BuildCADStructure with CustomPath Elements
Creating VDI input manifest file for an external PDM system output
The recommended way to provide input for VDI workflow is through VDI input manifest xml file. There are two ways how this can be done.
Creating VDI input manifest as output of the external PDM system
The external PDM system can utilize some of its standard extension/export points to transform its export format into VDI input manifest xml conforming to the guidelines described in previous sections. The publicly available VDI input manifest xsd document can be used to verify if the generated VDI manifest xml conforms to the VDI input manifest schema. The produced file, if it is valid, can finally be submitted to VEG via web API or hot-folder.
Creating VDI input manifest from output of the external PDM system using VDI workflow extension point
In situations where it is not feasible to output VDI input manifest directly from the external PDM system, an extension point can be implemented in VDI workflow to perform transformation from the input format supplied by the external PDM system, to the VDI input manifest xml.
This transformation should almost always happen in the “Visual Data Integration – Input Transformation” workflow step. A good example how it can be done can be found in “Visual Data Integration – PrepareForProcessing” VEG process. A Python script callback operation of type “ei_transformInputIntoProcessingManifest” named “transformIntoProcessingManifest” is used for this purpose. In this script operation, a method “TransformIntoProcessingManifest” needs to be implemented that transforms the input supplied by the external PDM system into VDI input manifest.
Python scriptable operation “ei_transformInputIntoProcessingManifest” is the default solution for this task. It is possible to use any Windows executable that VEG can invoke that transforms the input file into VDI input manifest – an example would be invoking a third-party executable file that will perform xslt transformation from xml supplied by external PDM system to VDI input manifest xml.
Example: A python script to create VDI input manifest xml from Siemens PLMXML schema example available from Siemens website
VDI Processing Xml
There are a number of internal xml documents used by VDI workflow to create, maintain and store viewables and SAP DMS document structures. The most important one is the VDI processing xml, also known in the processes as “The Build Structure”.
VDI Build Structure is an internal xml document that represents the fully resolved viewable assembly structure – this document is used in almost all tasks the VDI workflow executes to produce viewables and manage SAP DMS document structures. This is why VDI Build Structure must always be created for any VDI workflow type but due to the differences in VDI workflow types, the process of its creation will vary.
The most important element in VDI Build Structure xml document is the Document element. This element, if it has a PDM identifier assigned, will correspond to a SAP DMS document info record in ERP and a viewable stored against it. Document elements without a PDM identifier will not be assigned to any SAP DMS documents.
VDI Build Structure xml will be used as a base for building of the viewable assembly structure by converting the CAD files referenced by it. Once the viewables are built, they will be associated to their corresponding Document elements so that they can be stored against the matching SAP DMS document info records.
Creating VDI Processing Xml for resolved assembly structure supplied in VDI input manifest xml
In case when VDI input manifest file contains the BuildResolvedStructure element, the fully resolved assembly structure is specified through PdmDocumentInstances element. In the “Visual Data Integration – Document Structure Creation” process the operation ei_ProcessingManifestIntoERPStorage transforms the VDI input manifest with BuildResolvedStructure into VDI Build Structure xml. The ei_ProcessingManifestIntoERPStorage operation has a Python callback method “PostProcess” that allows additional customization/modification of the VDI Build Structure after the transformation has been performed.
The transformation that occurs will produce viewables and SAP DMS document structures exactly as specified in the VDI input manifest xml. This will include positional information from the supplied transformation matrices and accompanying metadata such as SAP materials, document attributes and characteristics that were present in the VDI input manifest xml.
Creating VDI Processing Xml when resolved assembly structure needs to be extracted from CAD files specified in VDI Input manifest xml
In this case, VDI input manifest file contains the BuildCADStructure element and the fully resolved assembly structure will be inferred from the referenced CAD files. The situation gets more complicated because we are dealing with two structures – a “flat” structure of files provided in the VDI input manifest, and the “fully resolved assembly structure” that will be extracted from the CAD files. The resulting VDI Build Structure will contain a combination of data from both input structures.
The process “Visual Data Integration – Document Structure Creation” creates the VDI Build Structure from VDI Input Manifest and an xml file that contains the CAD structure (will be further referred to as “CAD structure file”).
CAD structure file is an xml file created from what is marked as the “Top level document” in the VDI input manifest file – this CAD file is being read by Visual Enterprise CAD readers, and its fully resolved assembly structure is exported to disk along with positional information and CAD metadata. The CAD structure xml file will contain references to all dependent CAD files and some internal data used in CAD conversion process. The creation of CAD structure xml file happens at an early point in the process, before any of the other extension points mentioned below.
The operation at the end of the process called “createBuildStructure” performs the actual creation of the VDI Processing xml file. This operation will take in consideration a number of auxiliary xml files in order to determine what elements from CAD structure file will become elements of the VDI Build Structure, and what metadata, attributes, classifications and SAP materials will be associated with them.
Each auxiliary xml file is a simple mapping that allows assignment of a different kind of data to elements of the CAD structure file: there are separate xml files for PDM identifiers, SAP Document Attributes, SAP Material Ids, SAP Classifications and general purpose metadata. These files are typically just lists that map an “Entity Id” of an element from CAD structure file to a set of properties. When the VDI Build Structure is generated by “createBuildStructure” operation, these xml mappings are used to associate additional properties to elements of CAD structure file.
A standard way of managing how the VDI Processing Xml gets created and what data gets assigned to its elements is by controlling on what gets exported into those auxiliary mapping xml files. Each of the mapping files can be created and modified by any tool that can manipulate xml, but the out-of-the box way to do this provided in the process is through various Python script callback operations.
Validating VDI Processing Xml
All information related to creating and maintaining viewables and SAP DMS document structures is being kept in the VDI Processing xml document. This document is being modified multiple times at different stages of VDI workflow processing. After some of these modifications, it is possible to add custom code that will assert and validate whether the content of the VDI Processing Xml conforms to specific business rules and requirements. This can be done in a few different workflow steps provided for this purpose. Custom validation code is not limited to these steps – it can be added to other steps that modify the VDI Processing Xml of this is required.
Standard extension point to validate the VDI build structure is the Python scriptable operation ei_validateBuildStructure. This operation has a Python callback method ValidateBuildStructure that has two arguments: buildStructure and integrationManifest. buildStructure will contain a VDI build structure object corresponding to VDI build structure xml, and integrationManifest will contain object corresponding to VDI input manifest xml. The actual script can analyze these two models and test if they conform to the specified business rules. If this is not the case, the call should raise an exception that will result in the entire job failing. The error message supplied to the exception will be displayed in the job report as the cause for job failure.
Implementing python extension points
Python extension points are usually VEG operations that execute a python script to perform a specific task. This task can involve:
- Performing an action on a specific file (for example, modify VDI Processing xml to add extra information involving business data)
- Implement a callback that returns a value that will be used at a later point in the process (for example extract PDM identifiers from input CAD data)
- Perform a set of actions to interact with one or more external systems (for example, download files to the job workspace)
- Customizing a behavior based on an event (for example, in case of a particular warning choose to emit an error and abort further processing)
Most of the operations in the workflow with potential to be customized are realized as python scripts/extension points that have a default implementation provided.
More information on how to use Python in VEG can be found in VEG SDK reference guide and VEG Python Developer guide.
Handling multiple context dependent geometrical representations of a single CAD file
As explained in “Core concepts” section, the viewables stored in SAP ERP have to be fully resolved. This means that each individual configuration needs to have a different corresponding viewable and different viewable SAP DMS Document Info Record.
A PDM identifier is used to identify each individual viewable – it is a three part key consisting of “PDM Identifier” string, “PDM Version” string and “PDM Configuration” string.
Since each external system source document can in theory contain multiple different geometrical representations depending on the context in which the document was used, the entire three-part PDM identifier structure will contain the shared PDM external system document identifier in the “PDM Identifier” section and the shared PDM external system document version in the “PDM Version” section that will both be the same for all different geometrical representations. The “PDM configuration” section of the three-part PDM identifier will be different for each different geometrical representation of that particular version of the PDM external system document.
If the external system supplied input structure is fully resolved (if the “Fully resolved assembly structure supplied” mode is used), then each of the supplied CAD files will contain only a single configuration and it is the responsibility of the creator of input xml to ensure that each file’s geometry is supplied correctly and that it has a correct matching PDM identifier. In this case, no additional work is done on the VDI workflow side to resolve the configurations. The results will solely depend on the correctness of the input data.
In cases when “fully resolved assembly structure extracted from CAD files” mode is used, the external PDM system who supplies the data will most likely be unable to guarantee that each of the supplied CAD files is “fully resolved”. The resolution of CAD files with multiple geometrical representations to fully resolved viewable files happens in VDI workflow: the actual CAD assembly structure will be read by CAD readers from the supplied input CAD files. VDI workflow extension points will be customized to ensure that these configurations are correctly identified and that a correct unambiguous 3 part PDM identifier is associated with each fully resolved viewable. The process usually consists of analysis of the assembly structure xml produced by the CAD readers (in the job workspace folder this xml is being referred to as CAD structure xml) and use of metadata associated with elements of the CAD assembly structure to resolve each of the geometrical representations for each file.
The extension point intended to be the place where the described resolution happens is the python operation for extraction of PDM ids in the Document Structure Creation step. This operation needs to be customized so that different geometrical representations of same CAD file have the same ‘PDMIdentifier’ and ‘PDMVersion’ values, and unique/different ‘PDMConfiguration’ values.
There is a special case when a specific assembly or part’s named configuration is context dependent. An example would be a part hose.prt that has three different configurations: RED, GREEN and BLUE. If the geometrical representation of the hose is driven by its color and shape, and if the hose is used in multiple places in an assembly, it’s shape can vary depending on the context in which it has been used across the assembly. So each of the different configurations (RED,GREEN, BLUE) will potentially have different geometrical representation. In order to get correct viewable for each of the representations, an additional field ‘Context’ can be provided next to the return value. This is so that different representations of the three basic configurations can be further distinguished. Effectively, usage of the ‘Context’ field will cause the first geometric occurrence of each configuration to be stored with ‘PDMConfiguration’ set to the configuration name (RED, GREEN or BLUE), while other successive geometric occurrences will be stored with a ‘PDMConfiguration’ set to a value inferred from the configuration name and the provided value of the ‘Context’ field.
Using Processing Flags
A processing flag is a helper element that can be added to VDI Input Manifest XML, VDI Processing XML and the accompanying XML mapping files. The purpose of this flag is to store information that is available and easily accessible in one processing stage in the XML document, so that it will be automatically carried over across different operations on the XML to the later stages of the processing pipeline where it can be accessed again.
Processing flag can be added to the <ProcessingFlags> element. In VDI Input manifest XML, the <ProcessingFlags> element can be added as a child element to the <PDMDocumentFromFile> and <PDMDocumentWithChildren> elements. In VDI ERP storage XML, the <ProcessingFlags> can be added as a child element of the <Document> element. In the xml mapping file that contains PDM identifier mappings, <ProcessingFlags> element can be added to the <EntityPdmId> element.
<ProcessingFlags> element will be, whenever possible, automatically copied to the corresponding element in the next stage.
The <ProcessingFlag> element itself has one mandatory string attribute : “Name” and one optional string attribute “Value”.
Since PDM identifiers are used to identify viewable document structures corresponding to documents supplied by external PDM system, if the external PDM system supplies a set of components that ultimately resolves to a set of PDM identifiers that matches a set of existing viewable SAP DMS document info records, the matching set of viewable documents in SAP DMS will be overwritten.
This is to illustrate that except for the logic that is responsible for determining a PDM identifier for an external PDM system supplied component, the entire responsibility for driving changes to the viewable document info records is on the external PDM system/input side. Externally supplied set of PDM identifiers (along with the logic for their extraction in VDI workflow) fully determines what set of viewable document info records will be updated.
There is no default mechanism that triggers updates of parent viewables and their corresponding documents if their children change – the responsibility for this is solely on the external system, or on an implementer who would come up with a custom solution for this complex problem.
For more information, please see Visual Data Integration - Handling changes in source data
Customizing file retrieval
If the CAD files that need to be used for conversion of viewables need to be retrieved from an external system, the default extension point for that is in the Download Controller. Download controller child job will be spawned by VDI processing job in order to retrieve the files from the external location.
The VDI input manifest xml schema has <CustomPath> element that can contain any child xml elements. The information required to download these files from external location should be added to this element. The script that creates the download request for the download controller can then populate the request with information required to retrieve the files (an example how to do that can be found in Input Transformation SAP PLM step, GenerateDownloadRequest python method of createDownloadRequest operation).
The actual file retrieval is implemented in a python script operation (an example is download_file in Download Files – Download File Batch (KPRO)) that implements a customized download from SAP KPRO content server.
This mechanism can be used for pretty much any file retrieval protocol that can be called from a python script running in VEG environment. This will of course involve additional customizations of the VDI workflow based on a specific use case.
More details on how this can be done can be found in the Download Controller documentation.
Reusing of existing viewables
Reuse of existing viewables that have already been built is possible only when the fully resolved assembly structure was supplied by the external system. Reuse will be driven by the fully resolved structure in the VDI input xml document.
The rules for reuse are:
If a <PDMDocumentFromFile> element is specified without an actual file reference, it’s PDM identifier will be used to retrieve a corresponding viewable if one exists. If the viewable does not exist, one with an empty node will be created and stored against an SAP DMS document corresponding to this PDM identifier. If a viewable corresponding to that PDM identifier already exists in an SAP DMS document, the existing viewable will be retrieved and used everywhere where that particular <PDMDocumentFromFile> has been referenced – it will not be re-built.
If a <PDMDocumentWithChildren> element is specified without children, it’s PDM identifier will be used to retrieve a corresponding viewable if one exists. If the viewable does not exist, one with an empty node will be created and stored against an SAP DMS document corresponding to this PDM identifier. If a viewable corresponding to that PDM identifier already exists in an SAP DMS document, the existing viewable will be retrieved and used everywhere where that particular <PDMDocumentWithChildren> has been referenced – it will not be re-built.
Making RFC calls to SAP ERP
VEG exposes SAP .Net Connector library that can be used to communicate with SAP ERP from python scripts. An example process that is using this is Visual Data Integration – ERP Unlock For Job operation ei_UnlockAllIdentifiersInERP.
The first line “dest = sap.rfc.GetDestination()” will retrieve the SAP RFC destination authenticated with SAP ERP credentials that the VDI workflow job is configured to use. This destination can then be used to make RFC calls via SAP .Net connector.
A more detailed example on how this is done can be found in the VEG python programmer’s guide.
More information on SAP .Net Connector library can be found at:
Enriching metadata content of the viewables
It is possible to enrich the metadata content of the produced viewables using a dedicated extension point. Viewables normally contain metadata read from CAD files, but this extension point allows for addition of arbitrary metadata to accompany the CAD one. The extension point for the conversion mode when the assembly structure is extracted from CAD is normally the extractViewableMetadata operation in the Build Structure Creation step. See section 220.127.116.11 for more details.
Creating SAP materials
Ideally, SAP materials would already be present in the SAP ERP when the VDI workflow produces viewables and SAP DMS document structures. However, if this is not the case, it is possible to use a VDI workflow extension point to create SAP materials. It is the implementer’s responsibility to supply the correct parameters to the material creation function, and to ensure that no excess or duplicate materials for the same thing are being created. Typically, this will happen in the Create new DIRs and read existing data step – the operation createMaterials would be the extension point that, if the right data is returned from the ExtractMaterialCreationData callback implemented by the user, SAP materials will be created based on supplied parameters.
The python callback ExtractMaterialCreationData will be called for every document in the VDI Processing xml file.
In the example below, if the document for which the callback is being called has existing SAP materials, no new SAP materials will be created. Existing materials can come from Create Document Structure step where they are extracted in extractMaterials extension point operation, or they can be assigned to the VDI Processing Xml earlier in Create new DIRs and read existing data step if the document elements in the xml are already represented by SAP DMS Document Info Records and those SAP DMS Document Info Records have SAP material object links, then these materials will be added to the VDI Processing Xml and will appear in the “materials” argument of ExtractMaterialCreationData callback. This can be used to detect existence of SAP materials.
The input arguments to the callback present various different objects related to the document element for which the callback has been invoked. These are described in more detail in the section 6.24
If new SAP material (or materials) are to be created, the return data structure needs to contain a list of dictionaries containing the following keys:
MaterialNumber – if set to None, SAP will generate a new SAP material number. If supplied, the value needs to conform SAP ERP rules for material number (maximum 18 characters)
IndustrySector, MaterialType – need to be set to valid industry sector and material type in SAP ERP
CustomAtributes – this is an array of dictionaries containing Category Name Value triplets that can be used to pass system specific SAP object fields. Standard set would be Category:MAKT Name:MKTEXT Value:”specific material name” for material name and Category:MARA Name:MEINS Value:”EA” for quantity. Consult CDESK CAD services documentation for more details – the RFC function being used is CDESK_SRV_CREATE_MATERIAL and everything mentioned in related documentation applies to the parameters returned from the ExtractMaterialCreationData callback.
Note : all the SAP material creation fields required by the SAP ERP that is being called need to be supplied in order for material creation to be successful.
Creating links between SAP DMS document info records and other SAP objects
An extension point exists to create SAP Document Object links between the viewable SAP DMS Document Info Records and other SAP ERP objects is usually located in Update DIR structures and properties step, in the batchStoreProperties operation.
There are two callbacks : ExtractDocumentObject and ExtractLinkObjects, each of which receives two arguments: documentData and allDocumentData. The values of these arguments are guaranteed to be the same for a document element from the VDI Processing xml.
The documentData argument object will have the following properties:
- Key – this is the value of the Id attribute of the Document element from the VDI Processing XML
- Name – this is the value of the Name attribute of the Document element from the VDI Processing XML
- Attributes – this is a list of Attribute elements associated with the Document element whose Id attribute has value equal to the value of Key property. Each Attribute object will have Category, Name and Value string properties. These correspond to SAP Document Info Record attributes associated with the SAP DMS Document Info Record represented by the Document element
- Characteristics – this is a list of Characteristic elements associated with the Document element whose Id attribute has value equal to the value of Key property. Each Characteristic object will have ClassRef, Name and Value string properties . These correspond to SAP Document Info Record classifications associated with the SAP DMS Document Info Record represented by the Document element.
- DocumentId – this is the Document Info Record object associated with the Document element whose Id attribute has value equal to the value of Key property. This object has string properties DocumentNumber, DocumentPart, DocumentType and DocumentVersion that correspond to matching SAP ERP Document Info Record identifier.
- Materials – this is a list of Material elements associated with the Document element whose Id attribute has value equal to the value of Key property. Each Material object has MaterialId string property that contains the value of the SAP Material number associated with the Document element.
- Metadata – this is a list of MetadataGroup objects associated with the Document element whose Id attribute has value equal to the value of Key property. Each MetadataGroup object has a string property Name that corresponds to the metadata category name of the viewable metadata, and a MetadataValues property that is a list of metadata fields. Each metadata field object has string property Name that corresponds to viewable metadata key, and generic Value property that corresponds to viewable metadata value.
- PdmIds – this is a list of PdmId objects associated with the Document element whose Id attribute has value equal to the value of Key property. A PdmId object has PdmConfiguration, PdmIdentifier and PdmVersion string properties.
- Files – this is a list of File objects associated with the Document element whose Id attribute has value equal to the value of Key property. The File object has a Source string property that contains the absolute path of the file associated with this document.
- -ParentDocuments – this is a list of strings that contain Key values for the parent document elements to this document.
- The allDocumentData argument contains a mapping of documentData Key property value to the actual documentData object for all documentData objects contained in this VDI ERP Storage xml.
- In the below example, the ExtractDocumentObject callback returns a single Document Info Record data for the document for which the Object Links will be created. The returned data should be in a form of of a python dictionary, where keys ‘Id’, ‘Part’, ‘Type’ and ‘Version’ should be set to the corresponding Document Info Record key fields.
- ExtractObjectLinks callback should return data that contains information about the object link that needs to be created for the Document Info Record returned by ExtractDocumentObject callback.
- The return value of this callback should be a list of python dictionaries, where each dictionary will contain the following keys:
- ObjectDescription – this will be used to represent ObjectDescription in the created object link
- ObjectKey – depending on the object link type, this will contain information about the target of the link
- ObjectType – this determines the type of the object link to be created for this Document Info Record. For document object links, the value returned should be ‘DRAW’
The operation implementation uses the CAD CDESK services function CDESK_SRV_MODIFY_DIR to create object links. Consult the relevant CDESK services documentation for more details.
Selecting derivative types to be built
Standard VEG Build Derivatives controller can be used to build various different viewable derivatives for each of the viewables produced by conversion. This workflow should then be added as a child workflow of VDI workflow in the “Child Workflows” slot under VDI workflow instance workflow settings.
For more information on how to configure build derivatives workflow, see standard VEG documentation.
Controlling what document attributes, object links and classifications get stored in SAP ERP
The batchStoreProperties operation in Update DIR structures and properties step is used not just for creation of document object links (as described in section 6.24) but for storing document attributes, material object links and classifications as specified in the VDI Processing xml. Since this is an extension point operation, it is possible to customize it further, and implement fine-grained filtering of the data that will be sent to SAP ERP.
- FilterSourceObjectLinks callback accepts the list of source documents for which the links will be created, and has the opportunity to return only a subset of those based on some criteria. The default implementation returns the whole list that was passed in.
- FilterMaterialObjectLinks callback accepts the list materials to which links from the documents will be created, and has the opportunity to return only a subset of those based on some criteria. The default implementation returns the whole list that was passed in.
- FilterAttributes callback accepts the list of SAP Document Info Record attributes that will be associated with a document, and has the opportunity to return only a subset of those based on some criteria. The default implementation returns the whole list that was passed in.
- FilterClassifications callback accepts the list of SAP Document Info Record classifications that will be associated with a document, and has the opportunity to return only a subset of those based on some criteria. The default implementation returns the whole list that was passed in.
Testing and Correction Phase
This phase involves validation that the entire system behaves according to the implementation requirements by ensuring that the actual outputs of the conversion are identical to previously defined expected outputs.
- Identify relevant test cases based on the conclusions from the planning phase. Each test case should have defined:
- Input data
- Expected output data
- Import test data for the test cases in the source PLM/PDM
- Trigger VDI jobs to process the test data
- Verify that for each of the test cases that the output data matches the expected output data
- For tests where the output data does not match the expected output data make required corrections in implementation and re-run the tests
- Repeat step e) until for all tests the actual output data matches the expected output data