Filtering on the comparison
Sometimes not all data existing in the systems should be compared. The reason might be that the comparison of some data is meaningless or that you would like to restrict the data to different organizational units.
An example for the first case would be if data has to be released before it is send to the target systems (e.g. service orders have to be released in a central customer care system before they are replicated to a maintenance systems). In this case an comparison of unreleased service orders would be useless as they exist by definition only in the source system. As this behaviour is build into the process it would not make sense to allow end users to change this filter but it should be build into your data model. To allow this type of filtering, model specific filters can be maintained which are valid for all comparisons using this model.
An example of the second case could be required if multiple organizational units with different responsibles work in the same system. In this case it might be sensible to define comparisons for each responsible where the employee sees only the data he or she is responsible for. The actual data model for each responsible is identical but he should see only a subset of the data. To do this comparison specific filters can be defined as well.
Sometimes you would like to filter on values for multiple fields in the same comparison. In this case it is important that you provide a well-defined relation ship between the different filter values. An example could be that you would like to compare sales orders for certain sales organisations and distribution channel. If you would provide only the information you are interested in data for sales organization 1, sales organization 2, distribution channel A and distribution channel B the data selection would consider all possible combinations which are the value pairs (sales organization 1, distribution channel A), (sales organization 1, distribution channel B), (sales organization 2, distribution channel A), (sales organization 2, distribution channel B).
This might be okay for most cases but if you are only interested in the combinations (sales organization 1, distribution channel A), (sales organization 2, distribution channel B) you need to provide the relationship, which can be done via filter sets.
Types of Filters
You can use various types of filters in cross-data comparison to restrict the data being compared for the sources. The following filter types are available in Cross Database Comparison.
Filter Type | Data Model | Comparison | Comparison Run |
---|---|---|---|
Object filters with fixed values (type 1) | You can define a fixed filter value at source level. You can define them for both sources or for one only). The filter value is used during the generation to create a hard-coded where clause. | Not visible on the comparison level | Comparison Run fetches data from generated extractor with its hard-coded where-clause. |
Comparison filters with variable values (type 2) | Variable filter flag is selected in the data model, either at mapping level (relevant for both sources) or at source level (relevant for one source only). | On the comparison level the actual filter values can be maintained. These are used for all Comparison Runs of this Comparison Instance. | Comparison Run hands over the filters to the extractor, which handles the filter values dynamically. |
Object filters with relative dates (type 3, variant of type 1) | If a filter for a Data model contains the syntax for relative dates (starts with ‘$’), it will not be generated as hard-coded filter in the extractor. | Not visible on the comparison level. Valid for all Comparison Instances of this Comparison Object. | Comparison Run translates the relative date syntax into an absolute timestamp and hands over the filters to the extractor, which handles the filter values dynamically. |
Comparison filters with relative dates (type 4, variant of type 2) | Variable Filter flag is selected in comparison object, either at mapping level (relevant for both sources) or at source level (relevant for one source only). | On the comparison level, relative date syntax can be maintained (starts with ‘$’). These are used for all Comparison Runs of this Comparison Instance. | Comparison Run translates the relative date syntax into an absolute timestamp and hands over the filters to the extractor, which handles the filter values dynamically. |
Model specific filters are fixed filter values at source level. You can define them for both sources or for one only). The filter value is used during the generation to create a hard-coded where clause. Also if a filter for a data model contains the syntax for relative dates (starts with ‘$’), it will not be generated as hard-coded filter in the extractor.
Below is an example of fixed filters.
Below is an example of using relative dates syntax for one source system. See Using relative date syntax for more information.
The Variable Filter flag is selected in comparison object, either at mapping level (relevant for both sources) or at source level (relevant for one source only). In the Comparison, the actual filter values can be maintained. These are used for all Comparison Runs of this Comparisons. Comparison Run hands over the filters to the extractor, which handles the filter values dynamically. In the Comparison, relative date syntax can be maintained (starts with ‘$’). These are used for all Comparison Runs of this Comparison. Comparison Run translates the relative date syntax into an absolute timestamp and hands over the filters to the extractor, which handles the filter values dynamically. See Using relative date syntax for more information.
On object level, set flag and date type for “Variable Filter” on mapping details popup
On selecting the filter and clicking OK, In the comparison-specific filters group, a new option appears with the heading filters from mapping.
Using relative date syntax
There are many use cases, where the comparison shall be time-dependent. For example, you only want to compare documents created yesterday, or during the last month. Entering static date filter values does not make sense here, because you would have to adapt the comparison filters before each run. To achieve a relative date filtering, which means calculating relative dates instead of absolute dates, CDC provides a special syntax on how define fix points and offsets for relative dates.
Syntax for Relative Dates
Instead of a fixed (absolute) date, you can enter a special keyword for the start date (prefixed by a $ character) and optionally an additional offset as difference in days.
Syntax = <StartDate>[<Difference>]
For <StartDate> the following keywords are available:
Keyword for <StartDate> | Description | Example using 2013/04/25 as Reference Date |
---|---|---|
$TODAY | Current date (today) | 2013/04/25 |
$FDOCW | First day of current week | 2013/04/22 |
$LDOCW | Last day of current week | 2013/04/28 |
$FDOCM | First day of current month | 2013/04/01 |
$LDOCM | Last day of current month | 2013/04/30 |
$FDOCQ | First day of current quarter | 2013/04/01 |
$LDOCQ | Last day of current quarter | 2013/06/30 |
$FDOCY | First day of current year | 2013/01/01 |
$LDOCY | Last day of current year | 2013/12/31 |
$FDOPW | First day of previous week | 2013/04/15 |
$LDOPW | Last day of previous week | 2013/04/21 |
$FDOPQ | First day of previous quarter | 2013/01/01 |
$LDOPQ | Last day of previous quarter | 2013/03/31 |
$FDOPM | First day of previous month | 2013/03/01 |
$LDOPM | Last day of previous month | 2013/03/31 |
$FDOPY | First day of previous year | 2012/01/01 |
$LDOPY | Last day of previous year | 2012/12/31 |
$FDONW | First day of next week | 2013/04/29 |
$LDONW | Last day of next week | 2013/05/05 |
$FDONM | First day of next month | 2013/05/01 |
$LDONM | Last day of next month | 2013/05/31 |
$FDONQ | First day of next quarter | 2013/07/01 |
$LDONQ | Last day of next quarter | 2013/09/30 |
$FDONY | First day of next year | 2014/01/01 |
$LDONY | Last day of next year | 2014/12/31 |
$CWEEK | Current calendar week (offset in weeks) | 201317 |
$FIPEC | Fiscal Period (current) | 004.2013 |
$FIPEP | Fiscal Period (previous) | 003.2013 |
These additional keywords are meant for full timestamps only:
Keyword for <StartDate> | Description | Example for 2013/04/25 at 08:21:47 with offset -2 |
---|---|---|
$TIMES | Time stamp now (offset in seconds) | 2013/04/25 08:21:45 |
$TIMEM | Time stamp now (offset in minutes) | 2013/04/25 08:19:47 |
$TIMEH | Time stamp now (offset in hours) | 2013/04/25 06:21:47 |
$TIMED | Time stamp now (offset in days) | 2013/04/23 08:21:47 |
$HOURA | absolute hour | 2013/04/24 02:00:00 |
$HOURR | current full hour (offset in hours) | 2013/04/25 06:00:00 |
$DELTA | Delta mode (timestamp of last run) | n/a |
The optional <Difference> is entered as positive or negative offset (usually in days, unless otherwise noted)
- using '+' increments days (= move start date into the future)
- using '-' decrements days (= move start date into the past)
Examples:
- $TODAY-2 = day before yesterday
- $TODAY+2 = day after tomorrow
The relative date selection can be combined in ranges (using the LOW and HIGH fields of the selection criteria), e.g. to define "between" intervals.
Example: You want to filter the documents which have a creation date in last month. The selection criteria would be <FieldName> = $FDOPM to $LDOPM.