Page tree
Skip to end of metadata
Go to start of metadata

Filtering on the comparison

Sometimes not all data existing in the systems should be compared. The reason might be that the comparison of some data is meaningless or that you would like to restrict the data to different organizational units.

An example for the first case would be if data has to be released before it is send to the target systems (e.g. service orders have to be released in a central customer care system before they are replicated to a maintenance systems). In this case an comparison of unreleased service orders would be useless as they exist by definition only in the source system. As this behaviour is build into the process it would not make sense to allow end users to change this filter but it should be build into your data model. To allow this type of filtering, model specific filters can be maintained which are valid for all comparisons using this model. 

An example of the second case could be required if multiple organizational units with different responsibles work in the same system. In this case it might be sensible to define comparisons for each responsible where the employee sees only the data he or she is responsible for. The actual data model for each responsible is identical but he should see only a subset of the data. To do this comparison specific filters can be defined as well.

Sometimes you would like to filter on values for multiple fields in the same comparison. In this case it is important that you provide a well-defined relation ship between the different filter values. An example could be that you would like to compare sales orders for certain sales organisations and distribution channel. If you would provide only the information you are interested in data for sales organization 1, sales organization 2, distribution channel A and distribution channel B the data selection would consider all possible combinations which are the value pairs (sales organization 1, distribution channel A), (sales organization 1, distribution channel B), (sales organization 2, distribution channel A), (sales organization 2, distribution channel B).

This might be okay for most cases but if you are only interested in the combinations (sales organization 1, distribution channel A), (sales organization 2, distribution channel B) you need to provide the relationship, which can be done via filter sets.

Types of Filters

 

You can use various types of filters in cross-data comparison to restrict the data being compared for the sources. The following filter types are available in Cross Database Comparison.

Filter TypeData ModelComparisonComparison Run
Object filters with fixed values (type 1)You can define a fixed filter value at source level. You can define them for both sources or for one only). The filter value is used during the generation to create a hard-coded where clause.Not visible on the comparison levelComparison Run fetches data from generated extractor with its hard-coded where-clause.
Comparison filters with variable values (type 2)Variable filter flag is selected in the data model, either at mapping level (relevant for both sources) or at source level (relevant for one source only).On the comparison level the actual filter values can be maintained. These are used for all Comparison Runs of this Comparison Instance.Comparison Run hands over the filters to the extractor, which handles the filter values dynamically.
Object filters with relative dates (type 3, variant of type 1)If a filter for a Data model contains the syntax for relative dates (starts with ‘$’), it will not be generated as hard-coded filter in the extractor.Not visible on the comparison level. Valid for all Comparison Instances of this Comparison Object.Comparison Run translates the relative date syntax into an absolute timestamp and hands over the filters to the extractor, which handles the filter values dynamically.
Comparison filters with relative dates (type 4, variant of type 2)

Variable Filter flag is selected in comparison object, either at mapping level (relevant for both sources) or at source level (relevant for one source only).

On the comparison level, relative date syntax can be maintained (starts with ‘$’). These are used for all Comparison Runs of this Comparison Instance.Comparison Run translates the relative date syntax into an absolute timestamp and hands over the filters to the extractor, which handles the filter values dynamically.

Model specific filters are fixed filter values at source level. You can define them for both sources or for one only). The filter value is used during the generation to create a hard-coded where clause. Also if a filter for a data model contains the syntax for relative dates (starts with ‘$’), it will not be generated as hard-coded filter in the extractor.

Below is an example of fixed filters.

Below is an example of using relative dates syntax for one source system. See Using relative date syntax for more information.


The Variable Filter flag is selected in comparison object, either at mapping level (relevant for both sources) or at source level (relevant for one source only). In the Comparison, the actual filter values can be maintained. These are used for all Comparison Runs of this Comparisons. Comparison Run hands over the filters to the extractor, which handles the filter values dynamically. In the  Comparison, relative date syntax can be maintained (starts with ‘$’). These are used for all Comparison Runs of this Comparison. Comparison Run translates the relative date syntax into an absolute timestamp and hands over the filters to the extractor, which handles the filter values dynamically. See Using relative date syntax for more information.

On object level, set flag and date type for “Variable Filter” on mapping details popup

On selecting the filter and clicking OK, In the comparison-specific filters group, a new option appears with the heading filters from mapping.


Using relative date syntax

 

There are many use cases, where the comparison shall be time-dependent. For example, you only want to compare documents created yesterday, or during the last month. Entering static date filter values does not make sense here, because you would have to adapt the comparison filters before each run. To achieve a relative date filtering, which means calculating relative dates instead of absolute dates, CDC provides a special syntax on how define fix points and offsets for relative dates.

Syntax for Relative Dates

Instead of a fixed (absolute) date, you can enter a special keyword for the start date (prefixed by a $ character) and optionally an additional offset as difference in days.

Syntax = <StartDate>[<Difference>]

For <StartDate> the following keywords are available:

Keyword for <StartDate>

Description

Example using 2013/04/25 as Reference Date

$TODAY

Current date (today)

2013/04/25

$FDOCW

First day of current week

2013/04/22

$LDOCW

Last day of current week

2013/04/28

$FDOCM

First day of current month

2013/04/01

$LDOCM

Last day of current month

2013/04/30

$FDOCQFirst day of current quarter2013/04/01
$LDOCQLast day of current quarter2013/06/30

$FDOCY

First day of current year

2013/01/01

$LDOCY

Last day of current year

2013/12/31

$FDOPW

First day of previous week

2013/04/15

$LDOPW

Last day of previous week

2013/04/21

$FDOPQFirst day of previous quarter2013/01/01
$LDOPQLast  day of previous quarter2013/03/31

$FDOPM

First day of previous month

2013/03/01

$LDOPM

Last day of previous month

2013/03/31

$FDOPY

First day of previous year

2012/01/01

$LDOPY

Last day of previous year

2012/12/31

$FDONW

First day of next week

2013/04/29

$LDONW

Last day of next week

2013/05/05

$FDONM

First day of next month

2013/05/01

$LDONM

Last day of next month

2013/05/31

$FDONQFirst day of next quarter2013/07/01
$LDONQLast day of next quarter2013/09/30

$FDONY

First day of next year

2014/01/01

$LDONY

Last day of next year

2014/12/31

$CWEEKCurrent calendar week (offset in weeks)201317
$FIPECFiscal Period (current)

004.2013

$FIPEPFiscal Period (previous)003.2013

These additional keywords are meant for full timestamps only:

Keyword for <StartDate>

Description

Example for 2013/04/25 at 08:21:47 with offset -2

$TIMES

Time stamp now (offset in seconds)

2013/04/25 08:21:45

$TIMEM

Time stamp now (offset in minutes)

2013/04/25 08:19:47

$TIMEH

Time stamp now (offset in hours)

2013/04/25 06:21:47

$TIMED

Time stamp now (offset in days)

2013/04/23 08:21:47

$HOURA

absolute hour
(+=today / -=yesterday)

2013/04/24 02:00:00

$HOURR

current full hour (offset in hours)

2013/04/25 06:00:00

$DELTA

Delta mode (timestamp of last run)

n/a

The optional <Difference> is entered as positive or negative offset (usually in days, unless otherwise noted)

  • using '+' increments days (= move start date into the future)
  • using '-' decrements days (= move start date into the past)

Examples:

  • $TODAY-2 = day before yesterday
  • $TODAY+2 = day after tomorrow

The relative date selection can be combined in ranges (using the LOW and HIGH fields of the selection criteria), e.g. to define "between" intervals.

Example: You want to filter the documents which have a creation date in last month. The selection criteria would be <FieldName> = $FDOPM to $LDOPM.

  • No labels