How is the QueryProvider data packaged or why is DTP package size ignored?
Query data cannot be sorted and thus packaged as easily as most regular DTP data. Therefore, the package size set in the DTP will always be ignored. Semantic Grouping is to be used.
This has changed a bit over time and in older Support packages a different (but formally wrong behaviour can be observed, check with the 2 notes linked below to see).
The packaging on QueryProviders has never worked. The background is that a QueryProvider offers no "cursor" concept. The QPROV data must all be read in one package and then the OLAP processing takes place and then the data is ready for retrieval. This retrieval from OLAP to the DTP is not done in packages as the data is already completely in the OLAP engine. This has never changed. Only on the level of the DTP packaging is possible, see Semantic Grouping.
As of note 1913903, in the DTP extraction tab you can see a status message on the possibly used package level. The pack level describes the type of data packaging that is in general possible for the query features used in the Bex query:
- Pack level 1: package size is the upper limit. Empty packages and packages smaller than this size are also possible.
- Pack level 3: the package size is used as a guideline for internal packaging. However, packages may also be smaller or larger than this size.
- Pack level 5: packaging is not possible. The data is sent in one single package.
This pack level displayed in the DTP is only a recommendation and will be checked during DTP/QPROV runtime again. For a QueryProviders only 3 and 5 are allowed.
Since the introduction of semantic grouping (partitioning) for QPROV with SAP note: 2165017, this pack level is the basis to decide if semantic grouping is possible at all during DTP/QPROV runtime. The partitioning on QPROVs does only work with semantic groups. The group sizes will approximately be the size given in the DTP setting but this can also vary. Please see Semantic Grouping for further details(in particular check the examples)
For this DTP the package size is set to 1000 records. The packlevel determined is 1: package size is the upper limit.
However, running this DTP, the data is extracted in one big package of 237.882 records.
For retrieving the data in packages semantic partitioning (semantic grouping) has to be defined.