WAVE: extract/sfdcDigest is not producing the correct number of input-rows-processed for extracts that require multiple batches
Last updated 2021-05-03 ·Reference W-2834237 ·Reported By 2 users
If a salesforce extract is extracting a number of rows larger than a chunk size limit, then the extract will be broken into multiple batches. This means that the number of input-rows-processed will be the sum of the rows processed in each batch. However, because the heartbeat is written such that we look at the number of rows processed in the last batch rather than the sum, we get a number of "input-rows-processed" that is less than the actual number of rows processed.
1 - Within Wave Analytics, run a dataflow.
2 - When dataflow is complete, notice there is a discrepancy in the results between the “Input Rows Processed” and the “Output Rows Processed”.
Is it Fixed?
Any unreleased services, features, statuses, or dates referenced in this or other public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make their purchase decisions based upon features that are currently available.