Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
RecordBinManager.createThresholds has following code block:
if (MergeRecord.MERGE_STRATEGY_DEFRAGMENT.getValue().equals(mergeStrategy)) { fragmentCountAttribute = MergeContent.FRAGMENT_COUNT_ATTRIBUTE; if (!StringUtils.isEmpty(flowfile.getAttribute(fragmentCountAttribute))) { minRecords = Integer.parseInt(flowfile.getAttribute(fragmentCountAttribute)); } } else { fragmentCountAttribute = null; }
The code uses 'fragment.count' as the minRecords. This is wrong because 'fragment.count' represents the number of fragments, i.e. number of FlowFiles holding partial record set.
This causes a FlowFile to be sent 'failure' relationship where it should be hold in the incoming connection. For example, when a FlowFile is split into two FlowFiles, and each has 2 records in it, 'fragment.count' will be 2. In this case, MergeContent thinks the minRecords is 2, where 4 is correct. Then the first FlowFile is processed, while the 2nd one hasn't arrived, MergeContent misunderstood that the bin reached to the minimum number of records. But since there's only one FlowFile, it sends the FlowFile to 'failure'.
The issue can be reproduced by the attached template.
Attachments
Attachments
Issue Links
- links to