Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6395

CountText processor is not thread safe - concurrency error

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      The processor counters fail to execute multiple threads. The programming is not safe since they are not atomic operations. They are using a volatile instance variable accessed by multiple threads when onTrigger is called. The solution is to declare those local variables to onTrigger. Perform several tests with millions of records and the counter does not work correctly when it is executed with more than one task.

      The problem is in the declaration of these instance variables:

      private volatile int lineCount;
      private volatile int lineNonEmptyCount;
      private volatile int wordCount;
      private volatile int characterCount;

      This is not safe to perform atomic operations on these variables. As a result the counters register less amount of lines when executed with multiple threads.

      I propose the following solution:

      Fix bug pull request

      problem graph:

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ivan1221 Iván Ezequiel Rodriguez
            ivan1221 Iván Ezequiel Rodriguez
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 20m
                4h 20m

                Slack

                  Issue deployment