Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-3351

Taildir source data duplication

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.9.0
    • None
    • Sinks+Sources
    • None

    Description

      If the server restarts abnormally, taildir source may read data repeatedly. It's easy to replicate this phenomenon,such as using the command: reboot.
      below is my recurrence scenario:
      Agent one is deployed on server one, and it is configured taildir source, file channel, avro sink. While agent two is deployed on server two, and agent two is configured avro source, file channel, hdfs sink. This two agents are connected by avro. It means agent two receives data from agent one. Then i reboot server one, data on HDFS must be repeated after server one recovery from failure.

      Attachments

        Activity

          People

            Unassigned Unassigned
            l00454651 Confused
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: