Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7689

Temporary directory for WriteOperation may not be unique in FileBaseSink

Details

    • Bug
    • Status: Resolved
    • P0
    • Resolution: Fixed
    • 2.5.0, 2.6.0, 2.7.0, 2.8.0, 2.9.0, 2.10.0, 2.11.0, 2.12.0, 2.13.0
    • 2.14.0
    • io-java-files
    • None

    Description

      Temporary directory for WriteOperation in FileBasedSink is generated from a second-granularity timestamp (yyyy-MM-dd_HH-mm-ss) and unique increasing index. Such granularity is not enough to make temporary directories unique between different jobs. When two jobs share the same temporary directory, output file may not be produced in one job since the required temporary file can be deleted from another job.

      Attachments

        Activity

          People

            heejong Heejong Lee
            heejong Heejong Lee
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h 40m
                3h 40m