Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29236

Access 'executorDataMap' out of 'DriverEndpoint' should be protected by lock

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.4
    • 3.0.0
    • Spark Core
    • None

    Description

      Just as the comments:

      // Accessing `executorDataMap` in `DriverEndpoint.receive/receiveAndReply` doesn't need any
      // protection. But accessing `executorDataMap` out of `DriverEndpoint.receive/receiveAndReply`
      // must be protected by `CoarseGrainedSchedulerBackend.this`. Besides, `executorDataMap` should
      // only be modified in `DriverEndpoint.receive/receiveAndReply` with protection by
      // `CoarseGrainedSchedulerBackend.this`.

       

      `executorDataMap` is not threadsafe, it should be protected by lock when accessing it out of `DriverEndpoint`

      Attachments

        Issue Links

          Activity

            People

              coneyliu Xianyang Liu
              coneyliu Xianyang Liu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: