Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3902

Stabilize AsyncRDDActions and expose its methods in Java API

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0
    • Java API, Spark Core
    • None

    Description

      The AsyncRDDActions methods are currently the easiest way to determine Spark jobs' ids for use in progress-monitoring code (see SPARK-2636). AsyncRDDActions is currently marked as @Experimental; for 1.2, I think that we should stabilize this API and expose it in Java, too.

      One concern is whether there's a better async API design that we should prefer over this one as our stable API; I had some ideas for a more general API in SPARK-3626 (discussed in much greater detail on GitHub: https://github.com/apache/spark/pull/2482) but decided against the more general API due to its confusing cancellation semantics. Given this, I'd be comfortable stabilizing our current API.

      Attachments

        Issue Links

          Activity

            People

              joshrosen Josh Rosen
              joshrosen Josh Rosen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: