Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39398

when doing iteration compute in graphx, checkpoint need support storagelevel

    XMLWordPrintableJSON

Details

    • Bug
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 2.4.4
    • None
    • GraphX
    • None

    Description

      this issue related to spark-30502, in that issue ,it olny fix partial ml algorithm.

      in graphx pregel, inside iteration compute, the message checkpointer also should support setting storage level, other than the default memory only level

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-30502 PeriodicRDDCheckpointer supports storageLevel

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #36806 [Github] Pull Request #36806 (wwli05)

          Activity

            People

              Unassigned Unassigned
              wenli wangwenli
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: