Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-32578

PageRank not sending the correct values in Pergel sendMessage

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 2.3.0, 2.4.0, 3.0.0
    • None
    • GraphX
    • None

    Description

      The core sendMessage method is incorrect:

      def sendMessage(edge: EdgeTriplet[(Double, Double), Double]) = {
       if (edge.srcAttr._2 > tol) {
         Iterator((edge.dstId, edge.srcAttr._2 * edge.attr))
                                // *** THIS ^ ***
       } else {
         Iterator.empty
       }
      }

       

      Instead of using the source PR value, it's using the PR delta (2nd tuple arg). This is not the documented behavior, nor a valid PR algorithm AFAIK.

      This is a 7 years old code, all versions affected.

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            shay_elbaz Shay Elbaz
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: