Description
Test steps to reproduce:
1) bin/spark-shell
val dataset = spark.createDataFrame(Seq(
(0L, 1L, 1.0),
(1L,2L,1.0),
(3L, 4L,1.0),
(4L,0L,0.1))).toDF("src", "dst", "weight")
val model = new PowerIterationClustering().
setMaxIter(10).
setInitMode("degree").
setWeightCol("weight")
val prediction = model.assignClusters(dataset).select("id", "cluster")
2) Open storage tab of the UI. we can see many RDD block cached, even after running the PIC.
Attachments
Issue Links
- links to