Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.1.0
-
None
-
None
Description
The private variable blockInfo in mllib.linalg.distribtued.BlockMatrix is never unpersisted since a BlockMatrix instance is created.
private lazy val blockInfo = blocks.mapValues(block => (block.numRows, block.numCols)).cache()
I think we should add an API to unpersist this variable.