Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34270

Combine StateStoreMetrics should not override StateStoreCustomMetric

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.4, 2.4.7, 3.0.1, 3.1.2, 3.2.0
    • 2.4.8, 3.0.2, 3.1.1
    • Structured Streaming
    • None

    Description

      For stateful join in structured streaming, we need to combine StateStoreMetrics from both left and right side. Currently we simply take arbitrary one from custom metrics with same name from left and right. By doing this we miss half of metric number.

      Attachments

        Activity

          People

            viirya L. C. Hsieh
            viirya L. C. Hsieh
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: