Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-325

Add (de)serialization for non-string key types in data metric collection

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None

    Description

      I figured out that our data metric collection mechanism assumes that the key type of the data is String always.

      PartitionSizeEntry of our ControlMessage protobuf has string as the type of key, and DynOptDataOutputCollector just conducts String.valueOf(key) for the keys. (These strings are not deserialized after.)

      Because of this, if the type of the key of the data to analyze is not }}{{String, the skewness will not be detected properly.

      We need to add some (de)serialization step for the key instead of this assumption.

      Attachments

        Activity

          People

            Unassigned Unassigned
            sanha Sanha Lee
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: