Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-545

Make in-memory key-value store skip serde

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.9.0
    • None
    • kv
    • None

    Description

      SAMZA-256 added an in-memory implementation of the samza-kv store. Due to the layering in Samza's KV-store APIs, the in-memory store still holds raw bytes, and the Serde is used to transform objects back into POJOs. On the read-side, it is unnecessary to deserialize the bytes back into an object. The in-memory KV store should just hold the raw object.

      Semantically, this does change the behavior of the KV-store a bit, when using in-memory stores. If an object is mutated after it's been written to an in-memory store, and then store.get is called, the mutated object will be returned. This is not the case with regular (LevelDB/RocksDB) KV-stores.

      Writes will still require serializing the object if a changelog is attached. If a changelog is not attached, then I'd argue that the in-memory KV store should not be used at all, and a simple HashMap should be used instead.

      When updating the code, we should be mindful to keep the API as clean as possible, while shifting the layers around.

      Attachments

        Activity

          People

            Unassigned Unassigned
            criccomini Chris Riccomini
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: