Uploaded image for project: 'S2Graph'
  1. S2Graph
  2. S2GRAPH-26

Apply squash optimization on mutateElements and make it default behavior on Storage.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Currently AsynchbaseStorage that implement Storage trait using HBase use squash optimization on snapshot edges at mutateEdges.

      for example, if there are requests on same snapshotEdge that consists of (insert, delete, insert, delete, insert) in their timestamp order, then we only need to apply last insert.

      ex) lets assume that there are 5 requests on from “shon” to “dun” with label “friend”, insert(t0), delete(t1), insert(t2), delete(t3), insert(t4).

      without squashing same snapshot edges in memory, then 5 of following actions need to be done.

      1. fetch snapshot edge
      2. lock this snapshot edge.
      3. build new update and delete, insert, degree on IndexEdges.
      4. fire above update/delete/insert/degree mutations into HBase.
      5. release lock on this snapshotEdge.

      we can do 1, 2, 5(fetch, lock, release lock) one time and squash mutations that built from multiple requests and squash them.

      above logic needs to keep consistency between multiple indexEdges and purpose above logic is make only one thread on same snapshot edge be able to mutate(note the lock). since we will acquire lock on this edge, it is much efficient squash multiple requests on same snapshotEdge to avoid heavy operation, fetch, lock, release lock.

      Attachments

        Issue Links

          Activity

            People

              daewon Daewon Jeong
              steamshon Do Yung Yoon
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - 72h
                  72h
                  Remaining:
                  Remaining Estimate - 72h
                  72h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified