Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22414

Make LLAP CacheTags more memory efficient

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • llap
    • None

    Description

      MultiPartitionCacheTag relies on LinkedLists.

      A LinkedList object that holds e.g. 2 nodes consumes 112 bytes roughly in this composition:

      • 16 bytes for LinkedList object header
      • 8 bytes for referring head
      • 8 bytes for referring tail
      • 80 = 2 x (16 bytes for LinkedList$Node header, 3 x 8 bytes for referring prev, next, item)

      This is a lot, so I propose to replace LinkedList in MultiPartitionCacheTag with a simple String array. (For a similar scenario an array would take 16 + 8 + 2 x 8 = 40 bytes, as per header, count, and 2 references for our actual objects).

      Attachments

        1. HIVE-22414.0.patch
          4 kB
          Ádám Szita
        2. HIVE-22414.1.patch
          4 kB
          Ádám Szita
        3. HIVE-22414.2.patch
          5 kB
          Ádám Szita
        4. HIVE-22414.3.patch
          5 kB
          Ádám Szita
        5. HIVE-22414.4.patch
          5 kB
          Ádám Szita
        6. HIVE-22414.5.patch
          5 kB
          Ádám Szita
        7. HIVE-22414.6.patch
          12 kB
          Ádám Szita
        8. HIVE-22414.7.patch
          12 kB
          Ádám Szita
        9. HIVE-22414.8.patch
          12 kB
          Ádám Szita

        Activity

          People

            szita Ádám Szita
            szita Ádám Szita
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: