Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12631

LLAP IO: support ORC ACID tables

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: llap, Transactions
    • Labels:
      None

      Description

      LLAP uses a completely separate read path in ORC to allow for caching and parallelization of reads and processing. This path does not support ACID. As far as I remember ACID logic is embedded inside ORC format; we need to refactor it to be on top of some interface, if practical; or just port it to LLAP read path.
      Another consideration is how the logic will work with cache. The cache is currently low-level (CB-level in ORC), so we could just use it to read bases and deltas (deltas should be cached with higher priority) and merge as usual. We could also cache merged representation in future.

        Attachments

        1. HIVE-12631.1.patch
          95 kB
          Teddy Choi
        2. HIVE-12631.2.patch
          102 kB
          Teddy Choi
        3. HIVE-12631.3.patch
          44 kB
          Teddy Choi
        4. HIVE-12631.4.patch
          44 kB
          Teddy Choi
        5. HIVE-12631.5.patch
          44 kB
          Teddy Choi
        6. HIVE-12631.6.patch
          47 kB
          Teddy Choi
        7. HIVE-12631.7.patch
          46 kB
          Teddy Choi
        8. HIVE-12631.8.patch
          46 kB
          Teddy Choi
        9. HIVE-12631.8.patch
          46 kB
          Teddy Choi
        10. HIVE-12631.9.patch
          71 kB
          Teddy Choi
        11. HIVE-12631.10.patch
          71 kB
          Teddy Choi
        12. HIVE-12631.10.patch
          73 kB
          Teddy Choi
        13. HIVE-12631.11.patch
          1.26 MB
          Teddy Choi
        14. HIVE-12631.11.patch
          112 kB
          Teddy Choi
        15. HIVE-12631.12.patch
          112 kB
          Teddy Choi
        16. HIVE-12631.13.patch
          166 kB
          Teddy Choi
        17. HIVE-12631.15.patch
          56 kB
          Teddy Choi
        18. HIVE-12631.16.patch
          63 kB
          Teddy Choi
        19. HIVE-12631.17.patch
          73 kB
          Teddy Choi
        20. HIVE-12631.18.patch
          161 kB
          Teddy Choi
        21. HIVE-12631.19.patch
          131 kB
          Teddy Choi
        22. HIVE-12631.20.patch
          73 kB
          Teddy Choi
        23. HIVE-12631.21.patch
          74 kB
          Teddy Choi
        24. HIVE-12631.22.patch
          76 kB
          Teddy Choi
        25. HIVE-12631.23.patch
          76 kB
          Teddy Choi
        26. HIVE-12631.24.patch
          97 kB
          Teddy Choi
        27. HIVE-12631.25.patch
          96 kB
          Teddy Choi
        28. HIVE-12631.26.patch
          97 kB
          Teddy Choi
        29. HIVE-12631.27.patch
          97 kB
          Sergey Shelukhin
        30. HIVE-12631.28.patch
          97 kB
          Sergey Shelukhin
        31. HIVE-12631.29.patch
          113 kB
          Teddy Choi
        32. HIVE-12631.30.patch
          114 kB
          Teddy Choi
        33. HIVE-12631.31.patch
          106 kB
          Teddy Choi

          Issue Links

            Activity

              People

              • Assignee:
                teddy.choi Teddy Choi
                Reporter:
                sershe Sergey Shelukhin
              • Votes:
                3 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: