Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21164

ACID: explore how we can avoid a move step during inserts/compaction

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.1.1
    • 4.0.0-alpha-1
    • Transactions
    • None

    Description

      Currently, we write compacted data to a temporary location and then move the files to a final location, which is an expensive operation on some cloud file systems. Since HIVE-20823 is already in, it can control the visibility of compacted data for the readers. Therefore, we can perhaps avoid writing data to a temporary location and directly write compacted data to the intended final path.

      Attachments

        1. HIVE-21164.1.patch
          56 kB
          Vaibhav Gumashta
        2. HIVE-21164.10.patch
          97 kB
          Vaibhav Gumashta
        3. HIVE-21164.11.patch
          96 kB
          László Bodor
        4. HIVE-21164.11.patch
          96 kB
          László Bodor
        5. HIVE-21164.12.patch
          120 kB
          Marta Kuczora
        6. HIVE-21164.13.patch
          159 kB
          Marta Kuczora
        7. HIVE-21164.14.patch
          160 kB
          Marta Kuczora
        8. HIVE-21164.14.patch
          160 kB
          Marta Kuczora
        9. HIVE-21164.15.patch
          159 kB
          Marta Kuczora
        10. HIVE-21164.16.patch
          161 kB
          Marta Kuczora
        11. HIVE-21164.17.patch
          237 kB
          Marta Kuczora
        12. HIVE-21164.18.patch
          238 kB
          Marta Kuczora
        13. HIVE-21164.19.patch
          238 kB
          Marta Kuczora
        14. HIVE-21164.2.patch
          59 kB
          Vaibhav Gumashta
        15. HIVE-21164.20.patch
          227 kB
          Marta Kuczora
        16. HIVE-21164.21.patch
          227 kB
          Marta Kuczora
        17. HIVE-21164.22.patch
          227 kB
          Marta Kuczora
        18. HIVE-21164.3.patch
          61 kB
          Vaibhav Gumashta
        19. HIVE-21164.4.patch
          67 kB
          Vaibhav Gumashta
        20. HIVE-21164.5.patch
          83 kB
          Vaibhav Gumashta
        21. HIVE-21164.6.patch
          73 kB
          Vaibhav Gumashta
        22. HIVE-21164.7.patch
          73 kB
          Vaibhav Gumashta
        23. HIVE-21164.8.patch
          74 kB
          Vaibhav Gumashta
        24. HIVE-21164.9.patch
          93 kB
          Vaibhav Gumashta

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-22350 ACID: disable query re-execution when doing transactional inserts

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. HIVE-23442 ACID major compaction doesn't read base directory correctly if it was written by insert overwrite

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-23410 ACID: Improve the delete and update operations to avoid the move step

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-23114 Insert overwrite with dynamic partitioning is not working correctly with direct insert

          • Major - Major loss of function.
          • Closed
          requires

          New Feature - A new feature of the product, which has yet to be developed. HIVE-20699 Query based compactor for full CRUD Acid tables

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Review Board

          Activity

            People

              kuczoram Marta Kuczora
              vgumashta Vaibhav Gumashta
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: