Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9277

Hybrid Hybrid Grace Hash Join

    XMLWordPrintableJSON

Details

    Description

      We are proposing an enhanced hash join algorithm called “hybrid hybrid grace hash join”.

      We can benefit from this feature as illustrated below:

      • The query will not fail even if the estimated memory requirement is slightly wrong
      • Expensive garbage collection overhead can be avoided when hash table grows
      • Join execution using a Map join operator even though the small table doesn't fit in memory as spilling some data from the build and probe sides will still be cheaper than having to shuffle the large fact table

      The design was based on Hadoop’s parallel processing capability and significant amount of memory available.

      Attachments

        1. HIVE-9277.15.patch
          138 kB
          Wei Zheng
        2. HIVE-9277.14.patch
          138 kB
          Wei Zheng
        3. HIVE-9277.13.patch
          138 kB
          Wei Zheng
        4. HIVE-9277.08.patch
          114 kB
          Wei Zheng
        5. HIVE-9277.07.patch
          88 kB
          Wei Zheng
        6. HIVE-9277.06.patch
          86 kB
          Wei Zheng
        7. HIVE-9277.05.patch
          87 kB
          Wei Zheng
        8. HIVE-9277.04.patch
          95 kB
          Wei Zheng
        9. HIVE-9277.03.patch
          84 kB
          Wei Zheng
        10. HIVE-9277.02.patch
          86 kB
          Wei Zheng
        11. HIVE-9277.01.patch
          89 kB
          Wei Zheng
        12. High-leveldesignforHybridHybridGraceHashJoinv1.0.pdf
          453 kB
          Wei Zheng

        Issue Links

          Activity

            People

              wzheng Wei Zheng
              wzheng Wei Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: