Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17188

ObjectStore runs out of memory for large batches of addPartitions().

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0, 2.4.0, 3.0.0
    • Metastore
    • None

    Description

      For large batches (e.g. hundreds) of addPartitions(), the ObjectStore runs out of memory. Flushing the PersistenceManager alleviates the problem.

      Note: The problem being addressed here isn't so much with the size of the hundreds of Partition objects, but the cruft that builds with the PersistenceManager, in the JDO layer, as confirmed through memory-profiling.

      (Raising this on behalf of cdrome and thiruvel.)

      Attachments

        1. HIVE-17188.1.patch
          0.8 kB
          Mithun Radhakrishnan

        Activity

          People

            cdrome Chris Drome
            mithun Mithun Radhakrishnan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: