Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17204 support un-bucketed tables in acid
  3. HIVE-17215

Streaming Ingest API writing unbucketed tables

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Transactions
    • None

    Description

      Currently the API expects the target table to be bucketed.
      It creates 1 writer per bucket per connection/partition.
      The simplest is to allow the API to create a single writer for unbucketed tables.
      If this doesn't provide enough write throughput, the client can create another connection.

      Could add a parameter to the API to specify writer parallelism for unbucketed tables. If it's set to 2 for example, the writer will write delta_x_y_0000 and delta_x_y_00001 using statementId. Maybe as a followup.

      Attachments

        Activity

          People

            ekoifman Eugene Koifman
            ekoifman Eugene Koifman
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: