Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-17330

[C#] Extend ArrowBuffer.BitmapBuilder to improve performance of array concatenation

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Reopened
    • Minor
    • Resolution: Unresolved
    • 8.0.0
    • None
    • C#

    Description

      Extend ArrowBuffer.BitmapBuilder with Append method overloaded with ReadOnlySpan<byte> parameter.

      This allows to add validity bits to the builder more efficiently (especially for cases when initial validity bits are added to newly created empty builder). More over it makes BitmapBuilder API more consistent (for example ArrowBuffer.Builder<T> does have such method).

      Currently adding new bits to existing bitmap is implemented in ArrayDataConcatenator, Code adds bit by bit in a cycle converting each to a boolean value:

      for (int i = 0; i < length; i++)

      { builder.Append(span.IsEmpty || BitUtility.GetBit(span, i)); }

      Initial problem was described in this email: https://lists.apache.org/thread/kls6tjq2hclsvd16tw901ooo5soojrmb

      PR: https://github.com/apache/arrow/pull/13810

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              asmirnov82 Alexey Smirnov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h