Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12509

AggregateFn: only accumulator types with default coders are supported.

Details

    • New Feature
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • dsl-sql-zetasql

    Description

      AggregateFn infers the coder for the accumulator from its type [1]. This only works for types with default coders registered [2].

      To enable the use custom coders, AggregateFn will need a method to set the accumulator coder. However, the Coder class itself is a part of the Beam core SDK, a dependency we were trying to avoid in org.apache.beam.sdk.extensions.sql.udf.

      [1] https://github.com/apache/beam/blob/9418994e11dbede889b508c91261ec2092f2d8a7/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/LazyAggregateCombineFn.java#L88

      [2] https://github.com/apache/beam/blob/9418994e11dbede889b508c91261ec2092f2d8a7/sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CoderRegistry.java#L106-L143

      Attachments

        Activity

          People

            Unassigned Unassigned
            ibzib Kyle Weaver
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: