Details
-
Improvement
-
Status: In Progress
-
P3
-
Resolution: Unresolved
-
None
-
None
Description
Introduce the concept of Options in Beam Schema’s to add extra context to fields and schemas. In contrast to the current Beam metadata that is present in a FieldType, options would be added to fields, logical types and schemas. The schema convertors (ex. Avro, Proto, …) can add options/annotations/decorators that were in the original schema to the Beam schema with these options. These options, that add contextual metadata, can be used in the pipeline for specific transformations or augment the end schema in the target output.
Attachments
Issue Links
- mentioned in
-
Page Loading...
1.
|
BIP-1: Typed options for Row Schema and Fields |
|
Triage Needed | Alex Van Boxel |
|
|||||||
2.
|
BIP-1: Convert protobuf options to Schema options |
|
Triage Needed | Alex Van Boxel |
|
|||||||
3.
|
BIP-1: Convert avro metadata to Schema options |
|
Resolved | Alex Van Boxel |
|
|||||||
4.
|
BIP-1: Remove schema metadata usage for Protobuf extension |
|
Resolved | Alex Van Boxel |
|
|||||||
5.
|
BIP-1: Rename setRowOption to setOption on Option builder |
|
Resolved | Alex Van Boxel |
|
|||||||
6.
|
BIP-1: Deprecate and remove FieldType metadata |
|
Triage Needed | Unassigned |
|
