Details
-
Sub-task
-
Status: Triage Needed
-
P0
-
Resolution: Fixed
-
None
-
None
Description
In Python 2 you could write bytes data to BigQuery. This is tested in
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/big_query_query_to_table_it_test.py#L186
Python 3 does not support
json.dumps({'test': b'test'})
which is used to encode the data in
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery_tools.py#L959
How should writing bytes to BigQuery be handled in Python 3?
- Forbid writing bytes into BigQuery on Python 3
- Guess the encoding (utf-8?)
- Pass the encoding to BigQuery
cc: tvalentyn