Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4608

Improve / Fix documentation examples for Beam 2.4.0 for TextIO and BigQueryIO

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • 2.4.0
    • None
    • io-java-gcp, io-java-text
    • None

    Description

      The documentation that is part of the code of TextIO has some usage examples.

      The description of how to achieve custom naming with the new FileIO in combination with the TextIO could be improved as the code in the "Advanced Features" is not really fully fleshed out and probably not in sync with how it should be done in Beam 2.4.0.

      https://beam.apache.org/documentation/sdks/javadoc/2.4.0/index.html?org/apache/beam/sdk/io/TextIO.html

      See SO Question: 

      https://stackoverflow.com/questions/50930409/how-to-replace-withfilenamepolicy-in-apache-beam-2-4

      The second part that could be improved is in the gcp.BigQueryIO class:

      https://github.com/apache/beam/blob/release-2.4.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L217

      The example is also not fully fleshed out, but most importantly the
      BigQueryIO.write() should be replaced with BigQueryIO.<Quote>write() as otherwise it won't compile.

      Attachments

        Activity

          People

            Unassigned Unassigned
            james-woods Tobias Kaymak
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: