Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7906

Perf regression in SQL Query3 in Dataflow

Details

    • Bug
    • Status: Resolved
    • P3
    • Resolution: Cannot Reproduce
    • None
    • Missing
    • dsl-sql, runner-dataflow
    • None

    Description

      Nexmark shows perf regression in SQL Query3 starting on July 30 2019: https://apache-beam-testing.appspot.com/explore?dashboard=5670405876482048

      There doesn't seem to be a lot of changes to SQL around that date and the one that was there doesn't seem relevan to the query: https://github.com/apache/beam/commits/master/sdks/java/extensions/sql

      Direct runner shows a slight perf decrease as well: https://apache-beam-testing.appspot.com/explore?dashboard=5084698770407424 while Spark runner doesn't: https://apache-beam-testing.appspot.com/explore?dashboard=5138380291571712

      The query in question is a join with a simple filter condition: https://github.com/apache/beam/blob/b8aa8486f336df6fc9cf581f29040194edad3b87/sdks/java/testing/nexmark/src/main/java/org/apache/beam/sdk/nexmark/queries/sql/SqlQuery3.java#L69

      Other queries don't seem to be affected

      Attachments

        1. direct.png
          26 kB
          Anton Kedin
        2. dataflow.png
          29 kB
          Anton Kedin

        Activity

          People

            Unassigned Unassigned
            kedin Anton Kedin
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: