Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20914

Javadoc contains code that is invalid

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 2.1.1
    • 2.1.2, 2.2.0, 2.3.0
    • Documentation
    • None

    Description

      i was looking over the dataset and noticed the code on top that does not make sense in java.

       // To create Dataset<Row> using SparkSession
         Dataset<Row> people = spark.read().parquet("...");
         Dataset<Row> department = spark.read().parquet("...");
      
         people.filter("age".gt(30))
           .join(department, people.col("deptId").equalTo(department("id")))
           .groupBy(department.col("name"), "gender")
           .agg(avg(people.col("salary")), max(people.col("age")));
      

      invalid parts:

      • "age".gt(30)
      • department("id")

      Attachments

        Activity

          People

            srowen Sean R. Owen
            cristian.teodor Cristian Teodor
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: