Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17799 InterfaceStability annotation
  3. SPARK-17900

Mark the following Spark SQL APIs as stable

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0
    • SQL
    • None

    Description

      Mark the following stable:

      Dataset/DataFrame

      • functions, since 1.3
      • ColumnName, since 1.3
      • DataFrameNaFunctions, since 1.3.1
      • DataFrameStatFunctions, since 1.4
      • UserDefinedFunction, since 1.3
      • UserDefinedAggregateFunction, since 1.5
      • Window and WindowSpec, since 1.4

      Data sources:

      • DataSourceRegister, since 1.5
      • RelationProvider, since 1.3
      • SchemaRelationProvider, since 1.3
      • CreatableRelationProvider, since 1.3
      • BaseRelation, since 1.3
      • TableScan, since 1.3
      • PrunedScan, since 1.3
      • PrunedFilteredScan, since 1.3
      • InsertableRelation, since 1.3

      Keep the following experimental / evolving:

      Data sources:

      • CatalystScan (tied to internal logical plans so it is not stable by definition)

      Structured streaming:

      • all classes (introduced new in 2.0 and will likely change)

      Dataset typed operations (introduced in 1.6 and 2.0 and might change, although probability is low)

      • all typed methods on Dataset
      • KeyValueGroupedDataset
      • o.a.s.sql.expressions.javalang.typed
      • o.a.s.sql.expressions.scalalang.typed
      • methods that return typed Dataset in SparkSession

      Attachments

        Activity

          People

            rxin Reynold Xin
            rxin Reynold Xin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: