Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39235

Make Catalog API be compatible with 3-layer-namespace

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • PySpark, R, SQL
    • None

    Description

      We can make Catalog API support 3 layer namespace: catalog_name.database_name.table_name

      Attachments

        1.
        		 Make CreateTable API and ListTables API compatible Sub-task Resolved Rui Wang
        2.
        		 GetTable, TableExists and DatabaseExists Sub-task Resolved Rui Wang
        3.
        		 CacheTable, isCached, UncacheTable, setCurrentCatalog, currentCatalog, listCatalogs Sub-task Resolved Rui Wang
        4.
        		 Add session catalog name for v1 database table and function Sub-task Resolved XiDuo You
        5.
        		 Make createTable and listTables in the python side support 3-layer-namespace Sub-task Resolved Ruifeng Zheng
        6.
        		 Make ListFunctions/getFunction/functionExists API compatible Sub-task Resolved Ruifeng Zheng
        7.
        		 Make RefreshTable be compatible with 3 layer namespace Sub-task Resolved Rui Wang
        8.
        		 Make GetTable, TableExists and DatabaseExists in the python side support 3-layer-namespace Sub-task Resolved Ruifeng Zheng
        9.
        		 Make *cache*, *catalog* in the python side support 3-layer-namespace Sub-task Resolved Ruifeng Zheng
        10.
        		 Make listColumns be compatible with 3 layer namespace Sub-task Resolved Ruifeng Zheng
        11.
        		 Make getDatabase and listDatabases compatible with 3 layer namespace Sub-task Resolved Zach Schuermann
        12.
        		 Make setCurrentDatabase compatible with 3 layer namespace Sub-task Resolved Zach Schuermann
        13.
        		 Make listDatabases / getDatabase / listColumns / refreshTable in PySpark support 3-layer-namespace Sub-task Resolved Ruifeng Zheng
        14.
        		 Make sure new catalog methods listed in API reference Sub-task Resolved Ruifeng Zheng
        15.
        		 Update two-parameter listColumns/getTable/getFunction/tableExists/functionExists functions docs to mention limitation Sub-task Resolved Rui Wang
        16.
        		 Make currentDatabase/setCurrentDatabase/listCatalogs in SparkR support 3L namespace Sub-task Resolved Ruifeng Zheng
        17.
        		 Implement databaseExists/getDatabase in SparkR support 3L namespace Sub-task Resolved Ruifeng Zheng
        18.
        		 Implement tableExists/getTable in SparkR for 3L namespace Sub-task Resolved Ruifeng Zheng
        19.
        		 Deprecate databaseName in listColumns in SparkR if needed Sub-task Reopened Unassigned
        20.
        		 Implement functionExists/getFunc in SparkR for 3L namespace Sub-task Resolved Ruifeng Zheng
        21.
        		 namespace should be null when database is null in the old constructors Sub-task Resolved Ruifeng Zheng
        22.
        		 Catalog.tableExists should handle nested namespace Sub-task Resolved Rui Wang
        23.
        		 Catalog.listTables() should respect currentCatalog Sub-task Resolved Wenchen Fan
        24.
        		 listCatalogs should also return spark_catalog even spark_catalog implementation is defaultSessionCatalog Sub-task Resolved Rui Wang

        Activity

          People

            Unassigned Unassigned
            amaliujia Rui Wang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: