Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29903

Add documentation for recursiveFileLookup

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Documentation
    • None

    Description

      SPARK-27990 added a new option, recursiveFileLookup, for recursively loading data from a source directory. There is currently no documentation for this option.

      We should document this both for the DataFrame API as well as for SQL.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-30128 Promote remaining "hidden" PySpark DataFrameReader options to load APIs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-29899 Can not recursively lookup files in Hive table via SQL

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. SPARK-27990 Recursive data loading from file sources

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              nchammas Nicholas Chammas
              nchammas Nicholas Chammas
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: