Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16428

Spark file system watcher not working on Windows

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 1.6.2
    • None
    • Ubuntu 15.10 64 bit, Windows 7 Enterprise 64 bit

    Description

      Two people tested Apache Spark on their computers...

      [Spark Download - http://i.stack.imgur.com/z1oqu.png]

      We downloaded the version of Spark prebuild for Hadoop 2.6, went to the folder /spark-1.6.2-bin-hadoop2.6/, created a "tmp" directory, went to that directory, and ran:

      $ bin/run-example org.apache.spark.examples.streaming.HdfsWordCount tmp

      I added arbitrary files content1 and content2dssdgdg to that "tmp" directory.

      -------------------------------------------
      Time: 1467921704000 ms
      -------------------------------------------
      (content1,1)
      (content2dssdgdg,1)

      -------------------------------------------
      Time: 1467921706000 ms

      Spark detected those files with the above terminal output on my Ubuntu 15.10 laptop, but not on my colleague's Windows 7 Enterprise laptop.

      This is preventing us from getting work done with Spark.

      Link: http://stackoverflow.com/questions/38254405/spark-file-system-watcher-not-working-on-windows

      Attachments

        Activity

          People

            Unassigned Unassigned
            johnreed2 John-Michael Reed
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: