Uploaded image for project: 'Nutch'
  1. Nutch
  2. NUTCH-2911

Add cleanup call in Fetcher.java

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 1.18
    • 1.19
    • fetcher
    • Patch Available

    Description

      Fetcher's inner class, FetcherRun overrides Hadoop Mapper's run().
      Even though Nutch's FetcherRun doesn't need an explicit call the Mapper's cleanup() (Which is a blank function), it would increase the readability and completeness of the run Method to do so.
      Ideally, every implementation of Mapper is supposed to do the following tasks:
      1. Perform Setup

      2.  Call map on the data set

      3. Perform cleanups.

      Moreover, in case a custom Fetcher is written extending Fetcher.java cleanup could get easily missed out.

      PR for Fix: here

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              prakharchaube Prakhar Chaube
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: