Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6945

Use minExtInfo=true to reduce the amount of data queried from Atlas

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.11.0
    • Extensions
    • None

    Description

      When ReportLineageToAtlas queries the entities from Atlas, the AtlasEntityWithExtInfo response object also contains all the referred entities with all their data by default which leads to huge data transfers between Atlas and NiFi. On the other hand, this extended data of the related entities are not used by the reporting task, so they could be excluded from the query (using minExtInfo=true parameter) in order to reduce the response size.

      Attachments

        Issue Links

          Activity

            People

              turcsanyip Peter Turcsanyi
              turcsanyip Peter Turcsanyi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h