Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-12975

Limit number of POST on /subjects for Confluent Schema Registry

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Extensions
    • None

    Description

      Right now, when retrieving a schema by ID, we first do

      // GET /schemas/ids/{int: id}

      but that does not give us everything, so... we get the subjects associated

      // GET /schemas/ids/{int: id}/subjects

      which is a list of <topic>-key or <topic>-value depending on what the schema is associated with for the topic

      Then, we recursively use the subjects calling

      // POST /subjects/(string: subject)

      until we get one 200 response. From this response we get all the information we want.

      While this is working, the problem is that NiFi may not be authorized to call this API for some subjects. So depending on the order of the list of subjects, we may be causing a lot of 403 errors on the Schema Registry side and its audit logs.

      When the reader is called from a Kafka processor, we may be able to pass the topic name information that would give us a way to be smarter about which subjects to call and avoid the 403 errors. That would not solve the problem in all possible scenarios but that would likely help in most cases.

      Attachments

        Activity

          People

            Unassigned Unassigned
            pvillard Pierre Villard
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: