Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33905

Can't configure response header size

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.0.0
    • None
    • Deploy
    • None

    Description

      We use JWT tokens to secure your spark UI. These tokens can get quite large. We have configured `spark.ui.requestHeaderSize`. However we also use the spark master UI as a proxy for our workers and for driver apps. When the HTTP request flow through the spark master UI we get this error when building the responses.

      We believe a `spark.ui.responseHeaderSize` would solve this issue. Jetty's HttpConfig has a property to set the response header max size.

       

      org.sparkproject.jetty.http.BadMessageException: 500: Request header too large
      at org.sparkproject.jetty.http.HttpGenerator.generateRequest(HttpGenerator.java:279)
      at org.sparkproject.jetty.client.http.HttpSenderOverHTTP$HeadersCallback.process(HttpSenderOverHTTP.java:231)
      at org.sparkproject.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
      at org.sparkproject.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
      at org.sparkproject.jetty.client.http.HttpSenderOverHTTP.sendHeaders(HttpSenderOverHTTP.java:62)
      at org.sparkproject.jetty.client.HttpSender.send(HttpSender.java:214)
      at org.sparkproject.jetty.client.http.HttpChannelOverHTTP.send(HttpChannelOverHTTP.java:85)
      at org.sparkproject.jetty.client.HttpChannel.send(HttpChannel.java:128)
      at org.sparkproject.jetty.client.HttpConnection.send(HttpConnection.java:201)

      Attachments

        Activity

          People

            Unassigned Unassigned
            jcc jcc
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: