Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.0.0
-
None
-
None
Description
We use JWT tokens to secure your spark UI. These tokens can get quite large. We have configured `spark.ui.requestHeaderSize`. However we also use the spark master UI as a proxy for our workers and for driver apps. When the HTTP request flow through the spark master UI we get this error when building the responses.
We believe a `spark.ui.responseHeaderSize` would solve this issue. Jetty's HttpConfig has a property to set the response header max size.
org.sparkproject.jetty.http.BadMessageException: 500: Request header too large
at org.sparkproject.jetty.http.HttpGenerator.generateRequest(HttpGenerator.java:279)
at org.sparkproject.jetty.client.http.HttpSenderOverHTTP$HeadersCallback.process(HttpSenderOverHTTP.java:231)
at org.sparkproject.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
at org.sparkproject.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
at org.sparkproject.jetty.client.http.HttpSenderOverHTTP.sendHeaders(HttpSenderOverHTTP.java:62)
at org.sparkproject.jetty.client.HttpSender.send(HttpSender.java:214)
at org.sparkproject.jetty.client.http.HttpChannelOverHTTP.send(HttpChannelOverHTTP.java:85)
at org.sparkproject.jetty.client.HttpChannel.send(HttpChannel.java:128)
at org.sparkproject.jetty.client.HttpConnection.send(HttpConnection.java:201)