Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Not A Bug
-
5.5
-
None
-
None
Description
I'm facing a weird issue where Basic authentications are failing randomly. The error is originating as "Invalid key" from PKIAuthenticationPlugin.java followed by missing userPrincipal. Here's the stacktrace:
ERROR923629[qtp466002798-20] -
org.apache.solr.security.PKIAuthenticationPlugin.doAuthenticate(PKIAuthenticationPlugin.java:125)
- Invalid key
INFO923630[qtp466002798-20] -
org.apache.solr.security.RuleBasedAuthorizationPlugin.checkPathPerm(RuleBasedAuthorizationPlugin.java:144) - request has come without principal. failed permission
org.apache.solr.security.RuleBasedAuthorizationPlugin$Permission@1a343033
INFO923630[qtp466002798-20] -
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:429) -
USER_REQUIRED auth header null context : userPrincipal: [null] type:
[READ], collections: [knowledge,], Path: [/select] path : /select params
:df=text&distrib=false&qt=/select&preferLocalShards=false&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=
http://xx.xxx.x.222:8983/solr/knowledge/|http://xx.xxx.xxx.246:8983/solr/knowledge/&rows=3&version=2&q=:&NOW=1463512962899&isShard=true&wt=javabin
My security.json
{
"authentication": {
"blockUnknown": false,
"class": "solr.BasicAuthPlugin",
"credentials":
},
"authorization": {
"class": "solr.RuleBasedAuthorizationPlugin",
"user-role":
,
"permissions": [
,
,
,
,
,
,
,
{ "name": "config-edit", "role": [ "admin" ] } ]
}
}
Sample Java client:
SolrClient client = new CloudSolrClient("zoohost1:2181,zoohost2:2181,zoohost3:2181");
((CloudSolrClient)client).setDefaultCollection(DEFAULT_COLLECTION);
ModifiableSolrParams param = getSearchSolrQuery();
SolrRequest<?> solrRequest = new QueryRequest(param);
solrRequest.setBasicAuthCredentials(USER, PASSWORD);
try{
for(int j=0;j<20;j++)
}catch(Exception ex){
}
private static ModifiableSolrParams getSearchSolrQuery() {
ModifiableSolrParams solrParams = new ModifiableSolrParams();
solrParams.set("q", ":");
solrParams.set("qt","/select");
solrParams.set("rows", "3");
return solrParams;
}
Sometimes, the error is being thrown at the very first call, otherwise in the middle of the iteration. It's consistent with my custom user or the default "solr/SolrRocks". I even cleaned up the zookeeper data, started the cluster from fresh, uploaded the security.json, but without any luck.
Incidentally, I'm also seeing similar exception if I try to start and stop a node in the cluster while indexing is in process. Here's the log:
ERROR 19543[qtp466002798-21] - org.apache.solr.security.PKIAuthenticationPlugin.doAuthenticate(PKIAuthenticationPlugin.java:125) - Invalid key
INFO 19543[qtp466002798-21] - org.apache.solr.security.RuleBasedAuthorizationPlugin.checkPathPerm(RuleBasedAuthorizationPlugin.java:144) - request has come without principal. failed permission org.apache.solr.security.RuleBasedAuthorizationPlugin$Permission@101fe889
INFO 19543[qtp466002798-21] - org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:429) - USER_REQUIRED auth header null context : userPrincipal: [null] type: [WRITE], collections: [knowledge,], Path: [/update] path : /update params :update.distrib=FROMLEADER&distrib.from=http://xx.xxx.xxx.246:8983/solr/knowledge/&wt=javabin&version=2
Based on the source code, it seems like the error is generated due to timeout issues. I bumped up SOLR_OPTS="$SOLR_OPTS -Dpkiauth.ttl=50000" to 50 sec, but didn't make any difference.
My cluster contains 2 shards with 1 replica each.
I'll appreciate if someone can take a look and provide me some pointers.