Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12615

HashQParserPlugin will throw an NPE for string hash key and documents have empty value

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 7.5, 8.0
    • None
    • None

    Description

      If I index documents where the partitionKeys keys field is missing from a few docs then the stream expression throws an NPE

      docs:

      [
      {"id" : "1", "search_term_s" : "query1"},
      {"id" : "2", "search_term_s" : "query1"},
      {"id" : "3"},
      {"id" : "4"}
      ]

       

      query

      search(test_empty, q="*:*", fl="search_term_s,id" , sort="search_term_s desc", qt="/export", partitionKeys="search_term_s")

       

      logs

      INFO - 2018-08-03 01:44:36.156; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.core.SolrCore.Request; [test_empty_shard1_replica_n1] webapp=/solr path=/stream params={expr=search(test_empty,+q%3D"*:*",+fl%3D"search_term_s,id"+,+sort%3D"search_term_s+desc",+qt%3D"/export",+partitionKeys%3D"search_term_s")&_=1533260573672} status=0 QTime=11
      INFO - 2018-08-03 01:44:36.160; [ ] org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
      INFO - 2018-08-03 01:44:36.162; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.common.cloud.ZkStateReader; Updated live nodes from ZooKeeper... (0) -> (1)
      INFO - 2018-08-03 01:44:36.164; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at localhost:9888 ready
      INFO - 2018-08-03 01:44:36.207; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.core.SolrCore.Request; [test_empty_shard1_replica_n1] webapp=/solr path=/export params={q=*:*&distrib=false&indent=off&fl=search_term_s,id&sort=search_term_s+desc&partitionKeys=search_term_s&fq={!hash+workers%3D1+worker%3D0}&wt=json&version=2.2} status=500 QTime=36
      ERROR - 2018-08-03 01:44:36.209; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.servlet.HttpSolrCall; null:java.io.IOException: java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:130)
      at org.apache.lucene.search.IndexSearcher.createWeight(IndexSearcher.java:743)
      at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:463)
      at org.apache.solr.search.DocSetUtil.createDocSetGeneric(DocSetUtil.java:151)
      at org.apache.solr.search.DocSetUtil.createDocSet(DocSetUtil.java:140)
      at org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1196)
      at org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:836)
      at org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1044)
      at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1563)
      at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1439)
      at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:586)
      at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1435)
      at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:375)
      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
      at org.apache.solr.handler.ExportHandler.handleRequestBody(ExportHandler.java:37)
      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
      at org.apache.solr.core.SolrCore.execute(SolrCore.java:2539)
      at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
      at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.Server.handle(Server.java:531)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:215)
      at org.apache.solr.search.HashQParserPlugin$HashQuery.createWeight(HashQParserPlugin.java:127)
      ... 54 more
      Caused by: java.lang.NullPointerException
      at org.apache.lucene.util.CharsRefBuilder.copyUTF8Bytes(CharsRefBuilder.java:122)
      at org.apache.solr.schema.FieldType.indexedToReadable(FieldType.java:378)
      at org.apache.solr.search.HashQParserPlugin$BytesHash.hashCode(HashQParserPlugin.java:300)
      at org.apache.solr.search.HashQParserPlugin$HashQuery$SegmentPartitioner.run(HashQParserPlugin.java:210)
      ... 55 more
      
      ERROR - 2018-08-03 01:44:36.213; [c:test_empty s:shard1 r:core_node2 x:test_empty_shard1_replica_n1] org.apache.solr.client.solrj.io.stream.ExceptionStream; java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:400)
      at org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:275)
      at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:54)
      at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:397)
      at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:83)
      at org.apache.solr.response.JSONWriter.writeMap(JSONResponseWriter.java:539)
      at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:181)
      at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:209)
      at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:325)
      at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:120)
      at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:71)
      at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
      at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:787)
      at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:524)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.Server.handle(Server.java:531)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.util.concurrent.ExecutionException: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
      at java.util.concurrent.FutureTask.report(FutureTask.java:122)
      at java.util.concurrent.FutureTask.get(FutureTask.java:192)
      at org.apache.solr.client.solrj.io.stream.CloudSolrStream.openStreams(CloudSolrStream.java:394)
      ... 49 more
      Caused by: java.io.IOException: --> http://192.168.0.4:8888/solr/test_empty_shard1_replica_n1/:java.lang.RuntimeException: java.lang.NullPointerException
      at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:222)
      at org.apache.solr.client.solrj.io.stream.CloudSolrStream$TupleWrapper.next(CloudSolrStream.java:484)
      at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:507)
      at org.apache.solr.client.solrj.io.stream.CloudSolrStream$StreamOpener.call(CloudSolrStream.java:494)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      ... 1 more

      The problem lies with HashQParserPlugin where we try to partition the data and if a document doesn't have a value for the partition field it throws an NPE

      Attachments

        1. SOLR-12615.patch
          4 kB
          Varun Thacker
        2. SOLR-12615.patch
          3 kB
          Varun Thacker

        Activity

          People

            Unassigned Unassigned
            varun Varun Thacker
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: