Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7691

SolrEntityProcessor as SubEntity doesn't work with delta-import

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 5.0, 5.1, 5.2, 5.2.1
    • None
    • None

    Description

      I've used the SolrEntityProcessor as sub-entity in the dataimporter like this

      <dataConfig>
          <document name="products">
              <entity
                  name="outer"
                  dataSource="my_datasource"
                  pk="id"
                  query="..."
                  deltaQuery="..."
                  deltaImportQuery="..."
              >
                  <entity
                      name="solr"
                      processor="SolrEntityProcessor"
                      url="http://127.0.0.1:8983/solr/${solr.core.name}"
                      query="Xid:${outer.Xid}"
                      rows="1"
                      fl="Id,FieldA,FieldB"
                      wt="javabin"
                  />
              </entity>
          </document>
      </dataConfig>
      

      Recently I decided to upgrade to 5.x, but the delta-import stopped working. At all it looks like the http-connection used by the SolrEntityProcessor is closed right after the request/response, because the first document is indexed properly and for the second connection the dataimport fetches the record from the database, but after that exists

      This is the stacktrace taken from the log

      java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
          at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270)
          at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:444)
          at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:482)
          at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:461)
      Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
          at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)
          at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:363)
          at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:224)
          ... 3 more
      Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.IllegalStateException: Connection pool shut down
          at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:62)
          at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246)
          at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
          at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:514)
          at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
          ... 5 more
      Caused by: java.lang.IllegalStateException: Connection pool shut down
          at org.apache.http.util.Asserts.check(Asserts.java:34)
          at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:184)
          at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:217)
          at org.apache.http.impl.conn.PoolingClientConnectionManager.requestConnection(PoolingClientConnectionManager.java:184)
          at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:415)
          at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
          at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:466)
          at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
          at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
          at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
          at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
          at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
          at org.apache.solr.handler.dataimport.SolrEntityProcessor.doQuery(SolrEntityProcessor.java:198)
          at org.apache.solr.handler.dataimport.SolrEntityProcessor.buildIterator(SolrEntityProcessor.java:147)
          at org.apache.solr.handler.dataimport.SolrEntityProcessor.nextRow(SolrEntityProcessor.java:128)
          at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)
          ... 8 more
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              KingCrunch Sebastian Krebs
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: