Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12460

Filtercache getting filled up when domain switches are involved in Json facets

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 8.0
    • Fix Version/s: None
    • Component/s: Facet Module
    • Labels:
      None

      Description

      FacetProcessor.java :: handleJoinField calls SolrIndexSearcher.getDocSet(..domainquery..) which eventually cache join queries like:

      key:{  !join from=[vin_l-value] to=[vin_l-value] }   {   !cache=false }   ConstantScore(BitSetDocTopFilter) 
      value:   org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
      

      and when we execute the same query again, some of the entries are not getting used at all. Please note: the filterCache entries are not getting used strictly when we have more than 1 shard in the collection.

      Sample: car_stuff collection with multiple doc-types: vehicle, claims, defects - 10 shards

      http://localhost:8983/solr/car_stuff/query?rows=0&q=doc_type_s:vehicle AND v_model_s:model_009&json.facet={  
         models:{  
            type:terms,
            field:"v_model_s",
            limit:-1,
            facet:{  
               year_per_model:{  
                  type:terms,
                  field:"v_year_i",
                  limit:-1,
                  facet:{  
                     claim_month:{  
                        domain:{  
                           join:{  
                              from:"vin_l",
                              to:"vin_l"
                           },
                           filter:"doc_type_s:claim"
                        },
                        type:terms,
                        field:"claim_opcode_s",
                        limit:-1
                     }
                  }
               }
            }
         }
      }
      

      After executing this query for first time, filterCache for one of the cores looks like:

            "CACHE.searcher.filterCache":{
              "lookups":151,
              "hits":49,
              "cumulative_evictions":0,
              "size":103,
              "hitratio":0.32,
              "evictions":0,
              "cumulative_lookups":151,
              "cumulative_hitratio":0.32,
              "warmupTime":0,
              "inserts":103,
              "cumulative_inserts":103,
              "cumulative_hits":49}
      

      2nd time executing same query:

            "CACHE.searcher.filterCache":{
              "lookups":303,
              "hits":151,
              "cumulative_evictions":0,
              "size":153,
              "hitratio":0.5,
              "evictions":0,
              "cumulative_lookups":303,
              "cumulative_hitratio":0.5,
              "warmupTime":0,
              "inserts":153,
              "cumulative_inserts":153,
              "cumulative_hits":151}
      

      Looking into the entries of the filterCache looks like this:

      .....
       key: v_year_i:[1977 TO 1977] value: org.apache.solr.search.BitDocSet@1524fa9c lastAccessed:457,
       key: v_model_s:model_003 value: org.apache.solr.search.BitDocSet@61f348dd lastAccessed:157,
      key:{  !join from=[vin_l-value] to=[vin_l-value] }   {   !cache=false }   ConstantScore(BitSetDocTopFilter) 
      value:   org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
      .....
      

      The collection backup is uploaded on the JIRA.

        Attachments

        1. car_stuff.zip
          15.97 MB
          Amrit Sarkar

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sarkaramrit2@gmail.com Amrit Sarkar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: