Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12407

edismax boost performance regression from switch to FunctionScoreQuery

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 7.3
    • 9.0
    • None
    • None

    Description

      Assertion: FunctionScoreQuery uses the iterator style API (advanceExact + doubleValue). BoostedQuery uses the "old" api (just a single call to doubleValue). In an edismax boost this means the boost function is called twice for every document being scored in 7.3 instead of once in 7.2.

      I'm seeing ~50% increase in query response time after upgrading from 7.2 to 7.3 (600ms to 900ms). My queries use an edismax boost something like:

      if(termfreq(type,"A"),product(map(field1,3,3,1.5,1),map(field1,4,4,1.9,1),if(def(field2,false),product(map(field1,1,1,0.6,1),map(field1,2,2,0.7,1),if(not(exists(field1)),0.6,1),map(field3,0,0,1.3,1)),product(map(field1,1,1,0.7,1),map(field1,2,2,1.1,1),if(not(exists(field1)),0.90,1),map(field3,0,0,1.50,1)))),1)

      This boost is likely (surely?) suboptimal but LUCENE-8099 appears to have introduced this performance regression (poured proverbial oil on my smouldering fire). If I change ExtendedDismaxQParser back to using the deprecated BoostedQuery I get the 600ms solr 7.2 response time back.

      It appears FunctionScoreQuery invokes the boost function twice for each document. Once with a call to exists() from advanceExact(), then a second time from the call chain following scores.doubleValue().

      I don't know if that's the cause of the slowdown but I'm definitely seeing a slowdown that disappears when I revert part of LUCENE-8099.

      I've attached some flamegraphs comparing 7.2 and 7.3. The frame FunctionScoreQuery$FunctionScoreWeight$1.score in solr-7.3.svg show 2 "towers". One for advanceExact (calling exists()), the other for doubleValue() which ends up similar to solr-7.2.svg.

      Attachments

        1. restore-boosted-query.patch
          1 kB
          Will Currie
        2. solr-7.2.svg
          54 kB
          Will Currie
        3. solr-7.3.svg
          66 kB
          Will Currie

        Issue Links

          Activity

            People

              dsmiley David Smiley
              wscurrie Will Currie
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: