Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9609

When the term of more than 16, highlight the query does not return

Details

    • Wish
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 7.7.3
    • None
    • core/search
    • None
    • New

    Description

      I noticed that when there are too many terms, the highlighted query is restricted

      I know that in TermInSetQuery, when there are fewer terms, BOOLEAN_REWRITE_TERM_COUNT_THRESHOLD = 16 will be used to improve query efficiency

      static final int BOOLEAN_REWRITE_TERM_COUNT_THRESHOLD = 16;
      
      public Query rewrite(IndexReader reader) throws IOException {
          final int threshold = Math.min(BOOLEAN_REWRITE_TERM_COUNT_THRESHOLD, BooleanQuery.getMaxClauseCount());
          if (termData.size() <= threshold) {
            BooleanQuery.Builder bq = new BooleanQuery.Builder();
            TermIterator iterator = termData.iterator();
            for (BytesRef term = iterator.next(); term != null; term = iterator.next()) {
              bq.add(new TermQuery(new Term(iterator.field(), BytesRef.deepCopyOf(term))), Occur.SHOULD);
            }
            return new ConstantScoreQuery(bq.build());
          }
          return super.rewrite(reader);
        }
      

       When the term of the query statement exceeds 16, the createWeight method in TermInSetQuery will be used

      public Weight createWeight(IndexSearcher searcher, boolean needsScores, float boost) throws IOException {
          return new ConstantScoreWeight(this, boost) {
      
            @Override
            public void extractTerms(Set<Term> terms) {
              // no-op
              // This query is for abuse cases when the number of terms is too high to
              // run efficiently as a BooleanQuery. So likewise we hide its terms in
              // order to protect highlighters
            }
      
            ......
        }
      

      I want to ask, why do you say "we hide its terms in order to protect highlighters"

      How to implement such " protect highlighters"?

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            wang_feicheng WangFeiCheng
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: