Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-17050

Store the models compacted so we can store bigger models on zk

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • main (10.0), 9.4
    • 9.5
    • contrib - LTR, ltr
    • None

    Description

      We reached the limit of zk for storing LTR models. I want to avoid the
      usage of DefaultWrapperModel for as long as possible because we have
      deployed in an container orchestrator and this implementation can be
      really risky if you can not guarantee the presence of the model on
      disk all the time.
      So I want to use the managed model feature to upload some bigger
      models but zk is dying with OOM. What we noticed is that solr stores
      the models in a json indented file. By saving the models in compacted
      json our models will be 60% smaller.

      user mailing list thread: https://lists.apache.org/thread/5s5dq4j4bd74sfym7frwoqdokoh64s5l

      Attachments

        Issue Links

          Activity

            People

              cpoerschke Christine Poerschke
              florin.babes Florin Babes
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 20m
                  4h 20m