Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9735

Umbrella JIRA for Auto Scaling and Cluster Management in SolrCloud

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 7.4
    • AutoScaling
    • None

    Description

      As SolrCloud is now used at fairly large scale, most users end up writing their own cluster management tools. We should have a framework for cluster management in Solr.
      In a discussion with noble.paul, we outlined the following steps w.r.t. the approach to having this implemented:

      • Basic API calls for cluster management e.g. utilize added nodes, remove a node etc. These calls would need explicit invocation by the users to begin with. It would also specify the strategy to use. For instance I can have a strategy called optimizeCoreCount which would target to have an even no:of cores in each node . The strategy could optionally take parameters as well
      • Metrics and stats tracking e.g. qps, etc. These would be required for any advanced cluster management tasks e.g. maintain a qps of 'x' by auto-adding a replica (using a recipe) etc. We would need collection/shard/node level views of metrics for this.
      • Recipes: combination of multiple sequential/parallel API calls based on rules. This would be complicated specially as most of these would be long running series of tasks which would either have to be rolled back or resumed in case of a failure.
      • Event based triggers that would not require explicit cluster management calls for end users.

      Attachments

        Issue Links

          depends upon

          New Feature - A new feature of the product, which has yet to be developed. SOLR-9241 Rebalance API for SolrCloud

          • Major - Major loss of function.
          • Open
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SOLR-11995 Introduce a swap operation in computing autoscaling suggestions

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11359 An autoscaling/suggestions endpoint to recommend operations

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SOLR-12496 Support an enum of values in Tags in Autoscaling policies

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SOLR-12500 Add support for '<=' and '>=' operators in the autoscaling policy syntax

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11990 Make it possible to co-locate replicas of multiple collections together in a node

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-12495 Enhance the Autoscaling policy syntax to evenly distribute replicas among nodes

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-9731 Add jvm-wide JMX statistics for Solr

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-11005 inconsistency when maxShardsPerNode used along with policies

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11371 Allow TriggerListeners and Actions to be created from the blob store

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11989 Make it possible to place a replica of a collection on all nodes satisfying a given property

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11991 Configure min and max replication factor in policy rules

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SOLR-9317 ADDREPLICA command should be more flexible and add 'n' replicas to a collection,shard

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11081 Expose metrics for AutoScaling plugins

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11189 Provisioning actions for autoscaling

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11985 Allow percentage in replica attribute in policy

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11986 Allow percentage in freedisk attribute in autoscaling policy rules

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SOLR-11987 Support a comment attribute in policy rules and preferences

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-11158 set-cluster-preferences & set-cluster-policy must accept null value to delete the entries

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-12038 SPLITSHARD should check disk space before performing the operation

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-12240 Triggers don't properly track "waitFor"

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-11975 searchRate metrics are not handled correctly

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-11065 ExecutePlanAction should execute operations in parallel

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-11996 Suggestions API/UI should warn if there exist preferences that cannot ever be used

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-11997 Suggestions API/UI should show a message when violations exist but no suggestions are possible

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SOLR-12208 Don't use "INDEX.sizeInBytes" as a tag name in policy calculations

          • Major - Major loss of function.
          • Closed
          There are no Sub-Tasks for this issue.

          Activity

            People

              shalin Shalin Shekhar Mangar
              anshum Anshum Gupta
              Votes:
              5 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1,344h
                  1,344h
                  Remaining:
                  Remaining Estimate - 1,344h
                  1,344h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified