Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14673

Add CLI for Streaming Expressions

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • streaming expressions
    • None

    Description

      This ticket will provide a simple CLI that will run a Streaming Expression from the command line and return the results as a delimited result set. This will allow Streaming Expressions to be used from the command line to extract data as well as load data into Solr. 

      Sample syntax:

      bin/expr expr_file

      This will run the expression in expr_file.

      Output will be to standard out as delimited records.  

      Standard-In, Standard-out and Pipe Composition.

      The CLI can read from standard-in and write to standard-out in delimited records. This allows Streaming Expressions to be composed using unix pipes with other command line tools and other streaming expressions. Example syntax:

      cat data.csv | ./bin/expr load.expr 

      The load.expr file uses the stndin() function to read from standard-in and load date to Solr. Here is a sample load.expr

      let(cli-zkhost="localhost:9983",
          
          update(collection1,
                 parseCSV(stndin()))
       

      In the example above the let expression is used to set the cli-zkhost and then run the Streaming Expression:

      update(collection1,
             parseCSV(stndin())

      The stndin function reads the data from standard-in. The parseCSV function parses the lines into tuples and the update function sends the data to collection1 for indexing.

       

      Querying Solr and Pipe Composition

      The CLI can also be used to query Solr and pipe the result to other applications on the command line. This can automate activities like alerting, data backup, replication etc...

       

       

       

       

      Attachments

        1. SOLR-14673.patch
          13 kB
          Joel Bernstein
        2. SOLR-14673.patch
          14 kB
          Joel Bernstein
        3. SOLR-14673.patch
          16 kB
          Joel Bernstein
        4. SOLR-14673.patch
          20 kB
          Joel Bernstein
        5. SOLR-14673.patch
          21 kB
          Joel Bernstein

        Issue Links

          Activity

            People

              jbernste Joel Bernstein
              jbernste Joel Bernstein
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: