Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-390 High-Level Language for Samza
  3. SAMZA-483

A common representation of relational algebra for streaming SQL

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.9.0
    • None
    • sql

    Description

      Per discussion with criccomini and milinda, we agreed that it seems to be a good idea to define a common representation of relational algebra on top of the operators defined in the operator layer (see SAMZA-482), which can be the common base that we can use to generate the description/configuration of a Samza job.

      This common layer can also be used by DSL-like language parser as a result of parsing a DSL program.

      Some additional requirements needed in addition to pure relational algebra:
      1) the common representation should include window operators and stream operators (i.e. IStream/DStream/RStream)
      2) the common representation should include description on parallelism of the jobs (i.e. how many partitions the resultant Samza job will use)

      Some references:
      http://web.cs.wpi.edu/~mukherab/i/DCAPE.pdf
      https://cs.uwaterloo.ca/~david/cs848/stream-cql.pdf
      http://davis.wpi.edu/dsrg/PROJECTS/CAPE/publications.htm
      http://davis.wpi.edu/dsrg/PROJECTS/CAPE/slides.htm

      Attachments

        1. calcite-integration-prototype.patch
          39 kB
          Milinda Pathirage
        2. SAMZA-483-1.patch
          169 kB
          Milinda Pathirage
        3. SAMZA-483-2.patch
          187 kB
          Milinda Pathirage
        4. SAMZA-483-3.patch
          33 kB
          Milinda Pathirage
        5. SAMZA-483-4.patch
          1 kB
          Milinda Pathirage
        6. SAMZA-483-5.patch
          34 kB
          Milinda Pathirage
        7. SAMZA-483-6.patch
          34 kB
          Milinda Pathirage
        8. SAMZA-483-7.patch
          34 kB
          Milinda Pathirage

        Issue Links

          Activity

            People

              milinda Milinda Pathirage
              nickpan47 Yi Pan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: