Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-3030

Separate operator information form generated code in SPOOF ops

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • SystemDS 2.2
    • None
    • Compiler

    Description

      Certain information about generated operators, like aggregation type, number of intermediate vectors needed, transpose side input, sequence input, etc, is stored directly in the compiled class. Until now four problems related to this fact were identified:

      • The information is needed by the Java and the CUDA verison of SPOOF ops, leading to either code/data duplication or
      • Need to compile the Java version also when CUDA codegen is used to have that operator information available
      • The compile cache is identifying already compiled plans via the generated java class, again requiring Java compilation when using CUDA codegen
      • Memory estimates of intermediates in SPOOF ops, which are currently missing, require information which is stored in the compiled operator.

      Factoring out the meta information into its own class mitigates the described shortcomings.

      Attachments

        Activity

          People

            markd Mark Dokter
            markd Mark Dokter
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: