Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
  3. FLINK-13983

Launch task executor with new memory calculation logics

    XMLWordPrintableJSON

Details

    • The usage of environment variable FLINK_TM_HEAP is deprecated. Please, use Flink option "taskmanager.memory.total-process.size" in yaml configuration file to set the total task executor memory. See FLIP-49 for more details.

    Description

      • Invoke data structures and utilities introduced in Step 2 to generate JVM parameters and dynamic configurations for launching new task executors.
        • In startup scripts
        • In resource managers
      • Task executor uses data structures and utilities introduced in Step 2 to set memory pool sizes and slot resource profiles.
        • MemoryManager
        • ShuffleEnvironment
        • TaskSlotTable

      Implement this step as separate code paths only for the new mode.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-14404 the jvm heap size from calculateTaskManagerHeapSizeMB in config.sh is different from TaskManagerServices.calculateHeapSizeMB

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #9801

          Activity

            People

              xtsong Xintong Song
              xtsong Xintong Song
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m