XMLWordPrintableJSON

Details

    Description

      Now after FLINK-14063 , operators will get all manage memory of TaskManager,  The cost of over allocate memory is very high, lead to performance regression of small batch sql jobs:

      • Allocate memory will have the cost of memory management algorithm.
      • Allocate memory will have the cost of memory initialization, will set all memory to zero. And this initialization will require the operating system to actually allocate physical memory.
      • Over allocate memory will squash the file cache too.

      We can optimize the operator algorithm, apply lazy allocation, and avoid meaningless memory allocation.

      Attachments

        Issue Links

          Activity

            People

              lzljs3620320 Jingsong Lee
              lzljs3620320 Jingsong Lee
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m