Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3398

Inaccurate arithmetic operation in LookupTableToHFileJob#calculateShardNum

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • v2.4.0
    • None
    • None

    Description

      There're two issues with the following code:

          private int calculateShardNum(KylinConfig kylinConfig, long dataSize) {
              long shardSize = kylinConfig.getExtTableSnapshotShardingMB() * 1024 * 1024;
              return dataSize < shardSize ? 1 : (int) (Math.ceil(dataSize / shardSize));
      

      getExtTableSnapshotShardingMB returns an int. The multiplication is done using 32-bit arithmetic, and then used in a context that expects an expression of type "long".

      Second, Math.ceil expects a double. The integer division would lose some precision.

      Attachments

        Issue Links

          Activity

            People

              nichunen nichunen
              yuzhihong@gmail.com Ted Yu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: