Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15446

Per-thread stack size is too small on aarch64 CentOS

    XMLWordPrintableJSON

Details

    • Code - Bug - Unclear Impact
    • Normal
    • Low Hanging Fruit
    • User Report
    • Java8, Java11, OpenJDK, Linux, ARM
    • Hide

      CI

      Show
      CI

    Description

      Hi all,

      I found an issue when I tried to start cassandra on my aarch64 CentOS7.6, however no errors on Ubuntu. Of course I could increase -Xss in jvm.options to fix it, but this issue also caused Cassandra's docker images from docker hub could not run containers on this OS.

      The information of my current environment and root cause of this issue were shown below.

      Error

      The stack size specified is too small, Specify at least 328k
      Error: Could not create the Java Virtual Machine.
      Error: A fatal exception has occurred. Program will exit.

      Version

      Cassandra 2.1.21 2.2.15 3.0.19 3.11.5 

      Environment

      $ lscpu

      Architecture: aarch64
      Byte Order: Little Endian

      $ uname -m

      aarch64

      $ java -version

      openjdk version "1.8.0_181"
      OpenJDK Runtime Environment (build 1.8.0_181-b13)
      OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
      [root@localhost apache-cassandra-3.11.5]# cat /etc/os-release

      $ cat /etc/os-release

      NAME="CentOS Linux"
      VERSION="7 (AltArch)"
      ID="centos"
      ID_LIKE="rhel fedora"
      VERSION_ID="7"
      PRETTY_NAME="CentOS Linux 7 (AltArch)"
      ANSI_COLOR="0;31"
      CPE_NAME="cpe:/o:centos:centos:7"
      HOME_URL="https://www.centos.org/"
      BUG_REPORT_URL="https://bugs.centos.org/"

      Root Cause

      Checked openjdk-1.8.0 source code, the min stack size is calculated by StackYellowPage, StackRedPage, StackShadowPage, OS page size. Among those parameters, default OS page size of aarch64 CentOS 7.6 is 64K, however aarch64 Ubuntu 18.04 and X86 CentOS are both 4K.

      This difference causes JVM on aarch64 Ubuntu 18.04 needs 164K per-thread stack size, but 328K required on aarch64 CentOS 7.6.

      The formula is 

      os::Linux::min_stack_allowed = MAX2(os::Linux::min_stack_allowed,
      (size_t)(StackYellowPages+StackRedPages+StackShadowPages) * Linux::page_size() +
      (2*BytesPerWord COMPILER2_PRESENT(+1)) * Linux::vm_default_page_size());

      Parameters on aarch64 CentOS7.6

      intx StackRedPages = 1 
      intx StackShadowPages = 1 
      intx StackYellowPages = 1 

      pageSize 64K

      BytesPerWord 8

      vm_default_page_size 8K

      As a result, we have min_stack_allowed = (1 + 1 + 1) * 64K + (2 * 8 + 1) * 8K = 328K

       

      I could see some similar issues asked for specified achitecture, but no root cause analyzed. I hope this could help you decide proper stack size for all common OS.

      If you have any suggestion, pls let me know.

       

      Attachments

        Issue Links

          Activity

            People

              mck Michael Semb Wever
              maaaace Heming Fu
              Michael Semb Wever
              David Capwell, Michael Semb Wever
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: