Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-12964

Reported load higher than available disk space

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Duplicate
    • None
    • None
    • None
    • cassandra 3.9

    • Normal

    Description

      Nodetool status is reporting a node to have almost 2x the amount of space that it actually physically have. We are running 6x825 GB HDDs in JBOD per node. Currently the last node alcas11 is in bootstrap state. Nodetool is consistently reporting the following:

      [alcas11-p2e] ~ # /opt/cassandra/bin/nodetool -h alcas11-p2e status -r
      Datacenter: datacenter1
      =======================
      Status=Up/Down

      / State=Normal/Leaving/Joining/Moving
      – Address Load Tokens Owns (effective) Host ID Rack
      UN node8 3.27 TiB 256 29.6% 44cdab3e-ebb2-4f9a-9c7a-0b720ab81b7e rack1
      UN node5 8.29 TiB 256 31.1% 46d2dc3e-e2c8-487a-ab9f-cc499f45d68f rack1
      UN node9 3.32 TiB 256 30.2% 08e0bd5e-4247-42f1-976b-c4478f52277b rack1
      UN node10 3.36 TiB 256 29.7% 0d62d6df-e49c-44bd-a749-834ea8e612c0 rack1
      UN node1 3.42 TiB 256 30.5% e6763183-0a1c-4078-a8a9-8d28ab8b42ab rack1
      UN node4 3.34 TiB 256 30.4% d40d4db3-36d3-4311-adac-55754d1a13c3 rack1
      UN node3 3.28 TiB 256 29.5% 5a689467-e835-472d-807f-be38c03a4231 rack1
      UN node2 3.21 TiB 256 29.1% 0b66f5e9-a05c-45ec-9723-3b8fb700f510 rack1
      UN node7 3.23 TiB 256 29.4% d43c8527-edd8-4683-ba3e-54fc10a414c7 rack1
      UN node6 3.49 TiB 256 30.7% a64cb95d-ba63-4340-9e81-63d5e189217f rack1
      UJ node11-p2e 1.49 TiB 256 ? 6df0c42a-f3aa-4408-8876-51e0772ca2f2 rack1

      while the physical disk utilization on the machine itself is:

      [node5] ~ # df -h
      Filesystem Size Used Avail Use% Mounted on
      /dev/sdb1 825G 638G 145G 82% /data1
      /dev/sdc1 825G 532G 251G 68% /data2
      /dev/sdd1 825G 601G 182G 77% /data3
      /dev/sde1 825G 572G 212G 74% /data5
      /dev/sdf1 825G 552G 231G 71% /data6
      /dev/sdg1 825G 724G 59G 93% /data4

      Any clue what might cause this or any other information you need me to provide?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              feribg Feras
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: