Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1367

Crash at startup after full disk due to corrupt tablet metadata

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 0.5.0
    • None
    • tablet
    • None

    Description

      As reported by Nick Wolf.

      The disk filled up on their tablet server, resulting in a bad metadata file being written. This was the error message on tablet server startup:

      F0309 06:57:50.281460 32105 tablet_server_main.cc:49] Check failed: _s.ok() Bad status:Corruption: Failed to load FS layout: Could not read records from container
      /var/lib/kudu/tserver/data/b1940ca83ce44da3bc6bf4a36f3d5e90:
      Could not read body from proto container file
      /var/lib/kudu/tserver/data/b1940ca83ce44da3bc6bf4a36f3d5e90.metadata:
      File size not large enough to be valid: Proto container file
      /var/lib/kudu/tserver/data/b1940ca83ce44da3bc6bf4a36f3d5e90.metadata:
      tried to read /var/lib/kudu/tserver/data/b1940ca83ce44da3bc6bf4a36f3d5e90.metadata
      bytes at offset 30 but file size is only 114688
      

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            mpercy Mike Percy
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment