Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14336

Procedure V2 Phase 1 - Procedure Framework and Making DDL Operations fault tolerant

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Major
    • Resolution: Done
    • 1.1.0, 1.2.0, 1.3.0, 2.0.0
    • None
    • proc-v2

    Description

      This is the first phase of Procedure V2 (HBASE-12439)

      • Core framework
      • re-implement Namespace/Table/Column DDLs to multi-steps procedure with a rollback/rollforward ability in case of failure

      Attachments

        Issue Links

          Is contained by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-12439 Procedure V2

          • Major - Major loss of function.
          • Closed
          1.
          		 Procedure v2 - core framework Sub-task Closed Matteo Bertozzi  
          2.
          		 Procedure v2 - master create/delete table Sub-task Closed Matteo Bertozzi  
          3.
          		 Procedure v2 - client create/delete table sync Sub-task Closed Matteo Bertozzi  
          4.
          		 Procedure V2 - master Add/Modify/Delete Column Family Sub-task Closed Stephen Yuan Jiang

          100%
          Original Estimate - 168h Original Estimate - 168h
          Time Spent - 384h
          5.
          		 Procedure V2 - master Modify table Sub-task Closed Stephen Yuan Jiang

          100%
          Original Estimate - 72h Original Estimate - 72h
          Time Spent - 168h
          6.
          		 Procedure V2 - master Enable/Disable table Sub-task Closed Stephen Yuan Jiang

          100%
          Original Estimate - 120h Original Estimate - 120h
          Time Spent - 216h
          7.
          		 Procedure V2 - master Create/Modify/Delete namespace Sub-task Closed Stephen Yuan Jiang

          100%
          Original Estimate - 168h
          Time Spent - 120h Time Not Required
          8.
          		 Procedure v2 - client enable/disable table sync Sub-task Closed Stephen Yuan Jiang

          100%
          Original Estimate - 120h
          Time Spent - 72h Time Not Required
          9.
          		 Procedure V2 - Use nonces for double submits from client Sub-task Closed Stephen Yuan Jiang  
          10.
          		 [branch-1.1] Procedure V2 - Make procedure v2 configurable in branch-1.1 Sub-task Closed Stephen Yuan Jiang  
          11.
          		 Procedure V2 - master truncate table Sub-task Closed Matteo Bertozzi  
          12.
          		 High level Integration test for master DDL operations Sub-task Closed Sophia Feng  
          13.
          		 Procedure v2 - WAL Improvements Sub-task Closed Matteo Bertozzi  
          14.
          		 Procedure V2 - Procedure classes should not be InterfaceAudience.Public Sub-task Closed Enis Soztutar  
          15.
          		 Procedure v2 - Add maxProcId to the wal header Sub-task Closed Matteo Bertozzi  
          16.
          		 Improve procedure yielding Sub-task Closed Matteo Bertozzi  
          17.
          		 Procedure V2: master fail to start due to WALProcedureStore sync failures when HDFS data nodes count is low Sub-task Closed Matteo Bertozzi  
          18.
          		 Add a NoopProcedureStore for testing Sub-task Closed Matteo Bertozzi  
          19.
          		 WALProcedureStore fencing is not effective if new WAL rolls Sub-task Closed Enis Soztutar  
          20.
          		 Procedure v2 - MasterProcedureQueue fix concurrency issue on table queue deletion Sub-task Closed Matteo Bertozzi  
          21.
          		 Undo workarounds in IntegrationTestDDLMasterFailover for client double submit Sub-task Closed Stephen Yuan Jiang  
          22.
          		 Procedure V2 - Administrative Task: Provide an API to List all procedures Sub-task Closed Stephen Yuan Jiang  
          23.
          		 Procedure V2 - Administrative Task: provide an API to abort a procedure Sub-task Closed Stephen Yuan Jiang  
          24.
          		 Add IT test for procedure-v2-based namespace DDL Sub-task Closed Stephen Yuan Jiang  
          25.
          		 Move ServerShutdownHandler to Pv2 Sub-task Closed Michael Stack  
          26.
          		 Procedure V2 - shell command to list all procedures Sub-task Closed Stephen Yuan Jiang  
          27.
          		 Procedure V2 - shell command to abort a procedure Sub-task Closed Stephen Yuan Jiang  
          28.
          		 Procedure V2 - enforce ACL on procedure admin tasks Sub-task Closed Stephen Yuan Jiang  
          29.
          		 Add test to verify DDL operations from admin when the master is restarted Sub-task Closed Unassigned  
          30.
          		 Procedure v2 - Avoid double yield Sub-task Closed Matteo Bertozzi  
          31.
          		 Procedure V2 - Remove synchronized around nonce in Procedure submit Sub-task Closed Matteo Bertozzi  
          32.
          		 Move TestTableDeleteFamilyHandler and TestTableDescriptorModification handler tests to procedure Sub-task Closed Matteo Bertozzi  
          33.
          		 Split TestWALProcedureStore Sub-task Closed Matteo Bertozzi  
          34.
          		 Fix ProcedureTestingUtility#LoadCounter.setMaxProcId() Sub-task Closed Matteo Bertozzi  
          35.
          		 Procedure v2 - fix master crash for FileNotFound Sub-task Closed Matteo Bertozzi  
          36.
          		 Procedure v2 - use consts for conf properties in tests Sub-task Closed Matteo Bertozzi  
          37.
          		 Procedure v2 - complete child procedure support Sub-task Closed Matteo Bertozzi  
          38.
          		 Procedure v2 - Make ProcedureException extend HBaseException Sub-task Closed Matteo Bertozzi  
          39.
          		 Procedure v2 - Test WAL protobuf entry size limit Sub-task Closed Matteo Bertozzi  
          40.
          		 Procedure v2 - Cache system user and avoid IOException Sub-task Closed Matteo Bertozzi  
          41.
          		 Procedure v2 - Extract chore from the executor Sub-task Closed Matteo Bertozzi  
          42.
          		 Procedure-V2: Enforce procedure ownership at submission Sub-task Closed Ted Yu  
          43.
          		 Procedure v2 - Cleanup sched wait/lock semantic Sub-task Closed Matteo Bertozzi  
          44.
          		 Procedure v2 - Cleanup suspended proc execution Sub-task Closed Matteo Bertozzi  
          45.
          		 Procedure v2 - Fix yield while holding locks Sub-task Closed Matteo Bertozzi  
          46.
          		 Procedure V2 - Fix nonce submission to avoid unnecessary calling coprocessor multiple times Sub-task Closed Matteo Bertozzi  

          Activity

            People

              syuanjiang Stephen Yuan Jiang
              syuanjiang Stephen Yuan Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 648h Original Estimate - 648h
                  648h
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 960h
                  960h