Uploaded image for project: 'S2Graph'
  1. S2Graph
  2. S2GRAPH-10

Add create table if not exist on TransferHFile loader

Agile BoardAttach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: To Do
    • Trivial
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      for bulk load data into s2graph, loader project provide spark job to transfer TSV graph data into HFile.
      current implementation assumes that hbase table where this HFile will be loaded exist, but it would be convenient if TransferHFile check if given hbase table exist and create if it is not exist. creating hbase table with pre-split with byte ranges that s2graph use is not easily done on script level so it would be helpful to add create given hbase table if not exist with pre-split size on TransferHFile job.

      s2core project has Management.createTable which will create hbase table if table not exist so I think simply use Management.createTable would do necessary work for us on TransferHFile.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            steamshon Do Yung Yoon

            Dates

              Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 168h
              168h
              Remaining:
              Remaining Estimate - 168h
              168h
              Logged:
              Time Spent - Not Specified
              Not Specified

              Slack

                Issue deployment