Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8425

Notifying Interactive Beam user about Beam related cell deletion or re-execution

Details

    Description

      There is a general problem about Interactive Notebooks that when an end user deletes a cell that has been executed or re-executes a cell, those previous executions are hidden from the end user.

      However, hidden states will still have side effects in the notebook.

      This kind of problem bothers Beam even more because Beam's pipeline construction statements are note idempotent and pipeline execution is decoupled and deferred from pipeline construction.

      Re-executing a cell with Beam statements that build a pipeline would cause unexpected pipeline state and the user wouldn't notice it due to the problem of notebooks.

      We'll intercept each transform application invocation from the InteractiveRunner and record the ipython/notebook prompt number. Then each time a user executes a cell that applies PTransform, we'll compare the recorded list of prompt numbers with current notebook file's content and figure out if there is any missing number. If so, we know for sure that a re-execution happens and use display manager to notify the end user of potential side effects caused by hidden states of the notebook/ipython.

      Attachments

        Issue Links

          Activity

            People

              ningk Ning
              ningk Ning
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: