Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-13439

Correct type signature on ptransform_fn to better support typecheckers

Details

    Description

      Because the return type of the ptransform_fn is not specified, certain typecheckers will assume the type signature of functions decorated with it remain unchanged. However, ptransform_fn specifically adjusts the type signature such that the function no longer takes a pcoll.

      This can be corrected by changing the the definition of ptransform_fn from:

      def ptransform_fn(fn):
      

      to:

      def ptransform_fn(fn) -> Callable[..., _PTransformFnPTransform]: 

      This allows the typechecker to understand the return type of a function decorated with ptransform_fn will instead be a _PTransformFnPTransform type.

      Additionally, for Python 3.10, ParamSpec could be used to preserve the signature of the internal function as well.

      from typing import ParamSpec, TypeVar, Concatenate
      
      P = ParamSpec("P")
      T = TypeVar("T")
      R = TypeVar("R")
      
      def ptransform_fn(fn: Callable[Concatenate[PCollection[T], P], R]) -> Callable[[P], _PTransformFnPTransform]: ...

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Sandy Chapman Sandy Chapman
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 10m
                  4h 10m