apache_beam.utils.urns module

For internal use only; no backwards-compatibility guarantees.

class apache_beam.utils.urns.RunnerApiFn

Bases: object

Abstract base class that provides urn registration utilities.

A class that inherits from this class will get a registration-based from_runner_api and to_runner_api method that convert to and from beam_runner_api_pb2.FunctionSpec.

Additionally, register_pickle_urn can be called from the body of a class to register serialization via pickling.

to_runner_api_parameter(unused_context: PipelineContext) → Tuple[str, Any]

Returns the urn and payload for this Fn.

The returned urn(s) should be registered with register_urn.

classmethod register_urn(urn: str, parameter_type: Type[T]) → Callable[[Callable[[T, 'PipelineContext'], Any]], Callable[[T, 'PipelineContext'], Any]]

classmethod register_urn(urn: str, parameter_type: None) → Callable[[Callable[[bytes, 'PipelineContext'], Any]], Callable[[bytes, 'PipelineContext'], Any]]

classmethod register_urn(urn: str, parameter_type: Type[T], fn: Callable[[T, 'PipelineContext'], Any]) → None

classmethod register_urn(urn: str, parameter_type: None, fn: Callable[[bytes, 'PipelineContext'], Any]) → None

Registers a urn with a constructor.

For example, if ‘beam:fn:foo’ had parameter type FooPayload, one could write RunnerApiFn.register_urn(‘bean:fn:foo’, FooPayload, foo_from_proto) where foo_from_proto took as arguments a FooPayload and a PipelineContext. This function can also be used as a decorator rather than passing the callable in as the final parameter.

A corresponding to_runner_api_parameter method would be expected that returns the tuple (‘beam:fn:foo’, FooPayload)

classmethod register_pickle_urn(pickle_urn)

Registers and implements the given urn via pickling.

to_runner_api(context: PipelineContext) → FunctionSpec

Returns an FunctionSpec encoding this Fn.

Prefer overriding self.to_runner_api_parameter.

classmethod from_runner_api(fn_proto: FunctionSpec, context: PipelineContext) → RunnerApiFnT

Converts from an FunctionSpec to a Fn object.

Prefer registering a urn with its parameter type and constructor.