apache_beam.utils.subprocess_server module

class apache_beam.utils.subprocess_server.SubprocessServer(stub_class, cmd, port=None)

Bases: object

An abstract base class for running GRPC Servers as an external process.

This class acts as a context which will start up a server, provides a stub to connect to it, and then shuts the server down. For example:

with SubprocessServer(GrpcStubClass, [executable, arg, ...]) as stub:
    stub.CallService(...)

Creates the server object.

Parameters:

• stub_class – the auto-generated GRPC client stub class used for connecting to the GRPC service

• cmd – command (including arguments) for starting up the server, suitable for passing to subprocess.POpen.

• port – (optional) the port at which the subprocess will serve its service. If not given, one will be randomly chosen and the special string “{{PORT}}” will be substituted in the command line arguments with the chosen port.

classmethod cache_subprocesses()

A context that ensures any subprocess created or used in its duration stay alive for at least the duration of this context.

These subprocesses may be shared with other contexts as well.

start()

start_process()

stop()

stop_process()

class apache_beam.utils.subprocess_server.JavaJarServer(stub_class, path_to_jar, java_arguments, classpath=None, cache_dir=None)

Bases: SubprocessServer

MAVEN_CENTRAL_REPOSITORY = 'https://repo.maven.apache.org/maven2'

BEAM_GROUP_ID = 'org.apache.beam'

JAR_CACHE = '/home/runner/.apache_beam/cache/jars'

start_process()

stop_process()

classmethod jar_name(artifact_id, version, classifier=None, appendix=None)

classmethod path_to_maven_jar(artifact_id, group_id, version, repository='https://repo.maven.apache.org/maven2', classifier=None, appendix=None)

classmethod path_to_beam_jar(gradle_target, appendix=None, version='2.61.0', artifact_id=None)

classmethod local_jar(url, cache_dir=None)

classmethod beam_services(replacements)

classmethod make_classpath_jar(main_jar, extra_jars, cache_dir=None)

apache_beam.utils.subprocess_server.is_service_endpoint(path)

Checks whether the path conforms to the ‘beam_services’ PipelineOption.

apache_beam.utils.subprocess_server.pick_port(*ports)

Returns a list of ports, same length as input ports list, but replaces all None or 0 ports with a random free port.