Beam SQL extensions: SET and RESET Pipeline Options

Beam SQL’s SET and RESET statements allow the user to configure Pipeline Options via the SQL shell. These are the same Pipeline Options passed to other Beam applications on the command line in the --<option>=<value> format.

Syntax

SET option = value

The SET command sets a Pipeline Option.

• option: The case sensitive name of the Pipeline Option, specified as an Identifier.
• value: The case sensitive value of the Pipeline Option, specified as an Identifier. For flag options that have no value on the command line, use true.

RESET option

The RESET command resets a Pipeline Option to its default value.

• option: The case sensitive name of the Pipeline Option, specified as an Identifier.

Common Options

• SET project = `my_gcp_project` : Sets the default GCP project tomy_gcp_project.
• SET runner = DataflowRunner: Sets the pipeline to run on Dataflow.