apache_beam.io.gcp.bigquery_schema_tools module

Tools used tool work with Schema types in the context of BigQuery. Classes, constants and functions in this file are experimental and have no backwards compatibility guarantees. NOTHING IN THIS FILE HAS BACKWARDS COMPATIBILITY GUARANTEES.

apache_beam.io.gcp.bigquery_schema_tools.generate_user_type_from_bq_schema(the_table_schema, selected_fields: TableSchema | None = None) → type[source]

Convert a schema of type TableSchema into a pcollection element. :param the_table_schema: A BQ schema of type TableSchema :param selected_fields: if not None, the subset of fields to consider

Returns:: type that can be used to work with pCollections.
Return type:: type

apache_beam.io.gcp.bigquery_schema_tools.bq_field_to_type(field, mode)[source]

apache_beam.io.gcp.bigquery_schema_tools.convert_to_usertype(table_schema, selected_fields=None)[source]

class apache_beam.io.gcp.bigquery_schema_tools.BeamSchemaConversionDoFn(pcoll_val_ctor)[source]

Bases: DoFn

process(dict_of_tuples)[source]

infer_output_type(input_type)[source]