apache_beam.yaml.yaml_enrichment module

apache_beam.yaml.yaml_enrichment.enrichment_transform(enrichment_handler: str, handler_config: Dict[str, Any], timeout: float | None = 30)[source]

The Enrichment transform allows you to dynamically enhance elements in a pipeline by performing key-value lookups against external services like APIs or databases.

Example Usage:

- type: Enrichment
    config:
        enrichment_handler: 'BigTable'
        handler_config:
            project_id: 'apache-beam-testing'
            instance_id: 'beam-test'
            table_id: 'bigtable-enrichment-test'
            row_key: 'product_id'
      timeout: 30
Parameters:

  • enrichment_handler – Specifies the source from where data needs to be extracted into the pipeline for enriching data. It can be a string value in [“BigQuery”, “BigTable”, “FeastFeatureStore”, “VertexAIFeatureStore”].

  • handler_config –

    Specifies the parameters for the respective enrichment_handler in a dictionary format. To see the full set of handler_config parameters, see their corresponding doc pages: