Enrichment transform
|
The enrichment transform lets you dynamically enrich data in a pipeline by doing a key-value lookup to a remote service. The transform uses RequestResponeIO internally. This feature uses client-side throttling to ensure that the remote service isn’t overloaded with requests. If service-side errors occur, like TooManyRequests and Timeout exceptions, it retries the requests by using exponential backoff.
This transform is available in Apache Beam 2.54.0 and later versions.
Examples
The following examples demonstrate how to create a pipeline that use the enrichment transform to enrich data from external services.
| Service | Example |
|---|---|
| Cloud Bigtable | Enrichment with Bigtable |
| Vertex AI Feature Store | Enrichment with Vertex AI Feature Store |
| Vertex AI Feature Store (Legacy) | Enrichment with Legacy Vertex AI Feature Store |
Related transforms
Not applicable.
Pydoc |
Last updated on 2024/11/14
Have you found everything you were looking for?
Was it all useful and clear? Is there anything that you would like to change? Let us know!