Partition

Javadoc

Separates elements in a collection into multiple output collections. The partitioning function contains the logic that determines how to separate the elements of the input collection into each resulting partition output collection.

The number of partitions must be determined at graph construction time. You cannot determine the number of partitions in mid-pipeline.

See more information in the Beam Programming Guide.

Examples

Example: dividing a PCollection into percentile groups

Related transforms

• Filter is useful if the function is just deciding whether to output an element or not.
• ParDo is the most general element-wise mapping operation, and includes other abilities such as multiple output collections and side-inputs.
• CoGroupByKey performs a per-key equijoin.