apache_beam.io.gcp.datastore.v1new.util module

class apache_beam.io.gcp.datastore.v1new.util.MovingSum(window_ms, bucket_ms)[source]

Bases: object

Class that keeps track of a rolling window sum.

For use in tracking recent performance of the connector.

Intended to be similar to org.apache.beam.sdk.util.MovingFunction(…, Sum.ofLongs()), but for convenience we expose the count of entries as well so this doubles as a moving average tracker.

sum(now)[source]
add(now, inc)[source]
count(now)[source]
has_data(now)[source]
class apache_beam.io.gcp.datastore.v1new.util.DynamicBatchSizer[source]

Bases: object

Determines request sizes for future Datastore RPCs.

get_batch_size(now)[source]

Returns the recommended size for datastore RPCs at this time.

report_latency(now, latency_ms, num_mutations)[source]

Report the latency of a Datastore RPC.

Parameters:

  • now – double, completion time of the RPC as seconds since the epoch.

  • latency_ms – double, the observed latency in milliseconds for this RPC.

  • num_mutations – int, number of mutations contained in the RPC.