apache_beam.testing.test_utils module

Utility methods for testing

For internal use only; no backwards-compatibility guarantees.

class apache_beam.testing.test_utils.TempDir

Bases: object

Context Manager to create and clean-up a temporary directory.

get_path()

Returns the path to the temporary directory.

create_temp_file(suffix='', lines=None)

Creates a temporary file in the temporary directory.

Parameters:

• suffix (str) – The filename suffix of the temporary file (e.g. ‘.txt’)

• lines (List[str]) – A list of lines that will be written to the temporary file.

Returns:

The name of the temporary file created.

apache_beam.testing.test_utils.compute_hash(content, hashing_alg='sha1')

Compute a hash value of a list of objects by hashing their string representations.

apache_beam.testing.test_utils.patch_retry(testcase, module)

A function to patch retry module to use mock clock and logger.

Clock and logger that defined in retry decorator will be replaced in test in order to skip sleep phase when retry happens.

Parameters:

• testcase – An instance of unittest.TestCase that calls this function to patch retry module.

• module – The module that uses retry and need to be replaced with mock clock and logger in test.

apache_beam.testing.test_utils.delete_files(file_paths)

A function to clean up files or directories using FileSystems.

Glob is supported in file path and directories will be deleted recursively.

Parameters:

file_paths – A list of strings contains file paths or directories.

apache_beam.testing.test_utils.cleanup_subscriptions(sub_client, subs)

Cleanup PubSub subscriptions if exist.

apache_beam.testing.test_utils.cleanup_topics(pub_client, topics)

Cleanup PubSub topics if exist.

class apache_beam.testing.test_utils.PullResponseMessage(data, attributes=None, publish_time_secs=None, publish_time_nanos=None, ack_id=None)

Bases: object

Data representing a pull request response.

Utility class for create_pull_response.

apache_beam.testing.test_utils.create_pull_response(responses)

Create an instance of google.cloud.pubsub.types.ReceivedMessage.

Used to simulate the response from pubsub.SubscriberClient().pull().

Parameters:

responses – list of PullResponseMessage

Returns:

An instance of google.cloud.pubsub.types.PullResponse populated with responses.

apache_beam.testing.test_utils.create_file(path, contents)

Create a file to use as input to test pipelines

apache_beam.testing.test_utils.read_files_from_pattern(file_pattern)

Reads the files that match a pattern

class apache_beam.testing.test_utils.LCGenerator(a=25214903917, c=11, bits=48)

Bases: object

A pure Python implementation of linear congruential generator.

seed(seed)

Set the seed for the generator.

seed_jdk(seed)

Set the seed in the way equivalent to java.util.Random.seed.

This method provide a JDK equivalent random generator. Note that the JDK implementation has a known issue that is close seed generating close first pseudo-random number.

next_int()

Get the next integer value in the range of np.int32 ([0, 2^31-1]).

Equivalent to java.util.Random.nextInt.

next_uint()

Get the next unsigned integer value in the range of np.uint32.

Equivalent to java.util.Random.next(32).

rand_bytes(length)

Get random bytes of given length.

Equivalent to java.util.Random.nextBytes.

random_sample()

Get random sample between [0.0, 1.0), single precision float value.

Equivalent to java.util.Random.nextFloat.