pyfarm.agent.utility module

Utilities

Top level utilities for the agent to use internally. Many of these are copied over from the master (which we can’t import here).

pyfarm.agent.utility.validate_environment(values)

Ensures that values is a dictionary and that it only contains string keys and values.

pyfarm.agent.utility.validate_uuid(value)

Ensures that value can be converted to or is a UUID object.

pyfarm.agent.utility.TASKS_SCHEMA(values)

pyfarm.agent.utility.default_json_encoder(obj, return_obj=False)

pyfarm.agent.utility.json_safe(source)

Recursively converts source into something that should be safe for json.dumps() to handle. This is used in conjunction with default_json_encoder() to also convert keys to something the json encoder can understand.

pyfarm.agent.utility.quote_url(source_url)

This function serves as a wrapper around urlsplit() and quote() and a url that has the path quoted.

pyfarm.agent.utility.dumps(*args, **kwargs)

Agent’s implementation of json.dumps() or pyfarm.master.utility.jsonify()

pyfarm.agent.utility.request_from_master(request)

Returns True if the request appears to be coming from the master

class pyfarm.agent.utility.UTF8Recoder(f, encoding)

Bases: object

Iterator that reads an encoded stream and reencodes the input to UTF-8

next()

class pyfarm.agent.utility.UnicodeCSVReader(f, dialect=<class csv.excel>, encoding='utf-8', **kwds)

Bases: object

A CSV reader which will iterate over lines in the CSV file “f”, which is encoded in the given encoding.

next()

class pyfarm.agent.utility.UnicodeCSVWriter(f, dialect=<class csv.excel>, **kwds)

Bases: object

A CSV writer which will write rows to CSV file “f”, which is encoded in the given encoding.

writerow(row)

writerows(rows)

pyfarm.agent.utility.total_seconds(td)

Returns the total number of seconds in the time delta object. This function is provided for backwards comparability with Python 2.6.

class pyfarm.agent.utility.AgentUUID

Bases: object

This class wraps all the functionality required to load, cache and retrieve an Agent’s UUID.

log = <pyfarm.agent.logger.python.Logger object>

classmethod load(path)

A classmethod to load a UUID object from a path. If the provided path does not exist or does not contain data which can be converted into a UUID object None will be returned.

classmethod save(agent_uuid, path)

Saves agent_uuid to path. This classmethod will also create the necessary parent directories and handle conversion from the input type uuid.UUID.

classmethod generate()

Generates a UUID object. This simply wraps uuid.uuid4() and logs a warning.

pyfarm.agent.utility.remove_file(path, retry_on_exit=False, raise_=True, ignored_errnos=(2, ))

Simple function to remove the provided file or retry on exit if requested. This function standardizes the log output, ensures it’s only called once per path on exit and handles platform specific exceptions (ie. WindowsError).

Parameters:

• retry_on_exit (bool) – If True, retry removal of the file when Python exists.
• raise (bool) – If True, raise an exceptions produced. This will always be False if remove_file() is being executed by atexit
• ignored_errnos (tuple) – A tuple of ignored error numbers. By default this function only ignores ENOENT.

pyfarm.agent.utility.remove_directory(path, retry_on_exit=False, raise_=True, ignored_errnos=(2, ))

Simple function to recursively remove the provided directory or retry on exit if requested. This function standardizes the log output, ensures it’s only called once per path on exit and handles platform specific exceptions (ie. WindowsError).

Parameters:

• retry_on_exit (bool) – If True, retry removal of the file when Python exists.
• raise (bool) – If True, raise an exceptions produced. This will always be False if remove_directory() is being executed by atexit
• ignored_errnos (tuple) – A tuple of ignored error numbers. By default this function only ignores ENOENT.

exception pyfarm.agent.utility.LockTimeoutError

Bases: exceptions.Exception

Raised if we timeout while attempting to acquire a deferred lock

class pyfarm.agent.utility.TimedDeferredLock

Bases: twisted.internet.defer.DeferredLock

A subclass of DeferredLock which has a timeout for the acquire() call.

acquire(timeout=None)

This method operates the same as DeferredLock.acquire() does except it requires a timeout argument.

Parameters:timeout (int) – The number of seconds to wait before timing out. Raises:LockTimeoutError – Raised if the timeout was reached before we could acquire the lock.