pyfarm.models.job module

Job Models

Models and interface classes related to jobs.

class pyfarm.models.job.Job(**kwargs)

Bases: flask_sqlalchemy.Model, pyfarm.models.core.mixins.ValidatePriorityMixin, pyfarm.models.core.mixins.ValidateWorkStateMixin, pyfarm.models.core.mixins.WorkStateChangedMixin, pyfarm.models.core.mixins.ReprMixin, pyfarm.models.core.mixins.UtilityMixins

Defines the attributes and environment for a job. Individual commands are kept track of by Task

REPR_COLUMNS = ('id', 'state', 'project')

REPR_CONVERT_COLUMN = {'state': <built-in function repr>}

STATE_ENUM = ['done', 'paused', 'running', 'failed', None]

alter_frame_range(start, end, by)

autodelete_time

If not None, this job will be automatically deleted this number of seconds after it finishes.

batch

Number of tasks to run on a single agent at once. Depending on the capabilities of the software being run this will either cause a single process to execute on the agent or multiple processes one after the other.

by

The number of frames to count by between start and end. This column may also sometimes be referred to as ‘step’ by other software.

can_use_more_agents()

children

clear_assigned_counts()

completion_notify_sent

Whether or not the finish notification mail has already been sent out.

cpus

Number of cpus or threads each task should consume oneach agent. Depending on the job type being executed this may result in additional cpu consumption, longer wait times in the queue (2 cpus means 2 ‘fewer’ cpus on an agent), or all of the above... csv-table:: Special Values :header: Value, Result :widths: 10, 50 0, minimum number of cpu resources not required -1, agent cpu is exclusive for a task from this job

data

Json blob containing additional data for a job .. note:: Changes made directly to this object are not applied to the session.

environ

Dictionary containing information about the environment in which the job will execute. .. note:: Changes made directly to this object are not applied to the session.

get_batch(agent)

group

The job group this job belongs to

hidden

If True, keep the job hidden from the queue and web ui. This is typically set to True if you either want to save a job for later viewing or if the jobs data is being populated in a deferred manner.

id

Provides an id for the current row. This value should never be directly relied upon and it’s intended for use by relationships.

job_group_id

The foreign key which stores:class:JobGroup.id

job_queue_id

The foreign key which stores JobQueue.id

jobtype_version

jobtype_version_id

The foreign key which stores JobTypeVersion.id

maximum_agents

The scheduler will never assign more than this numberof agents to this job.

minimum_agents

The scheduler will try to assign at least this number of agents to this job as long as it can use them, before any other considerations.

notes

Notes that are provided on submission or added after the fact. This column is only provided for human consumption, is not scanned, indexed, or used when searching

notified_users

num_assigned_agents()

num_tiles

How many regions to split frames into for rendering.

output_link

An optional link to a URI where this job’s output can be viewed.

parents

paused()

priority

The priority of the job relative to others in the queue. This is not the same as task priority.

configured by: job.priority

queue

The queue for this job

ram

Amount of ram a task from this job will require to be free in order to run. A task exceeding this value will not result in any special behavior... csv-table:: Special Values :header: Value, Result :widths: 10, 500, minimum amount of free ram not required-1, agent ram is exclusive for a task from this job

ram_max

Maximum amount of ram a task is allowed to consume on an agent... warning:: If set, the task will be terminated if the ram in use by the process exceeds this value.

ram_warning

Amount of ram used by a task before a warning raised. A task exceeding this value will not cause any work stopping behavior.

requeue

Number of times to requeue failed tasks .. csv-table:: Special Values :header: Value, Result :widths: 10, 50 0, never requeue failed tasks -1, requeue failed tasks indefinitely

rerun()

Makes this job rerun all its task. Tasks that are currently running are left untouched.

rerun_failed()

Makes this job rerun all its failed tasks. Tasks that are done or are currently running are left untouched

software_requirements

state

The state of the job with a value provided by WorkState

tag_requirements

tags

Relationship between this job and Tag objects

tasks

tasks_done

Relationship between this job and any Task objects which are done.

tasks_failed

Relationship between this job and any Task objects which have failed.

tasks_queued

Relationship between this job and any Task objects which are queued.

tasks_running

Relationship between this job and any Task objects which are running.

time_finished

Time the job was finished. This will be set when the last task finishes and reset if a job is requeued.

time_started

The time this job was started. By default this value is set when state is changed to an appropriate value or when a job is requeued.

time_submitted

The time the job was submitted. By default this defaults to using datetime.datetime.utcnow() as the source of submission time. This value will not be set more than once and will not change even after a job is requeued.

title

The title of this job

to_be_deleted

If true, the master will stop all running tasks for this job and then delete it.

update_state()

user

The owner of this job

user_id

The id of the user who owns this job

validate_progress(key, value)

validate_resource(key, value)

Validation that ensures that the value provided for either ram or cpus is a valid value with a given range

weight

The weight of this job. The scheduler will distribute available agents between jobs and job queues in the same queue in proportion to their weights.