pyfarm.master.api.jobs module¶

Jobs¶

This module defines an API for managing and querying jobs

class pyfarm.master.api.jobs.JobIndexAPI[source]¶

Bases: flask.views.MethodView

get()[source]¶

A GET to this endpoint will return a list of all jobs.

GET /api/v1/jobs/ HTTP/1.1¶

Request

GET /api/v1/jobs/ HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "title": "Test Job",
        "state": "queued",
        "id": 1
    },
    {
        "title": "Test Job 2",
        "state": "queued",
        "id": 2
    }
]

Statuscode 200:	no error

methods = ['GET', 'POST']¶

post()[source]¶

A POST to this endpoint will submit a new job.

POST /api/v1/jobs/ HTTP/1.1¶

Request

POST /api/v1/jobs/ HTTP/1.1
Accept: application/json

{
    "end": 2.0,
    "title": "Test Job 2",
    "jobtype": "TestJobType",
    "data": {
        "foo": "bar"
    },
    "software_requirements": [
        {
        "software": "blender"
        }
    ],
    "start": 1.0
}

Response

HTTP/1.1 201 CREATED
Content-Type: application/json

{
    "time_finished": null,
    "time_started": null,
    "end": 2.0,
    "time_submitted": "2014-03-06T15:40:58.335259",
    "jobtype_version": 1,
    "jobtype": "TestJobType",
    "jobqueue": None
    "start": 1.0,
    "priority": 0,
    "state": "queued",
    "parents": [],
    "hidden": false,
    "project_id": null,
    "ram_warning": null,
    "title": "Test Job 2",
    "tags": [],
    "user": null,
    "by": 1.0,
    "data": {
        "foo": "bar"
    },
    "ram_max": null,
    "notes": "",
    "batch": 1,
    "project": null,
    "environ": null,
    "requeue": 3,
    "software_requirements": [
        {
            "min_version": null,
            "max_version": null,
            "max_version_id": null,
            "software_id": 1,
            "min_version_id": null,
            "software": "blender"
        }
    ],
    "tag_requirements": [
        {
            "tag": "workstation",
            "negate": true
        }
    ],
    "id": 2,
    "ram": 32,
    "cpus": 1,
    "children": []
}

Statuscode 201:	a new job item was created
Statuscode 400:	there was something wrong with the request (such as invalid columns being included)
Statuscode 404:	a referenced object, like a software or software version, does not exist
Statuscode 409:	a conflicting job already exists

class pyfarm.master.api.jobs.JobNotifiedUsersIndexAPI[source]¶

Bases: flask.views.MethodView

get(job_name)[source]¶

A GET to this endpoint will return a list of all users to be notified on events in this job.

GET /api/v1/jobs/[<str:name>|<int:id>]/notified_users/ HTTP/1.1¶

Request

GET /api/v1/jobs/Test%20Job%202/notified_users/ HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "id": 1,
        "username": "testuser",
        "email": "testuser@localhost"
    }
]

Statuscode 200:	no error
Statuscode 404:	job not found

methods = ['GET', 'POST']¶

post(job_name)[source]¶

A POST to this endpoint will add the specified user to the list of notified users for this job.

POST /api/v1/jobs/[<str:name>|<int:id>]/notified_users/ HTTP/1.1¶

Request

POST /api/v1/jobs/Test%20Job/notified_users/ HTTP/1.1
Accept: application/json

{
    "username": "testuser"
    "on_success": true,
    "on_failure": true,
    "on_deletion": false
}

Response

HTTP/1.1 201 CREATED
Content-Type: application/json

{
    "id": 1
    "username": "testuser"
    "email": "testuser@example.com"
}

Statuscode 201:	a new notified user entry was created
Statuscode 400:	there was something wrong with the request (such as invalid columns being included)
Statuscode 404:	the job or the specified user does not exist

class pyfarm.master.api.jobs.JobSingleNotifiedUserAPI[source]¶

Bases: flask.views.MethodView

delete(job_name, username)[source]¶

A DELETE to this endpoint will remove the specified user from the list of notified users for this job.

DELETE /api/v1/jobs/[<str:name>|<int:id>]/notified_users/<str:username> HTTP/1.1¶

Request

DELETE /api/v1/jobs/Test%20Job/notified_users/testuser HTTP/1.1
Accept: application/json

Response

HTTP/1.1 204 NO_CONTENT

Statuscode 204:	the notified user was removed from this job or wasn’t in the list in the first place
Statuscode 404:	the job or the specified user does not exist

methods = ['DELETE']¶

class pyfarm.master.api.jobs.JobSingleTaskAPI[source]¶

Bases: flask.views.MethodView

get(job_name, task_id)[source]¶

A GET to this endpoint will return the requested task

GET /api/v1/jobs/[<str:name>|<int:id>]/tasks/<int:task_id> HTTP/1.1¶

Request

GET /api/v1/jobs/Test%20Job%202/tasks/1 HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "time_finished": null,
    "agent": null,
    "attempts": 0,
    "frame": 2.0,
    "agent_id": null,
    "job": {
        "id": 1,
        "title": "Test Job"
    },
    "time_started": null,
    "state": "running",
    "project_id": null,
    "id": 2,
    "time_submitted": "2014-03-06T15:40:58.338904",
    "project": null,
    "parents": [],
    "job_id": 1,
    "hidden": false,
    "children": [],
    "priority": 0
}

Statuscode 200:	no error

methods = ['GET', 'POST']¶

post(job_name, task_id)[source]¶

A POST to this endpoint will update the specified task with the data in the request. Columns not specified in the request will be left as they are. The agent will use this endpoint to inform the master of its progress.

POST /api/v1/jobs/[<str:name>|<int:id>]/tasks/<int:task_id> HTTP/1.1¶

Request

PUT /api/v1/job/Test%20Job/tasks/1 HTTP/1.1
Accept: application/json

{
    "state": "running"
}

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "time_finished": null,
    "agent": null,
    "attempts": 0,
    "failures": 0,
    "frame": 2.0,
    "agent_id": null,
    "job": {
        "id": 1,
        "title": "Test Job"
    },
    "time_started": null,
    "state": "running",
    "project_id": null,
    "id": 2,
    "time_submitted": "2014-03-06T15:40:58.338904",
    "project": null,
    "parents": [],
    "job_id": 1,
    "hidden": false,
    "children": [],
    "priority": 0
}

Statuscode 200:	the task was updated
Statuscode 400:	there was something wrong with the request (such as invalid columns being included)

class pyfarm.master.api.jobs.JobTasksIndexAPI[source]¶

Bases: flask.views.MethodView

get(job_name)[source]¶

A GET to this endpoint will return a list of all tasks in a job.

GET /api/v1/jobs/[<str:name>|<int:id>]/tasks HTTP/1.1¶

Request

GET /api/v1/jobs/Test%20Job%202/tasks/ HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "hidden": false,
        "id": 3,
        "attempts": 0,
        "priority": 0,
        "time_started": null,
        "time_submitted": "2014-03-06T15:49:51.892228",
        "frame": 1.0,
        "time_finished": null,
        "job_id": 2,
        "project_id": null,
        "state": "queued",
        "agent_id": null
    },
    {
        "hidden": false,
        "id": 4,
        "attempts": 0,
        "priority": 0,
        "time_started": null,
        "time_submitted": "2014-03-06T15:49:51.892925",
        "frame": 2.0,
        "time_finished": null,
        "job_id": 2,
        "project_id": null,
        "state": "queued",
        "agent_id": null
    }
]

Statuscode 200:	no error

methods = ['GET']¶

exception pyfarm.master.api.jobs.ObjectNotFound[source]¶: Bases: Exception

class pyfarm.master.api.jobs.SingleJobAPI[source]¶

Bases: flask.views.MethodView

delete(job_name)[source]¶

A DELETE to this endpoint will mark the specified job for deletion and remove it after stopping and removing all of its tasks.

DELETE /api/v1/jobs/[<str:name>|<int:id>] HTTP/1.1¶

Request

DELETE /api/v1/jobs/1 HTTP/1.1
Accept: application/json

Response

HTTP/1.1 204 NO_CONTENT

Statuscode 204:	the specified job was marked for deletion
Statuscode 404:	the job does not exist

get(job_name)[source]¶

A GET to this endpoint will return the specified job, by name or id.

GET /api/v1/jobs/[<str:name>|<int:id>] HTTP/1.1¶

Request

GET /api/v1/jobs/Test%20Job%202 HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "ram_warning": null,
    "title": "Test Job",
    "state": "queued",
    "jobtype_version": 1,
    "jobtype": "TestJobType",
    "environ": null,
    "user": null,
    "priority": 0,
    "time_finished": null,
    "start": 2.0,
    "id": 1,
    "notes": "",
    "notified_users": []
    "ram": 32,
    "tags": [],
    "hidden": false,
    "data": {
        "foo": "bar"
    },
    "software_requirements": [
        {
            "software": "blender",
            "software_id": 1,
            "min_version": null,
            "max_version": null,
            "min_version_id": null,
            "max_version_id": null
        }
    ],
    "tag_requirements": [
        {
            "tag": "workstation",
            "negate": true
        }
    ],
    "batch": 1,
    "time_started": null,
    "time_submitted": "2014-03-06T15:40:58.335259",
    "requeue": 3,
    "end": 4.0,
    "parents": [],
    "cpus": 1,
    "ram_max": null,
    "children": [],
    "by": 1.0,
    "project_id": null
}

Statuscode 200:	no error
Statuscode 404:	job not found

methods = ['DELETE', 'GET', 'POST']¶

post(job_name)[source]¶

A POST to this endpoint will update the specified job with the data in the request. Columns not specified in the request will be left as they are. If the “start”, “end” or “by” columns are updated, tasks will be created or deleted as required.

POST /api/v1/jobs/[<str:name>|<int:id>] HTTP/1.1¶

Request

PUT /api/v1/jobs/Test%20Job HTTP/1.1
Accept: application/json

{
    "start": 2.0
}

Response

HTTP/1.1 201 OK
Content-Type: application/json

{
    "end": 4.0,
    "children": [],
    "jobtype_version": 1,
    "jobtype": "TestJobType",
    "time_started": null,
    "tasks_failed": [],
    "project_id": null,
    "id": 1,
    "software_requirements": [
        {
            "software": "blender",
            "min_version": null,
            "max_version_id": null,
            "software_id": 1,
            "max_version": null,
            "min_version_id": null
        }
    ],
    "tags": [],
    "environ": null,
    "requeue": 3,
    "start": 2.0,
    "ram_warning": null,
    "title": "Test Job",
    "batch": 1,
    "time_submitted": "2014-03-06T15:40:58.335259",
    "ram_max": null,
    "user": null,
    "notes": "",
    "data": {
        "foo": "bar"
    },
    "tag_requirements": [
        {
            "tag": "workstation",
            "negate": true
        }
    ],
    "ram": 32,
    "parents": [],
    "hidden": false,
    "priority": 0,
    "cpus": 1,
    "state": "queued",
    "by": 1.0,
    "time_finished": null
}

Statuscode 200:	the job was updated
Statuscode 400:	there was something wrong with the request (such as invalid columns being included)

class pyfarm.master.api.jobs.SingleTaskOnAgentFailureAPI[source]¶

Bases: flask.views.MethodView

delete(job_id, task_id, agent_id)[source]¶

A DELETE to this endpoint will remove the specified agent from the list of agents that failed to execute this task

DELETE /api/v1/jobs/<int:id>/tasks/<int:task_id>/failed_on_agents/<str:agent_id> HTTP/1.1¶

Request

DELETE /api/v1/jobs/12/tasks/1234/failed_on_agents/02f08241-c556-4355-9e5e-33243d8c4577 HTTP/1.1

Response

HTTP/1.1 204 NO_CONTENT

Statuscode 204:	the given agent was removed from the list of agents that failed to execute this task
Statuscode 404:	the job, task or agent does not exist

methods = ['DELETE']¶

class pyfarm.master.api.jobs.TaskFailedOnAgentsIndexAPI[source]¶

Bases: flask.views.MethodView

get(job_id, task_id)[source]¶

A GET to this endpoint will return a list of all agents that failed to execute this task

GET /api/v1/jobs/<int:job_id>/tasks/<int:task_id>/failed_on_agents/ HTTP/1.1¶

Request

GET /api/v1/jobs/12/tasks/1234/failed_on_agents/ HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

[
    {
        "id": "02f08241-c556-4355-9e5e-33243d8c4577",
        "hostname": "agent1"
    }
]

Statuscode 200:	no error
Statuscode 404:	job or task not found

methods = ['GET', 'POST']¶

post(job_id, task_id)[source]¶

A POST to this endpoint will add the specified agent to the list of agents that failed to execute this task

POST /api/v1/jobs/<int:id>/tasks/<int:task_id>/failed_on_agents/ HTTP/1.1¶

Request

POST /api/v1/jobs/12/tasks/1234/failed_on_agents/ HTTP/1.1
Accept: application/json
Content-Type: application/json

{
    "id": "02f08241-c556-4355-9e5e-33243d8c4577"
}

Response

HTTP/1.1 201 CREATED
Content-Type: application/json

{
    "id": "02f08241-c556-4355-9e5e-33243d8c4577",
    "hostname": "agent1"
}

Statuscode 201:	a new entry was created
Statuscode 400:	there was something wrong with the request (such as invalid columns being included)
Statuscode 404:	the job, task or agent specified does not exist

pyfarm.master.api.jobs.parse_requirements(requirements)[source]¶

Takes a list dicts specifying a software and optional min- and max-versions and returns a list of JobRequirement objects.

Raises TypeError if the input was not as expected or ObjectNotFound if a referenced software or version was not found.

Parameters:	requirements (list) – A list of of dicts specifying a software and optionally min_version and/or max_version.
Raises:	TypeError – Raised if `requirements` is not a list or if an entry in `requirements` is not a dictionary. ValueError – Raised if there’s a problem with the content of at least one of the requirement dictionaries. ObjectNotFound – Raised if the referenced software or version was not found

pyfarm.master.api.jobs.schema()[source]¶

Returns the basic schema of Job

GET /api/v1/jobs/schema HTTP/1.1¶

Request

GET /api/v1/jobs/schema HTTP/1.1
Accept: application/json

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
    "batch": "INTEGER",
    "by": "NUMERIC(10, 4)",
    "cpus": "INTEGER",
    "data": "JSONDict",
    "end": "NUMERIC(10,4)",
    "environ": "JSONDict",
    "hidden": "BOOLEAN",
    "id": "INTEGER",
    "jobtype": "VARCHAR(64)",
    "jobtype_version": "INTEGER",
    "jobqueue": "VARCHAR(255)",
    "notes": "TEXT",
    "priority": "INTEGER",
    "project_id": "INTEGER",
    "ram": "INTEGER",
    "ram_max": "INTEGER",
    "ram_warning": "INTEGER",
    "requeue": "INTEGER",
    "start": "NUMERIC(10,4)",
    "state": "WorkStateEnum",
    "time_finished": "DATETIME",
    "time_started": "DATETIME",
    "time_submitted": "DATETIME",
    "title": "VARCHAR(255)",
    "user": "VARCHAR(255)"
}

Statuscode 200:	no error