PyFarm - A Python Based Distributed Job System

Attention

These documents are still undergoing revision

PyFarm is a Python based distributed job system which is intended to be easy to deploy and maintain. Initially developed for individual use new revisions have been engineered with larger deployments in mind. It’s a system that’s focused on providing a framework for customized command execution while taking into account resource management.

Since this project is under active development, if you have specific questions or comments you’re always welcome to post to the google groups discussion list or via email to pyfarm@googlegroups.com.

Components

PyFarm has several distinct roles that it must fill in order to accomplish what it was designed to do. Because of this the project has been broken down into three components to help manage scope and limit interdependencies as much as possible.

Core

Build Status | Code Coverage | Documentation | Source Code

This is the base library which provides a few modules and objects which the other two components of PyFarm use. The primary purpose of this component is to centralize code basic functionality such as logging and configuration.

Master

Build Status | Code Coverage | Documentation | Source Code

This is the component responsible for storing jobs and tasks to run as well as allocation of work to remote hosts. This component is the largest of the three components and contains the code necessary to run the web interface, interact with the relational database, REST APIs and scheduler.

Indices and tables