Commands¶
Note
The default values provided are based on the configuration at the time this page was generated. They may not be the same defaults you will see.
Standard Commands¶
pyfarm-agent¶
usage: pyfarm-agent [status|start|stop]
positional arguments:
{start,stop,status} individual operations pyfarm-agent can run
start starts the agent
stop stops the agent
status query the 'running' state of the agent
optional arguments:
-h, --help show this help message and exit
Agent Network Service:
Main flags which control the network services running on the agent.
--port PORT The port number which the agent is either running on
or will run on when started. This port is also
reported the master when an agent starts. [default:
None]
--host HOST The host to communicate with or hostname to present to
the master when starting. Defaults to the fully
qualified hostname.
--agent-api-username AGENT_API_USERNAME
The username required to access or manipulate the
agent using REST. [default: agent]
--agent-api-password AGENT_API_PASSWORD
The password required to access manipulate the agent
using REST. [default: agent]
--agent-id AGENT_ID The UUID used to identify this agent to the master. By
default the agent will attempt to load a cached value
however a specific UUID could be provided with this
flag.
--agent-id-file AGENT_ID_FILE
The location to store the agent's id. By default the
path is platform specific and defined by the
`agent_id_file_platform_defaults` key in the
configuration. [default: /etc/pyfarm/agent/uuid.dat]
Network Resources:
Resources which the agent will be communicating with.
--master MASTER This is a convenience flag which will allow you to set
the hostname for the master. By default this value
will be substituted in --master-api
--master-api MASTER_API
The location where the master's REST api is located.
[default: None]
--master-api-version MASTER_API_VERSION
Sets the version of the master's REST api the agent
shoulduse [default: None]
Process Control:
These settings apply to the parent process of the agent and contribute to
allowing the process to run as other users or remain isolated in an
environment. They also assist in maintaining the 'running state' via a
process id file.
--pidfile PIDFILE The file to store the process id in. [default: None]
-n, --no-daemon If provided then do not run the process in the
background.
--chdir CHDIR The working directory to change the agent into upon
launch
--uid UID The user id to run the agent as. *This setting is
ignored on Windows.*
--gid GID The group id to run the agent as. *This setting is
ignored on Windows.*
--pdb-on-unhandled When set pdb.set_trace() will be called if an
unhandled error is caught in the logger
pyfarm-agent is a command line client for working with a local agent. You can
use it to stop, start, and report the general status of a running agent
process.
usage: pyfarm-agent [status|start|stop] status [-h]
optional arguments:
-h, --help show this help message and exit
usage: pyfarm-agent [status|start|stop] start [-h] [--state STATE]
[--time-offset TIME_OFFSET]
[--ntp-server NTP_SERVER]
[--ntp-server-version NTP_SERVER_VERSION]
[--no-pretty-json]
[--shutdown-timeout SHUTDOWN_TIMEOUT]
[--updates-drop-dir UPDATES_DROP_DIR]
[--run-control-file RUN_CONTROL_FILE]
[--farm-name FARM_NAME]
[--cpus CPUS] [--ram RAM]
[--ram-check-interval RAM_CHECK_INTERVAL]
[--ram-max-report-frequency RAM_MAX_REPORT_FREQUENCY]
[--ram-report-delta RAM_REPORT_DELTA]
[--master-reannounce MASTER_REANNOUNCE]
[--log LOG]
[--capture-process-output]
[--task-log-dir TASK_LOG_DIR]
[--ip-remote IP_REMOTE]
[--enable-manhole]
[--manhole-port MANHOLE_PORT]
[--manhole-username MANHOLE_USERNAME]
[--manhole-password MANHOLE_PASSWORD]
[--html-templates-reload]
[--static-files STATIC_FILES]
[--http-retry-delay-offset HTTP_RETRY_DELAY_OFFSET]
[--http-retry-delay-factor HTTP_RETRY_DELAY_FACTOR]
[--jobtype-no-cache]
optional arguments:
-h, --help show this help message and exit
General Configuration:
These flags configure parts of the agent related to hardware, state, and
certain timing and scheduling attributes.
--state STATE The current agent state, valid values are ['disabled',
'offline', 'running', 'online']. [default: online]
--time-offset TIME_OFFSET
If provided then don't talk to the NTP server at all
to calculate the time offset. If you know for a fact
that this host's time is always up to date then
setting this to 0 is probably a safe bet.
--ntp-server NTP_SERVER
The default network time server this agent should
query to retrieve the real time. This will be used to
help determine the agent's clock skew if any. Setting
this value to '' will effectively disable this query.
[default: None]
--ntp-server-version NTP_SERVER_VERSION
The version of the NTP server in case it's running an
olderor newer version. [default: None]
--no-pretty-json If provided do not dump human readable json via the
agent's REST api
--shutdown-timeout SHUTDOWN_TIMEOUT
How many seconds the agent should spend attempting to
inform the master that it's shutting down.
--updates-drop-dir UPDATES_DROP_DIR
The directory to drop downloaded updates in. This
should be the same directory pyfarm-supervisor will
look for updates in. [default: None]
--run-control-file RUN_CONTROL_FILE
The path to a file that will signal to the supervisor
that agent is supposed to be restarted if it stops for
whatever reason.[default:
/tmp/pyfarm/agent/should_be_running]
--farm-name FARM_NAME
The name of the farm the agent should join. If unset,
the agent will join any farm.
Physical Hardware:
Command line flags which describe the hardware of the agent.
--cpus CPUS The total amount of cpus installed on the system.
Defaults to the number of cpus installed on the
system.
--ram RAM The total amount of ram installed on the system in
megabytes. Defaults to the amount of ram the system
has installed.
Interval Controls:
Controls which dictate when certain internal intervals should occur.
--ram-check-interval RAM_CHECK_INTERVAL
How often ram resources should be checked for changes.
The amount of memory currently being consumed on the
system is checked after certain events occur such as a
process but this flag specifically controls how often
we should check when no such events are occurring.
[default: None]
--ram-max-report-frequency RAM_MAX_REPORT_FREQUENCY
This is a limiter that prevents the agent from
reporting memory changes to the master more often than
a specific time interval. This is done in order to
ensure that when 100s of events fire in a short period
of time cause changes in ram usage only one or two
will be reported to the master. [default: None]
--ram-report-delta RAM_REPORT_DELTA
Only report a change in ram if the value has changed
at least this many megabytes. [default: None]
--master-reannounce MASTER_REANNOUNCE
Controls how often the agent should reannounce itself
to the master. The agent may be in contact with the
master more often than this however during long period
of inactivity this is how often the agent will
'inform' the master the agent is still online.
Logging Options:
Settings which control logging of the agent's parent process and/or any
subprocess it runs.
--log LOG If provided log all output from the agent to this
path. This will append to any existing log data.
[default: None]
--capture-process-output
If provided then all log output from each process
launched by the agent will be sent through agent's
loggers.
--task-log-dir TASK_LOG_DIR
The directory tasks should log to.
Network Service:
Controls how the agent is seen or interacted with by external services
such as the master.
--ip-remote IP_REMOTE
The remote IPv4 address to report. In situation where
the agent is behind a firewall this value will
typically be different.
Manhole Service:
Controls the manhole service which allows a telnet connection to be made
directly into the agent as it's running.
--enable-manhole When provided the manhole service will be started once
the reactor is running.
--manhole-port MANHOLE_PORT
The port the manhole service should run on if enabled.
--manhole-username MANHOLE_USERNAME
The telnet username that's allowed to connect to the
manhole service running on the agent.
--manhole-password MANHOLE_PASSWORD
The telnet password to use when connecting to the
manhole service running on the agent.
HTTP Configuration:
Options for how the agent will interact with the master's REST api and how
it should run it's own REST api.
--html-templates-reload
If provided then force Jinja2, the html template
system, to check the file system for changes with
every request. This flag should not be used in
production but is useful for development and debugging
purposes.
--static-files STATIC_FILES
The default location where the agent's http server
should find static files to serve.
--http-retry-delay-offset HTTP_RETRY_DELAY_OFFSET
If a http request to the master has failed, wait at
least this amount of time before resending the
request.
--http-retry-delay-factor HTTP_RETRY_DELAY_FACTOR
The value provided here is used in combination with
--http-retry-delay-offset to calculate the retry
delay. This is used as a multiplier against random()
before being added to the offset.
Job Types:
--jobtype-no-cache If provided then do not cache job types, always
directly retrieve them. This is beneficial if you're
testing the agent or a new job type class.
usage: pyfarm-agent [status|start|stop] stop [-h] [--no-wait]
optional arguments:
-h, --help show this help message and exit
optional flags:
Flags that control how the agent is stopped
--no-wait If provided then don't wait on the agent to shut itself down. By
default we would want to wait on each task to stop so we can
catch any errors and then finally wait on the agent to shutdown
too. If you're in a hurry or stopping a bunch of agents at once
then setting this flag will let the agent continue to stop
itself without waiting for each agent
usage: pyfarm-supervisor [-h] [--updates-drop-dir UPDATES_DROP_DIR]
[--agent-package-dir AGENT_PACKAGE_DIR]
[--pidfile PIDFILE] [-n] [--chdir CHDIR] [--uid UID]
[--gid GID]
Start and monitor the agent process
optional arguments:
-h, --help show this help message and exit
--updates-drop-dir UPDATES_DROP_DIR
Where to look for agent updates
--agent-package-dir AGENT_PACKAGE_DIR
Path to the actual agent code
--pidfile PIDFILE The file to store the process id in. [default: None]
-n, --no-daemon If provided then do not run the process in the
background.
--chdir CHDIR The directory to chdir to upon launch.
--uid UID The user id to run the supervisor as. *This setting is
ignored on Windows.*
--gid GID The group id to run the supervisor as. *This setting
is ignored on Windows.*
Development Commands¶
pyfarm-dev-fakerender¶
usage: pyfarm-dev-fakerender [-h] [--ram RAM] [--duration DURATION]
[--return-code RETURN_CODE]
[--duration-jitter DURATION_JITTER]
[--ram-jitter RAM_JITTER] -s START [-e END]
[-b BY] [--spew] [--segfault]
Very basic command line tool which vaguely simulates a render.
optional arguments:
-h, --help show this help message and exit
--ram RAM How much ram in megabytes the fake command should
consume
--duration DURATION How many seconds it should take to run this command
--return-code RETURN_CODE
The return code to return, declaring this flag
multiple times will result in a random return code.
[default: [0]]
--duration-jitter DURATION_JITTER
Randomly add or subtract this amount to the total
duration
--ram-jitter RAM_JITTER
Randomly add or subtract this amount to the ram
-s START, --start START
The start frame. If no other flags are provided this
will also be the end frame.
-e END, --end END The end frame
-b BY, --by BY The by frame
--spew Spews lots of random output to stdout which is
generally a decent stress test for log processing
issues. Do note however that this will disable the
code which is consuming extra CPU cycles. Also, use
this option with care as it can generate several
gigabytes of data per frame.
--segfault If provided then there's a 25% chance of causing a
segmentation fault.
pyfarm-dev-fakework¶
usage: pyfarm-dev-fakework [-h] [--master-api MASTER_API]
[--agent-api AGENT_API] [--jobtype JOBTYPE]
[--job JOB]
Quick and dirty script to create a job type, a job, and some tasks which are
then posted directly to the agent. The primary purpose of this script is to
test the internal of the job types
optional arguments:
-h, --help show this help message and exit
--master-api MASTER_API
The url to the master's api [default:
http://127.0.0.1/api/v1]
--agent-api AGENT_API
The url to the agent's api [default:
http://127.0.0.1:50000/api/v1]
--jobtype JOBTYPE The job type to use [default: FakeRender]
--job JOB If provided then this will be the job we pull tasks
from and assign to the agent. Please note we'll only
be pulling tasks that aren't running or assigned.