mirror of
https://github.com/valitydev/salt.git
synced 2024-11-07 08:58:59 +00:00
487 lines
12 KiB
ReStructuredText
487 lines
12 KiB
ReStructuredText
.. _jobs:
|
|
|
|
=======================
|
|
:index:`Job Management`
|
|
=======================
|
|
|
|
.. versionadded:: 0.9.7
|
|
|
|
Since Salt executes jobs running on many systems, Salt needs to be able to
|
|
manage jobs running on many systems.
|
|
|
|
The :index:`Minion proc System`
|
|
===============================
|
|
|
|
Salt Minions maintain a *proc* directory in the Salt ``cachedir``. The *proc*
|
|
directory maintains files named after the executed job ID. These files contain
|
|
the information about the current running jobs on the minion and allow for
|
|
jobs to be looked up. This is located in the *proc* directory under the
|
|
cachedir, with a default configuration it is under ``/var/cache/salt/proc``.
|
|
|
|
Functions in the saltutil Module
|
|
================================
|
|
|
|
Salt 0.9.7 introduced a few new functions to the
|
|
:doc:`saltutil</ref/modules/all/salt.modules.saltutil>` module for managing
|
|
jobs. These functions are:
|
|
|
|
1. ``running``
|
|
Returns the data of all running jobs that are found in the *proc* directory.
|
|
|
|
2. ``find_job``
|
|
Returns specific data about a certain job based on job id.
|
|
|
|
3. ``signal_job``
|
|
Allows for a given jid to be sent a signal.
|
|
|
|
4. ``term_job``
|
|
Sends a termination signal (SIGTERM, 15) to the process controlling the
|
|
specified job.
|
|
|
|
5. ``kill_job``
|
|
Sends a kill signal (SIGKILL, 9) to the process controlling the
|
|
specified job.
|
|
|
|
These functions make up the core of the back end used to manage jobs at the
|
|
minion level.
|
|
|
|
The jobs Runner
|
|
===============
|
|
|
|
A convenience runner front end and reporting system has been added as well.
|
|
The jobs runner contains functions to make viewing data easier and cleaner.
|
|
|
|
The jobs runner contains a number of functions...
|
|
|
|
active
|
|
------
|
|
|
|
The active function runs saltutil.running on all minions and formats the
|
|
return data about all running jobs in a much more usable and compact format.
|
|
The active function will also compare jobs that have returned and jobs that
|
|
are still running, making it easier to see what systems have completed a job
|
|
and what systems are still being waited on.
|
|
|
|
.. code-block:: bash
|
|
|
|
# salt-run jobs.active
|
|
|
|
lookup_jid
|
|
----------
|
|
|
|
When jobs are executed the return data is sent back to the master and cached.
|
|
By default it is cached for 24 hours, but this can be configured via the
|
|
``keep_jobs`` option in the master configuration.
|
|
Using the lookup_jid runner will display the same return data that the initial
|
|
job invocation with the salt command would display.
|
|
|
|
.. code-block:: bash
|
|
|
|
# salt-run jobs.lookup_jid <job id number>
|
|
|
|
list_jobs
|
|
---------
|
|
|
|
Before finding a historic job, it may be required to find the job id. ``list_jobs``
|
|
will parse the cached execution data and display all of the job data for jobs
|
|
that have already, or partially returned.
|
|
|
|
.. code-block:: bash
|
|
|
|
# salt-run jobs.list_jobs
|
|
|
|
.. _scheduling-jobs:
|
|
|
|
Scheduling Jobs
|
|
===============
|
|
|
|
Salt's scheduling system allows incremental executions on minions or the
|
|
master. The schedule system exposes the execution of any execution function on
|
|
minions or any runner on the master.
|
|
|
|
Scheduling can be enabled by multiple methods:
|
|
|
|
- ``schedule`` option in either the master or minion config files. These
|
|
require the master or minion application to be restarted in order for the
|
|
schedule to be implemented.
|
|
|
|
- Minion pillar data. Schedule is implemented by refreshing the minion's pillar data,
|
|
for example by using ``saltutil.refresh_pillar``.
|
|
|
|
- The :doc:`schedule state</ref/states/all/salt.states.schedule>` or
|
|
:doc:`schedule module</ref/modules/all/salt.modules.schedule>`
|
|
|
|
.. note::
|
|
|
|
The scheduler executes different functions on the master and minions. When
|
|
running on the master the functions reference runner functions, when
|
|
running on the minion the functions specify execution functions.
|
|
|
|
A scheduled run has no output on the minion unless the config is set to info level
|
|
or higher. Refer to :conf_minion:`minion-logging-settings`.
|
|
|
|
States are executed on the minion, as all states are. You can pass positional
|
|
arguments and provide a YAML dict of named arguments.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 3600
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
|
|
This will schedule the command: ``state.sls httpd test=True`` every 3600 seconds
|
|
(every hour).
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 3600
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
splay: 15
|
|
|
|
This will schedule the command: ``state.sls httpd test=True`` every 3600 seconds
|
|
(every hour) splaying the time between 0 and 15 seconds.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 3600
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
splay:
|
|
start: 10
|
|
end: 15
|
|
|
|
This will schedule the command: ``state.sls httpd test=True`` every 3600 seconds
|
|
(every hour) splaying the time between 10 and 15 seconds.
|
|
|
|
Schedule by Date and Time
|
|
-------------------------
|
|
|
|
.. versionadded:: 2014.7.0
|
|
|
|
Frequency of jobs can also be specified using date strings supported by
|
|
the Python ``dateutil`` library. This requires the Python ``dateutil`` library
|
|
to be installed.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
when: 5:00pm
|
|
|
|
This will schedule the command: ``state.sls httpd test=True`` at 5:00 PM minion
|
|
localtime.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
when:
|
|
- Monday 5:00pm
|
|
- Tuesday 3:00pm
|
|
- Wednesday 5:00pm
|
|
- Thursday 3:00pm
|
|
- Friday 5:00pm
|
|
|
|
This will schedule the command: ``state.sls httpd test=True`` at 5:00 PM on
|
|
Monday, Wednesday and Friday, and 3:00 PM on Tuesday and Thursday.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 3600
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
range:
|
|
start: 8:00am
|
|
end: 5:00pm
|
|
|
|
This will schedule the command: ``state.sls httpd test=True`` every 3600 seconds
|
|
(every hour) between the hours of 8:00 AM and 5:00 PM. The range parameter must
|
|
be a dictionary with the date strings using the ``dateutil`` format.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 3600
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
range:
|
|
invert: True
|
|
start: 8:00am
|
|
end: 5:00pm
|
|
|
|
Using the invert option for range, this will schedule the command
|
|
``state.sls httpd test=True`` every 3600 seconds (every hour) until the current
|
|
time is between the hours of 8:00 AM and 5:00 PM. The range parameter must be
|
|
a dictionary with the date strings using the ``dateutil`` format.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: pkg.install
|
|
kwargs:
|
|
pkgs: [{'bar': '>1.2.3'}]
|
|
refresh: true
|
|
once: '2016-01-07T14:30:00'
|
|
|
|
This will schedule the function ``pkg.install`` to be executed once at the
|
|
specified time. The schedule entry ``job1`` will not be removed after the job
|
|
completes, therefore use ``schedule.delete`` to manually remove it afterwards.
|
|
|
|
The default date format is ISO 8601 but can be overridden by also specifying the
|
|
``once_fmt`` option, like this:
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: test.ping
|
|
once: 2015-04-22T20:21:00
|
|
once_fmt: '%Y-%m-%dT%H:%M:%S'
|
|
|
|
Maximum Parallel Jobs Running
|
|
-----------------------------
|
|
|
|
.. versionadded:: 2014.7.0
|
|
|
|
The scheduler also supports ensuring that there are no more than N copies of
|
|
a particular routine running. Use this for jobs that may be long-running
|
|
and could step on each other or pile up in case of infrastructure outage.
|
|
|
|
The default for ``maxrunning`` is 1.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
long_running_job:
|
|
function: big_file_transfer
|
|
jid_include: True
|
|
maxrunning: 1
|
|
|
|
Cron-like Schedule
|
|
------------------
|
|
|
|
.. versionadded:: 2014.7.0
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
cron: '*/15 * * * *'
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
|
|
The scheduler also supports scheduling jobs using a cron like format.
|
|
This requires the Python ``croniter`` library.
|
|
|
|
Job Data Return
|
|
---------------
|
|
|
|
.. versionadded:: 2015.5.0
|
|
|
|
By default, data about jobs runs from the Salt scheduler is returned to the
|
|
master. Setting the ``return_job`` parameter to False will prevent the data
|
|
from being sent back to the Salt master.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: scheduled_job_function
|
|
return_job: False
|
|
|
|
Job Metadata
|
|
------------
|
|
|
|
.. versionadded:: 2015.5.0
|
|
|
|
It can be useful to include specific data to differentiate a job from other
|
|
jobs. Using the metadata parameter special values can be associated with
|
|
a scheduled job. These values are not used in the execution of the job,
|
|
but can be used to search for specific jobs later if combined with the
|
|
``return_job`` parameter. The metadata parameter must be specified as a
|
|
dictionary, othewise it will be ignored.
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: scheduled_job_function
|
|
metadata:
|
|
foo: bar
|
|
|
|
Run on Start
|
|
------------
|
|
|
|
.. versionadded:: 2015.5.0
|
|
|
|
By default, any job scheduled based on the startup time of the minion will run
|
|
the scheduled job when the minion starts up. Sometimes this is not the desired
|
|
situation. Using the ``run_on_start`` parameter set to ``False`` will cause the
|
|
scheduler to skip this first run and wait until the next scheduled run:
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 3600
|
|
run_on_start: False
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
|
|
Until and After
|
|
---------------
|
|
|
|
.. versionadded:: 2015.8.0
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 15
|
|
until: '12/31/2015 11:59pm'
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
|
|
Using the until argument, the Salt scheduler allows you to specify
|
|
an end time for a scheduled job. If this argument is specified, jobs
|
|
will not run once the specified time has passed. Time should be specified
|
|
in a format supported by the ``dateutil`` library.
|
|
This requires the Python ``dateutil`` library to be installed.
|
|
|
|
.. versionadded:: 2015.8.0
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
job1:
|
|
function: state.sls
|
|
seconds: 15
|
|
after: '12/31/2015 11:59pm'
|
|
args:
|
|
- httpd
|
|
kwargs:
|
|
test: True
|
|
|
|
Using the after argument, the Salt scheduler allows you to specify
|
|
an start time for a scheduled job. If this argument is specified, jobs
|
|
will not run until the specified time has passed. Time should be specified
|
|
in a format supported by the ``dateutil`` library.
|
|
This requires the Python ``dateutil`` library to be installed.
|
|
|
|
Scheduling States
|
|
-----------------
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
log-loadavg:
|
|
function: cmd.run
|
|
seconds: 3660
|
|
args:
|
|
- 'logger -t salt < /proc/loadavg'
|
|
kwargs:
|
|
stateful: False
|
|
shell: /bin/sh
|
|
|
|
Scheduling Highstates
|
|
---------------------
|
|
|
|
To set up a highstate to run on a minion every 60 minutes set this in the
|
|
minion config or pillar:
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
highstate:
|
|
function: state.highstate
|
|
minutes: 60
|
|
|
|
Time intervals can be specified as seconds, minutes, hours, or days.
|
|
|
|
Scheduling Runners
|
|
------------------
|
|
|
|
Runner executions can also be specified on the master within the master
|
|
configuration file:
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
run_my_orch:
|
|
function: state.orchestrate
|
|
hours: 6
|
|
splay: 600
|
|
args:
|
|
- orchestration.my_orch
|
|
|
|
The above configuration is analogous to running
|
|
``salt-run state.orch orchestration.my_orch`` every 6 hours.
|
|
|
|
Scheduler With Returner
|
|
-----------------------
|
|
|
|
The scheduler is also useful for tasks like gathering monitoring data about
|
|
a minion, this schedule option will gather status data and send it to a MySQL
|
|
returner database:
|
|
|
|
.. code-block:: yaml
|
|
|
|
schedule:
|
|
uptime:
|
|
function: status.uptime
|
|
seconds: 60
|
|
returner: mysql
|
|
meminfo:
|
|
function: status.meminfo
|
|
minutes: 5
|
|
returner: mysql
|
|
|
|
Since specifying the returner repeatedly can be tiresome, the
|
|
``schedule_returner`` option is available to specify one or a list of global
|
|
returners to be used by the minions when scheduling.
|