valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-08 09:23:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'develop' into alternate_composefile

Browse Source
This commit is contained in:
Sjuul Janssen 2017-12-04 10:31:10 +01:00 committed by GitHub
commit 0d81aa83e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
39 changed files with 1487 additions and 335 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
conf/master
View File

@ -297,6 +297,11 @@
#batch_safe_limit: 100 #batch_safe_limit: 100
#batch_safe_size: 8 #batch_safe_size: 8
# Master stats enables stats events to be fired from the master at close
# to the defined interval
#master_stats: False
#master_stats_event_iter: 60
##### Security settings ##### ##### Security settings #####
########################################## ##########################################

23
doc/ref/configuration/master.rst
View File

@ -868,6 +868,29 @@ what you are doing! Transports are explained in :ref:`Salt Transports
ret_port: 4606 ret_port: 4606
zeromq: [] zeromq: []
.. conf_master:: master_stats
``master_stats``
----------------
Default: False
Turning on the master stats enables runtime throughput and statistics events
to be fired from the master event bus. These events will report on what
functions have been run on the master and how long these runs have, on
average, taken over a given period of time.
.. conf_master:: master_stats_event_iter
``master_stats_event_iter``
---------------------------
Default: 60
The time in seconds to fire master_stats events. This will only fire in
conjunction with receiving a request to the master, idle masters will not
fire these events.
.. conf_master:: sock_pool_size .. conf_master:: sock_pool_size
``sock_pool_size`` ``sock_pool_size``

2
doc/ref/states/writing.rst
View File

@ -111,6 +111,8 @@ This code will call the `managed` function in the :mod:`file
<salt.states.file>` state module and pass the arguments ``name`` and ``source`` <salt.states.file>` state module and pass the arguments ``name`` and ``source``
to it. to it.
.. _state-return-data:
Return Data Return Data
=========== ===========

28
doc/topics/orchestrate/orchestrate_runner.rst
View File

@ -222,6 +222,34 @@ To execute with pillar data.
"master": "mymaster"}' "master": "mymaster"}'
Return Codes in Runner/Wheel Jobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. versionadded:: Oxygen
State (``salt.state``) jobs are able to report failure via the :ref:`state
return dictionary <state-return-data>`. Remote execution (``salt.function``)
jobs are able to report failure by setting a ``retcode`` key in the
``__context__`` dictionary. However, runner (``salt.runner``) and wheel
(``salt.wheel``) jobs would only report a ``False`` result when the
runner/wheel function raised an exception. As of the Oxygen release, it is now
possible to set a retcode in runner and wheel functions just as you can do in
remote execution functions. Here is some example pseudocode:
.. code-block:: python
def myrunner():
...
do stuff
...
if some_error_condition:
__context__['retcode'] = 1
return result
This allows a custom runner/wheel function to report its failure so that
requisites can accurately tell that a job has failed.
More Complex Orchestration More Complex Orchestration
~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~

19
doc/topics/releases/oxygen.rst
View File

@ -25,6 +25,25 @@ by any master tops matches that are not matched via a top file.
To make master tops matches execute first, followed by top file matches, set To make master tops matches execute first, followed by top file matches, set
the new :conf_minion:`master_tops_first` minion config option to ``True``. the new :conf_minion:`master_tops_first` minion config option to ``True``.
Return Codes for Runner/Wheel Functions
---------------------------------------
When using :ref:`orchestration <orchestrate-runner>`, runner and wheel
functions used to report a ``True`` result if the function ran to completion
without raising an exception. It is now possible to set a return code in the
``__context__`` dictionary, allowing runner and wheel functions to report that
they failed. Here's some example pseudocode:
.. code-block:: python
def myrunner():
...
do stuff
...
if some_error_condition:
__context__['retcode'] = 1
return result
LDAP via External Authentication Changes LDAP via External Authentication Changes
---------------------------------------- ----------------------------------------
In this release of Salt, if LDAP Bind Credentials are supplied, then In this release of Salt, if LDAP Bind Credentials are supplied, then

120
doc/topics/tutorials/salt_bootstrap.rst
View File

@ -78,7 +78,7 @@ UNIX systems
**BSD**: **BSD**:
- OpenBSD (``pip`` installation) - OpenBSD
- FreeBSD 9/10/11 - FreeBSD 9/10/11
**SunOS**: **SunOS**:
@ -272,66 +272,118 @@ Here's a summary of the command line options:
$ sh bootstrap-salt.sh -h $ sh bootstrap-salt.sh -h
Usage : bootstrap-salt.sh [options] <install-type> <install-type-args>
Installation types: Installation types:
- stable (default) - stable Install latest stable release. This is the default
- stable [version] (ubuntu specific) install type
- daily (ubuntu specific) - stable [branch] Install latest version on a branch. Only supported
- testing (redhat specific) for packages available at repo.saltstack.com
- git - stable [version] Install a specific version. Only supported for
packages available at repo.saltstack.com
- daily Ubuntu specific: configure SaltStack Daily PPA
- testing RHEL-family specific: configure EPEL testing repo
- git Install from the head of the develop branch
- git [ref] Install from any git ref (such as a branch, tag, or
commit)
Examples: Examples:
- bootstrap-salt.sh - bootstrap-salt.sh
- bootstrap-salt.sh stable - bootstrap-salt.sh stable
- bootstrap-salt.sh stable 2014.7 - bootstrap-salt.sh stable 2017.7
- bootstrap-salt.sh stable 2017.7.2
- bootstrap-salt.sh daily - bootstrap-salt.sh daily
- bootstrap-salt.sh testing - bootstrap-salt.sh testing
- bootstrap-salt.sh git - bootstrap-salt.sh git
- bootstrap-salt.sh git develop - bootstrap-salt.sh git 2017.7
- bootstrap-salt.sh git v0.17.0 - bootstrap-salt.sh git v2017.7.2
- bootstrap-salt.sh git 8c3fadf15ec183e5ce8c63739850d543617e4357 - bootstrap-salt.sh git 06f249901a2e2f1ed310d58ea3921a129f214358
Options: Options:
-h Display this message -h Display this message
-v Display script version -v Display script version
-n No colours. -n No colours
-D Show debug output. -D Show debug output
-c Temporary configuration directory -c Temporary configuration directory
-g Salt repository URL. (default: git://github.com/saltstack/salt.git) -g Salt Git repository URL. Default: https://github.com/saltstack/salt.git
-G Instead of cloning from git://github.com/saltstack/salt.git, clone from https://github.com/saltstack/salt.git (Usually necessary on systems which have the regular git protocol port blocked, where https usually is not) -w Install packages from downstream package repository rather than
upstream, saltstack package repository. This is currently only
implemented for SUSE.
-k Temporary directory holding the minion keys which will pre-seed -k Temporary directory holding the minion keys which will pre-seed
the master. the master.
-s Sleep time used when waiting for daemons to start, restart and when checking -s Sleep time used when waiting for daemons to start, restart and when
for the services running. Default: 3 checking for the services running. Default: 3
-L Also install salt-cloud and required python-libcloud package
-M Also install salt-master -M Also install salt-master
-S Also install salt-syndic -S Also install salt-syndic
-N Do not install salt-minion -N Do not install salt-minion
-X Do not start daemons after installation -X Do not start daemons after installation
-C Only run the configuration function. This option automatically -d Disables checking if Salt services are enabled to start on system boot.
bypasses any installation. You can also do this by touching /tmp/disable_salt_checks on the target
host. Default: ${BS_FALSE}
-P Allow pip based installations. On some distributions the required salt -P Allow pip based installations. On some distributions the required salt
packages or its dependencies are not available as a package for that packages or its dependencies are not available as a package for that
distribution. Using this flag allows the script to use pip as a last distribution. Using this flag allows the script to use pip as a last
resort method. NOTE: This only works for functions which actually resort method. NOTE: This only works for functions which actually
implement pip based installations. implement pip based installations.
-F Allow copied files to overwrite existing(config, init.d, etc) -U If set, fully upgrade the system prior to bootstrapping Salt
-U If set, fully upgrade the system prior to bootstrapping salt
-K If set, keep the temporary files in the temporary directories specified
with -c and -k.
-I If set, allow insecure connections while downloading any files. For -I If set, allow insecure connections while downloading any files. For
example, pass '--no-check-certificate' to 'wget' or '--insecure' to 'curl' example, pass '--no-check-certificate' to 'wget' or '--insecure' to
'curl'. On Debian and Ubuntu, using this option with -U allows to obtain
GnuPG archive keys insecurely if distro has changed release signatures.
-F Allow copied files to overwrite existing (config, init.d, etc)
-K If set, keep the temporary files in the temporary directories specified
with -c and -k
-C Only run the configuration function. Implies -F (forced overwrite).
To overwrite Master or Syndic configs, -M or -S, respectively, must
also be specified. Salt installation will be ommitted, but some of the
dependencies could be installed to write configuration with -j or -J.
-A Pass the salt-master DNS name or IP. This will be stored under -A Pass the salt-master DNS name or IP. This will be stored under
${BS_SALT_ETC_DIR}/minion.d/99-master-address.conf ${BS_SALT_ETC_DIR}/minion.d/99-master-address.conf
-i Pass the salt-minion id. This will be stored under -i Pass the salt-minion id. This will be stored under
${BS_SALT_ETC_DIR}/minion_id ${BS_SALT_ETC_DIR}/minion_id
-L Install the Apache Libcloud package if possible(required for salt-cloud) -p Extra-package to install while installing Salt dependencies. One package
-p Extra-package to install while installing salt dependencies. One package
per -p flag. You're responsible for providing the proper package name. per -p flag. You're responsible for providing the proper package name.
-d Disable check_service functions. Setting this flag disables the -H Use the specified HTTP proxy for all download URLs (including https://).
'install_<distro>_check_services' checks. You can also do this by For example: http://myproxy.example.com:3128
touching /tmp/disable_salt_checks on the target host. Defaults ${BS_FALSE} -Z Enable additional package repository for newer ZeroMQ
-H Use the specified http proxy for the installation (only available for RHEL/CentOS/Fedora/Ubuntu based distributions)
-Z Enable external software source for newer ZeroMQ(Only available for RHEL/CentOS/Fedora/Ubuntu based distributions) -b Assume that dependencies are already installed and software sources are
-b Assume that dependencies are already installed and software sources are set up. set up. If git is selected, git tree is still checked out as dependency
If git is selected, git tree is still checked out as dependency step. step.
-f Force shallow cloning for git installations.
This may result in an "n/a" in the version number.
-l Disable ssl checks. When passed, switches "https" calls to "http" where
possible.
-V Install Salt into virtualenv
(only available for Ubuntu based distributions)
-a Pip install all Python pkg dependencies for Salt. Requires -V to install
all pip pkgs into the virtualenv.
(Only available for Ubuntu based distributions)
-r Disable all repository configuration performed by this script. This
option assumes all necessary repository configuration is already present
on the system.
-R Specify a custom repository URL. Assumes the custom repository URL
points to a repository that mirrors Salt packages located at
repo.saltstack.com. The option passed with -R replaces the
"repo.saltstack.com". If -R is passed, -r is also set. Currently only
works on CentOS/RHEL and Debian based distributions.
-J Replace the Master config file with data passed in as a JSON string. If
a Master config file is found, a reasonable effort will be made to save
the file with a ".bak" extension. If used in conjunction with -C or -F,
no ".bak" file will be created as either of those options will force
a complete overwrite of the file.
-j Replace the Minion config file with data passed in as a JSON string. If
a Minion config file is found, a reasonable effort will be made to save
the file with a ".bak" extension. If used in conjunction with -C or -F,
no ".bak" file will be created as either of those options will force
a complete overwrite of the file.
-q Quiet salt installation from git (setup.py install -q)
-x Changes the python version used to install a git version of salt. Currently
this is considered experimental and has only been tested on Centos 6. This
only works for git installations.
-y Installs a different python version on host. Currently this has only been
tested with Centos 6 and is considered experimental. This will install the
ius repo on the box if disable repo is false. This must be used in conjunction
with -x <pythonversion>. For example:
sh bootstrap.sh -P -y -x python2.7 git v2016.11.3
The above will install python27 and install the git version of salt using the
python2.7 executable. This only works for git and pip installations.

4
salt/cli/daemons.py
View File

@ -161,7 +161,6 @@ class Master(salt.utils.parsers.MasterOptionParser, DaemonsMixin): # pylint: di
v_dirs, v_dirs,
self.config['user'], self.config['user'],
permissive=self.config['permissive_pki_access'], permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'], root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir'], self.config['key_dir']], sensitive_dirs=[self.config['pki_dir'], self.config['key_dir']],
) )
@ -283,7 +282,6 @@ class Minion(salt.utils.parsers.MinionOptionParser, DaemonsMixin): # pylint: di
v_dirs, v_dirs,
self.config['user'], self.config['user'],
permissive=self.config['permissive_pki_access'], permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'], root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir']], sensitive_dirs=[self.config['pki_dir']],
) )
@ -472,7 +470,6 @@ class ProxyMinion(salt.utils.parsers.ProxyMinionOptionParser, DaemonsMixin): #
v_dirs, v_dirs,
self.config['user'], self.config['user'],
permissive=self.config['permissive_pki_access'], permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'], root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir']], sensitive_dirs=[self.config['pki_dir']],
) )
@ -582,7 +579,6 @@ class Syndic(salt.utils.parsers.SyndicOptionParser, DaemonsMixin): # pylint: di
], ],
self.config['user'], self.config['user'],
permissive=self.config['permissive_pki_access'], permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'], root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir']], sensitive_dirs=[self.config['pki_dir']],
) )

4
salt/client/mixins.py
View File

@ -385,6 +385,10 @@ class SyncClientMixin(object):
# Initialize a context for executing the method. # Initialize a context for executing the method.
with tornado.stack_context.StackContext(self.functions.context_dict.clone): with tornado.stack_context.StackContext(self.functions.context_dict.clone):
data[u'return'] = self.functions[fun](*args, **kwargs) data[u'return'] = self.functions[fun](*args, **kwargs)
try:
data[u'success'] = self.context.get(u'retcode', 0) == 0
except AttributeError:
# Assume a True result if no context attribute
data[u'success'] = True data[u'success'] = True
if isinstance(data[u'return'], dict) and u'data' in data[u'return']: if isinstance(data[u'return'], dict) and u'data' in data[u'return']:
# some functions can return boolean values # some functions can return boolean values

6
salt/config/__init__.py
View File

@ -165,6 +165,10 @@ VALID_OPTS = {
# The master_pubkey_signature must also be set for this. # The master_pubkey_signature must also be set for this.
'master_use_pubkey_signature': bool, 'master_use_pubkey_signature': bool,
# Enable master stats eveents to be fired, these events will contain information about
# what commands the master is processing and what the rates are of the executions
'master_stats': bool,
'master_stats_event_iter': int,
# The key fingerprint of the higher-level master for the syndic to verify it is talking to the # The key fingerprint of the higher-level master for the syndic to verify it is talking to the
# intended master # intended master
'syndic_finger': str, 'syndic_finger': str,
@ -1515,6 +1519,8 @@ DEFAULT_MASTER_OPTS = {
'svnfs_saltenv_whitelist': [], 'svnfs_saltenv_whitelist': [],
'svnfs_saltenv_blacklist': [], 'svnfs_saltenv_blacklist': [],
'max_event_size': 1048576, 'max_event_size': 1048576,
'master_stats': False,
'master_stats_event_iter': 60,
'minionfs_env': 'base', 'minionfs_env': 'base',
'minionfs_mountpoint': '', 'minionfs_mountpoint': '',
'minionfs_whitelist': [], 'minionfs_whitelist': [],

11
salt/loader.py
View File

@ -372,15 +372,18 @@ def tops(opts):
return FilterDictWrapper(ret, u'.top') return FilterDictWrapper(ret, u'.top')
def wheels(opts, whitelist=None): def wheels(opts, whitelist=None, context=None):
''' '''
Returns the wheels modules Returns the wheels modules
''' '''
if context is None:
context = {}
return LazyLoader( return LazyLoader(
_module_dirs(opts, u'wheel'), _module_dirs(opts, u'wheel'),
opts, opts,
tag=u'wheel', tag=u'wheel',
whitelist=whitelist, whitelist=whitelist,
pack={u'__context__': context},
) )
@ -836,17 +839,19 @@ def call(fun, **kwargs):
return funcs[fun](*args) return funcs[fun](*args)
def runner(opts, utils=None): def runner(opts, utils=None, context=None):
''' '''
Directly call a function inside a loader directory Directly call a function inside a loader directory
''' '''
if utils is None: if utils is None:
utils = {} utils = {}
if context is None:
context = {}
ret = LazyLoader( ret = LazyLoader(
_module_dirs(opts, u'runners', u'runner', ext_type_dirs=u'runner_dirs'), _module_dirs(opts, u'runners', u'runner', ext_type_dirs=u'runner_dirs'),
opts, opts,
tag=u'runners', tag=u'runners',
pack={u'__utils__': utils}, pack={u'__utils__': utils, u'__context__': context},
) )
# TODO: change from __salt__ to something else, we overload __salt__ too much # TODO: change from __salt__ to something else, we overload __salt__ too much
ret.pack[u'__salt__'] = ret ret.pack[u'__salt__'] = ret

39
salt/master.py
View File

@ -16,6 +16,7 @@ import errno
import signal import signal
import stat import stat
import logging import logging
import collections
import multiprocessing import multiprocessing
import salt.serializers.msgpack import salt.serializers.msgpack
@ -797,6 +798,7 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
:return: Master worker :return: Master worker
''' '''
kwargs[u'name'] = name kwargs[u'name'] = name
self.name = name
super(MWorker, self).__init__(**kwargs) super(MWorker, self).__init__(**kwargs)
self.opts = opts self.opts = opts
self.req_channels = req_channels self.req_channels = req_channels
@ -804,6 +806,8 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
self.mkey = mkey self.mkey = mkey
self.key = key self.key = key
self.k_mtime = 0 self.k_mtime = 0
self.stats = collections.defaultdict(lambda: {'mean': 0, 'runs': 0})
self.stat_clock = time.time()
# We need __setstate__ and __getstate__ to also pickle 'SMaster.secrets'. # We need __setstate__ and __getstate__ to also pickle 'SMaster.secrets'.
# Otherwise, 'SMaster.secrets' won't be copied over to the spawned process # Otherwise, 'SMaster.secrets' won't be copied over to the spawned process
@ -879,6 +883,19 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
u'clear': self._handle_clear}[key](load) u'clear': self._handle_clear}[key](load)
raise tornado.gen.Return(ret) raise tornado.gen.Return(ret)
def _post_stats(self, start, cmd):
'''
Calculate the master stats and fire events with stat info
'''
end = time.time()
duration = end - start
self.stats[cmd][u'mean'] = (self.stats[cmd][u'mean'] * (self.stats[cmd][u'runs'] - 1) + duration) / self.stats[cmd][u'runs']
if end - self.stat_clock > self.opts[u'master_stats_event_iter']:
# Fire the event with the stats and wipe the tracker
self.aes_funcs.event.fire_event({u'time': end - self.stat_clock, u'worker': self.name, u'stats': self.stats}, tagify(self.name, u'stats'))
self.stats = collections.defaultdict(lambda: {'mean': 0, 'runs': 0})
self.stat_clock = end
def _handle_clear(self, load): def _handle_clear(self, load):
''' '''
Process a cleartext command Process a cleartext command
@ -888,9 +905,16 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
the command specified in the load's 'cmd' key. the command specified in the load's 'cmd' key.
''' '''
log.trace(u'Clear payload received with command %s', load[u'cmd']) log.trace(u'Clear payload received with command %s', load[u'cmd'])
if load[u'cmd'].startswith(u'__'): cmd = load[u'cmd']
if cmd.startswith(u'__'):
return False return False
return getattr(self.clear_funcs, load[u'cmd'])(load), {u'fun': u'send_clear'} if self.opts[u'master_stats']:
start = time.time()
self.stats[cmd][u'runs'] += 1
ret = getattr(self.clear_funcs, cmd)(load), {u'fun': u'send_clear'}
if self.opts[u'master_stats']:
self._post_stats(start, cmd)
return ret
def _handle_aes(self, data): def _handle_aes(self, data):
''' '''
@ -903,10 +927,17 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
if u'cmd' not in data: if u'cmd' not in data:
log.error(u'Received malformed command %s', data) log.error(u'Received malformed command %s', data)
return {} return {}
cmd = data[u'cmd']
log.trace(u'AES payload received with command %s', data[u'cmd']) log.trace(u'AES payload received with command %s', data[u'cmd'])
if data[u'cmd'].startswith(u'__'): if cmd.startswith(u'__'):
return False return False
return self.aes_funcs.run_func(data[u'cmd'], data) if self.opts[u'master_stats']:
start = time.time()
self.stats[cmd][u'runs'] += 1
ret = self.aes_funcs.run_func(data[u'cmd'], data)
if self.opts[u'master_stats']:
self._post_stats(start, cmd)
return ret
def run(self): def run(self):
''' '''

4
salt/minion.py
View File

@ -2067,12 +2067,16 @@ class Minion(MinionBase):
self.schedule.run_job(name) self.schedule.run_job(name)
elif func == u'disable_job': elif func == u'disable_job':
self.schedule.disable_job(name, persist) self.schedule.disable_job(name, persist)
elif func == u'postpone_job':
self.schedule.postpone_job(name, data)
elif func == u'reload': elif func == u'reload':
self.schedule.reload(schedule) self.schedule.reload(schedule)
elif func == u'list': elif func == u'list':
self.schedule.list(where) self.schedule.list(where)
elif func == u'save_schedule': elif func == u'save_schedule':
self.schedule.save_schedule() self.schedule.save_schedule()
elif func == u'get_next_fire_time':
self.schedule.get_next_fire_time(name)
def manage_beacons(self, tag, data): def manage_beacons(self, tag, data):
''' '''

4
salt/modules/ansiblegate.py
View File

@ -214,9 +214,7 @@ def __virtual__():
''' '''
ret = ansible is not None ret = ansible is not None
msg = not ret and "Ansible is not installed on this system" or None msg = not ret and "Ansible is not installed on this system" or None
if msg: if ret:
log.warning(msg)
else:
global _resolver global _resolver
global _caller global _caller
_resolver = AnsibleModuleResolver(__opts__).resolve().install() _resolver = AnsibleModuleResolver(__opts__).resolve().install()

58
salt/modules/file.py
View File

@ -552,11 +552,11 @@ def lsattr(path):
raise SaltInvocationError("File or directory does not exist.") raise SaltInvocationError("File or directory does not exist.")
cmd = ['lsattr', path] cmd = ['lsattr', path]
result = __salt__['cmd.run'](cmd, python_shell=False) result = __salt__['cmd.run'](cmd, ignore_retcode=True, python_shell=False)
results = {} results = {}
for line in result.splitlines(): for line in result.splitlines():
if not line.startswith('lsattr'): if not line.startswith('lsattr: '):
vals = line.split(None, 1) vals = line.split(None, 1)
results[vals[1]] = re.findall(r"[acdijstuADST]", vals[0]) results[vals[1]] = re.findall(r"[acdijstuADST]", vals[0])
@ -5203,13 +5203,18 @@ def manage_file(name,
'Replace symbolic link with regular file' 'Replace symbolic link with regular file'
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = check_perms(name, # This function resides in win_file.py and will be available
ret, # on Windows. The local function will be overridden
kwargs.get('win_owner'), # pylint: disable=E1120,E1121,E1123
kwargs.get('win_perms'), ret = check_perms(
kwargs.get('win_deny_perms'), path=name,
None, ret=ret,
kwargs.get('win_inheritance')) owner=kwargs.get('win_owner'),
grant_perms=kwargs.get('win_perms'),
deny_perms=kwargs.get('win_deny_perms'),
inheritance=kwargs.get('win_inheritance', True),
reset=kwargs.get('win_perms_reset', False))
# pylint: enable=E1120,E1121,E1123
else: else:
ret, _ = check_perms(name, ret, user, group, mode, attrs, follow_symlinks) ret, _ = check_perms(name, ret, user, group, mode, attrs, follow_symlinks)
@ -5250,13 +5255,15 @@ def manage_file(name,
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
# This function resides in win_file.py and will be available # This function resides in win_file.py and will be available
# on Windows. The local function will be overridden # on Windows. The local function will be overridden
# pylint: disable=E1121 # pylint: disable=E1120,E1121,E1123
makedirs_(name, makedirs_(
kwargs.get('win_owner'), path=name,
kwargs.get('win_perms'), owner=kwargs.get('win_owner'),
kwargs.get('win_deny_perms'), grant_perms=kwargs.get('win_perms'),
kwargs.get('win_inheritance')) deny_perms=kwargs.get('win_deny_perms'),
# pylint: enable=E1121 inheritance=kwargs.get('win_inheritance', True),
reset=kwargs.get('win_perms_reset', False))
# pylint: enable=E1120,E1121,E1123
else: else:
makedirs_(name, user=user, group=group, mode=dir_mode) makedirs_(name, user=user, group=group, mode=dir_mode)
@ -5369,13 +5376,18 @@ def manage_file(name,
mode = oct((0o777 ^ mask) & 0o666) mode = oct((0o777 ^ mask) & 0o666)
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = check_perms(name, # This function resides in win_file.py and will be available
ret, # on Windows. The local function will be overridden
kwargs.get('win_owner'), # pylint: disable=E1120,E1121,E1123
kwargs.get('win_perms'), ret = check_perms(
kwargs.get('win_deny_perms'), path=name,
None, ret=ret,
kwargs.get('win_inheritance')) owner=kwargs.get('win_owner'),
grant_perms=kwargs.get('win_perms'),
deny_perms=kwargs.get('win_deny_perms'),
inheritance=kwargs.get('win_inheritance', True),
reset=kwargs.get('win_perms_reset', False))
# pylint: enable=E1120,E1121,E1123
else: else:
ret, _ = check_perms(name, ret, user, group, mode, attrs) ret, _ = check_perms(name, ret, user, group, mode, attrs)

10
salt/modules/rh_ip.py
View File

@ -643,12 +643,18 @@ def _parse_settings_eth(opts, iface_type, enabled, iface):
result[opt] = opts[opt] result[opt] = opts[opt]
if iface_type not in ['bond', 'vlan', 'bridge', 'ipip']: if iface_type not in ['bond', 'vlan', 'bridge', 'ipip']:
auto_addr = False
if 'addr' in opts: if 'addr' in opts:
if salt.utils.validate.net.mac(opts['addr']): if salt.utils.validate.net.mac(opts['addr']):
result['addr'] = opts['addr'] result['addr'] = opts['addr']
elif opts['addr'] == 'auto':
auto_addr = True
elif opts['addr'] != 'none':
_raise_error_iface(iface, opts['addr'], ['AA:BB:CC:DD:EE:FF', 'auto', 'none'])
else: else:
_raise_error_iface(iface, opts['addr'], ['AA:BB:CC:DD:EE:FF']) auto_addr = True
else:
if auto_addr:
# If interface type is slave for bond, not setting hwaddr # If interface type is slave for bond, not setting hwaddr
if iface_type != 'slave': if iface_type != 'slave':
ifaces = __salt__['network.interfaces']() ifaces = __salt__['network.interfaces']()

43
salt/modules/saltutil.py
View File

@ -474,7 +474,7 @@ def sync_returners(saltenv=None, refresh=True, extmod_whitelist=None, extmod_bla
''' '''
.. versionadded:: 0.10.0 .. versionadded:: 0.10.0
Sync beacons from ``salt://_returners`` to the minion Sync returners from ``salt://_returners`` to the minion
saltenv saltenv
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than
@ -585,6 +585,44 @@ def sync_engines(saltenv=None, refresh=False, extmod_whitelist=None, extmod_blac
return ret return ret
def sync_thorium(saltenv=None, refresh=False, extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: Oxygen
Sync Thorium modules from ``salt://_thorium`` to the minion
saltenv
The fileserver environment from which to sync. To sync from more than
one environment, pass a comma-separated list.
If not passed, then all environments configured in the :ref:`top files
<states-top>` will be checked for engines to sync. If no top files are
found, then the ``base`` environment will be synced.
refresh: ``True``
If ``True``, refresh the available execution modules on the minion.
This refresh will be performed even if no new Thorium modules are synced.
Set to ``False`` to prevent this refresh.
extmod_whitelist
comma-seperated list of modules to sync
extmod_blacklist
comma-seperated list of modules to blacklist based on type
CLI Examples:
.. code-block:: bash
salt '*' saltutil.sync_thorium
salt '*' saltutil.sync_thorium saltenv=base,dev
'''
ret = _sync('thorium', saltenv, extmod_whitelist, extmod_blacklist)
if refresh:
refresh_modules()
return ret
def sync_output(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blacklist=None): def sync_output(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blacklist=None):
''' '''
Sync outputters from ``salt://_output`` to the minion Sync outputters from ``salt://_output`` to the minion
@ -628,7 +666,7 @@ def sync_clouds(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blackl
''' '''
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
Sync utility modules from ``salt://_cloud`` to the minion Sync cloud modules from ``salt://_cloud`` to the minion
saltenv : base saltenv : base
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than
@ -864,6 +902,7 @@ def sync_all(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blacklist
ret['log_handlers'] = sync_log_handlers(saltenv, False, extmod_whitelist, extmod_blacklist) ret['log_handlers'] = sync_log_handlers(saltenv, False, extmod_whitelist, extmod_blacklist)
ret['proxymodules'] = sync_proxymodules(saltenv, False, extmod_whitelist, extmod_blacklist) ret['proxymodules'] = sync_proxymodules(saltenv, False, extmod_whitelist, extmod_blacklist)
ret['engines'] = sync_engines(saltenv, False, extmod_whitelist, extmod_blacklist) ret['engines'] = sync_engines(saltenv, False, extmod_whitelist, extmod_blacklist)
ret['thorium'] = sync_thorium(saltenv, False, extmod_whitelist, extmod_blacklist)
if __opts__['file_client'] == 'local': if __opts__['file_client'] == 'local':
ret['pillar'] = sync_pillar(saltenv, False, extmod_whitelist, extmod_blacklist) ret['pillar'] = sync_pillar(saltenv, False, extmod_whitelist, extmod_blacklist)
if refresh: if refresh:

193
salt/modules/schedule.py
View File

@ -10,6 +10,7 @@ Module for managing the Salt schedule on a minion
from __future__ import absolute_import from __future__ import absolute_import
import copy as pycopy import copy as pycopy
import difflib import difflib
import logging
import os import os
import yaml import yaml
@ -23,7 +24,6 @@ from salt.ext import six
__proxyenabled__ = ['*'] __proxyenabled__ = ['*']
import logging
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
__func_alias__ = { __func_alias__ = {
@ -58,6 +58,7 @@ SCHEDULE_CONF = [
'return_config', 'return_config',
'return_kwargs', 'return_kwargs',
'run_on_start' 'run_on_start'
'skip_during_range',
] ]
@ -353,7 +354,7 @@ def build_schedule_item(name, **kwargs):
for item in ['range', 'when', 'once', 'once_fmt', 'cron', for item in ['range', 'when', 'once', 'once_fmt', 'cron',
'returner', 'after', 'return_config', 'return_kwargs', 'returner', 'after', 'return_config', 'return_kwargs',
'until', 'run_on_start']: 'until', 'run_on_start', 'skip_during_range']:
if item in kwargs: if item in kwargs:
schedule[name][item] = kwargs[item] schedule[name][item] = kwargs[item]
@ -951,3 +952,191 @@ def copy(name, target, **kwargs):
ret['minions'] = minions ret['minions'] = minions
return ret return ret
return ret return ret
def postpone_job(name, current_time, new_time, **kwargs):
'''
Postpone a job in the minion's schedule
Current time and new time should be specified as Unix timestamps
.. versionadded:: Oxygen
CLI Example:
.. code-block:: bash
salt '*' schedule.postpone_job job current_time new_time
'''
ret = {'comment': [],
'result': True}
if not name:
ret['comment'] = 'Job name is required.'
ret['result'] = False
return ret
if not current_time:
ret['comment'] = 'Job current time is required.'
ret['result'] = False
return ret
else:
if not isinstance(current_time, six.integer_types):
ret['comment'] = 'Job current time must be an integer.'
ret['result'] = False
return ret
if not new_time:
ret['comment'] = 'Job new_time is required.'
ret['result'] = False
return ret
else:
if not isinstance(new_time, six.integer_types):
ret['comment'] = 'Job new time must be an integer.'
ret['result'] = False
return ret
if 'test' in __opts__ and __opts__['test']:
ret['comment'] = 'Job: {0} would be postponed in schedule.'.format(name)
else:
if name in list_(show_all=True, where='opts', return_yaml=False):
event_data = {'name': name,
'time': current_time,
'new_time': new_time,
'func': 'postpone_job'}
elif name in list_(show_all=True, where='pillar', return_yaml=False):
event_data = {'name': name,
'time': current_time,
'new_time': new_time,
'where': 'pillar',
'func': 'postpone_job'}
else:
ret['comment'] = 'Job {0} does not exist.'.format(name)
ret['result'] = False
return ret
try:
eventer = salt.utils.event.get_event('minion', opts=__opts__)
res = __salt__['event.fire'](event_data, 'manage_schedule')
if res:
event_ret = eventer.get_event(tag='/salt/minion/minion_schedule_postpone_job_complete', wait=30)
if event_ret and event_ret['complete']:
schedule = event_ret['schedule']
# check item exists in schedule and is enabled
if name in schedule and schedule[name]['enabled']:
ret['result'] = True
ret['comment'] = 'Postponed Job {0} in schedule.'.format(name)
else:
ret['result'] = False
ret['comment'] = 'Failed to postpone job {0} in schedule.'.format(name)
return ret
except KeyError:
# Effectively a no-op, since we can't really return without an event system
ret['comment'] = 'Event module not available. Schedule postpone job failed.'
return ret
def skip_job(name, time, **kwargs):
'''
Skip a job in the minion's schedule at specified time.
Time to skip should be specified as Unix timestamps
.. versionadded:: Oxygen
CLI Example:
.. code-block:: bash
salt '*' schedule.skip_job job time
'''
ret = {'comment': [],
'result': True}
if not name:
ret['comment'] = 'Job name is required.'
ret['result'] = False
if not time:
ret['comment'] = 'Job time is required.'
ret['result'] = False
if 'test' in __opts__ and __opts__['test']:
ret['comment'] = 'Job: {0} would be skipped in schedule.'.format(name)
else:
if name in list_(show_all=True, where='opts', return_yaml=False):
event_data = {'name': name,
'time': time,
'func': 'skip_job'}
elif name in list_(show_all=True, where='pillar', return_yaml=False):
event_data = {'name': name,
'time': time,
'where': 'pillar',
'func': 'skip_job'}
else:
ret['comment'] = 'Job {0} does not exist.'.format(name)
ret['result'] = False
return ret
try:
eventer = salt.utils.event.get_event('minion', opts=__opts__)
res = __salt__['event.fire'](event_data, 'manage_schedule')
if res:
event_ret = eventer.get_event(tag='/salt/minion/minion_schedule_skip_job_complete', wait=30)
if event_ret and event_ret['complete']:
schedule = event_ret['schedule']
# check item exists in schedule and is enabled
if name in schedule and schedule[name]['enabled']:
ret['result'] = True
ret['comment'] = 'Added Skip Job {0} in schedule.'.format(name)
else:
ret['result'] = False
ret['comment'] = 'Failed to skip job {0} in schedule.'.format(name)
return ret
except KeyError:
# Effectively a no-op, since we can't really return without an event system
ret['comment'] = 'Event module not available. Schedule skip job failed.'
return ret
def show_next_fire_time(name, **kwargs):
'''
Show the next fire time for scheduled job
.. versionadded:: Oxygen
CLI Example:
.. code-block:: bash
salt '*' schedule.show_next_fire_time job_name
'''
ret = {'comment': [],
'result': True}
if not name:
ret['comment'] = 'Job name is required.'
ret['result'] = False
try:
event_data = {'name': name, 'func': 'get_next_fire_time'}
eventer = salt.utils.event.get_event('minion', opts=__opts__)
res = __salt__['event.fire'](event_data,
'manage_schedule')
if res:
event_ret = eventer.get_event(tag='/salt/minion/minion_schedule_next_fire_time_complete', wait=30)
except KeyError:
# Effectively a no-op, since we can't really return without an event system
ret = {}
ret['comment'] = 'Event module not available. Schedule show next fire time failed.'
ret['result'] = True
log.debug(ret['comment'])
return ret
return event_ret

94
salt/modules/state.py
View File

@ -43,6 +43,7 @@ from salt.runners.state import orchestrate as _orchestrate
# Import 3rd-party libs # Import 3rd-party libs
from salt.ext import six from salt.ext import six
import msgpack
__proxyenabled__ = ['*'] __proxyenabled__ = ['*']
@ -165,6 +166,99 @@ def _snapper_post(opts, jid, pre_num):
log.error('Failed to create snapper pre snapshot for jid: {0}'.format(jid)) log.error('Failed to create snapper pre snapshot for jid: {0}'.format(jid))
def pause(jid, state_id=None, duration=None):
'''
Set up a state id pause, this instructs a running state to pause at a given
state id. This needs to pass in the jid of the running state and can
optionally pass in a duration in seconds. If a state_id is not passed then
the jid referenced will be paused at the begining of the next state run.
The given state id is the id got a given state execution, so given a state
that looks like this:
.. code-block:: yaml
vim:
pkg.installed: []
The state_id to pass to `pause` is `vim`
CLI Examples:
.. code-block:: bash
salt '*' state.pause 20171130110407769519
salt '*' state.pause 20171130110407769519 vim
salt '*' state.pause 20171130110407769519 vim 20
'''
jid = str(jid)
if state_id is None:
state_id = '__all__'
pause_dir = os.path.join(__opts__[u'cachedir'], 'state_pause')
pause_path = os.path.join(pause_dir, jid)
if not os.path.exists(pause_dir):
try:
os.makedirs(pause_dir)
except OSError:
# File created in the gap
pass
data = {}
if os.path.exists(pause_path):
with salt.utils.files.fopen(pause_path, 'rb') as fp_:
data = msgpack.loads(fp_.read())
if state_id not in data:
data[state_id] = {}
if duration:
data[state_id]['duration'] = int(duration)
with salt.utils.files.fopen(pause_path, 'wb') as fp_:
fp_.write(msgpack.dumps(data))
def resume(jid, state_id=None):
'''
Remove a pause from a jid, allowing it to continue. If the state_id is
not specified then the a general pause will be resumed.
The given state_id is the id got a given state execution, so given a state
that looks like this:
.. code-block:: yaml
vim:
pkg.installed: []
The state_id to pass to `rm_pause` is `vim`
CLI Examples:
.. code-block:: bash
salt '*' state.resume 20171130110407769519
salt '*' state.resume 20171130110407769519 vim
'''
jid = str(jid)
if state_id is None:
state_id = '__all__'
pause_dir = os.path.join(__opts__[u'cachedir'], 'state_pause')
pause_path = os.path.join(pause_dir, jid)
if not os.path.exists(pause_dir):
try:
os.makedirs(pause_dir)
except OSError:
# File created in the gap
pass
data = {}
if os.path.exists(pause_path):
with salt.utils.files.fopen(pause_path, 'rb') as fp_:
data = msgpack.loads(fp_.read())
else:
return True
if state_id in data:
data.pop(state_id)
with salt.utils.files.fopen(pause_path, 'wb') as fp_:
fp_.write(msgpack.dumps(data))
def orchestrate(mods, def orchestrate(mods,
saltenv='base', saltenv='base',
test=None, test=None,

398
salt/modules/win_file.py
View File

@ -1218,41 +1218,55 @@ def mkdir(path,
owner=None, owner=None,
grant_perms=None, grant_perms=None,
deny_perms=None, deny_perms=None,
inheritance=True): inheritance=True,
reset=False):
''' '''
Ensure that the directory is available and permissions are set. Ensure that the directory is available and permissions are set.
Args: Args:
path (str): The full path to the directory. path (str):
The full path to the directory.
owner (str): The owner of the directory. If not passed, it will be the owner (str):
account that created the directory, likely SYSTEM The owner of the directory. If not passed, it will be the account
that created the directory, likely SYSTEM
grant_perms (dict): A dictionary containing the user/group and the basic grant_perms (dict):
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``. A dictionary containing the user/group and the basic permissions to
You can also set the ``applies_to`` setting here. The default is grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
``this_folder_subfolders_files``. Specify another ``applies_to`` setting set the ``applies_to`` setting here. The default is
like this: ``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
.. code-block:: yaml .. code-block:: yaml
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}} {'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
To set advanced permissions use a list for the ``perms`` parameter, ie: To set advanced permissions use a list for the ``perms`` parameter,
ie:
.. code-block:: yaml .. code-block:: yaml
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}} {'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and deny_perms (dict):
permissions to deny along with the ``applies_to`` setting. Use the same A dictionary containing the user/group and permissions to deny along
format used for the ``grant_perms`` parameter. Remember, deny with the ``applies_to`` setting. Use the same format used for the
permissions supersede grant permissions. ``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
inheritance (bool): If True the object will inherit permissions from the inheritance (bool):
parent, if False, inheritance will be disabled. Inheritance setting will If True the object will inherit permissions from the parent, if
not apply to parent directories if they must be created ``False``, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created.
reset (bool):
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Returns: Returns:
bool: True if successful bool: True if successful
@ -1289,10 +1303,16 @@ def mkdir(path,
# Set owner # Set owner
if owner: if owner:
salt.utils.win_dacl.set_owner(path, owner) salt.utils.win_dacl.set_owner(obj_name=path, principal=owner)
# Set permissions # Set permissions
set_perms(path, grant_perms, deny_perms, inheritance) set_perms(
path=path,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=inheritance,
reset=reset)
except WindowsError as exc: except WindowsError as exc:
raise CommandExecutionError(exc) raise CommandExecutionError(exc)
@ -1303,22 +1323,35 @@ def makedirs_(path,
owner=None, owner=None,
grant_perms=None, grant_perms=None,
deny_perms=None, deny_perms=None,
inheritance=True): inheritance=True,
reset=False):
''' '''
Ensure that the parent directory containing this path is available. Ensure that the parent directory containing this path is available.
Args: Args:
path (str): The full path to the directory. path (str):
The full path to the directory.
owner (str): The owner of the directory. If not passed, it will be the .. note::
account that created the directly, likely SYSTEM
grant_perms (dict): A dictionary containing the user/group and the basic The path must end with a trailing slash otherwise the
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``. directory(s) will be created up to the parent directory. For
You can also set the ``applies_to`` setting here. The default is example if path is ``C:\\temp\\test``, then it would be treated
``this_folder_subfolders_files``. Specify another ``applies_to`` setting as ``C:\\temp\\`` but if the path ends with a trailing slash
like this: like ``C:\\temp\\test\\``, then it would be treated as
``C:\\temp\\test\\``.
owner (str):
The owner of the directory. If not passed, it will be the account
that created the directly, likely SYSTEM
grant_perms (dict):
A dictionary containing the user/group and the basic permissions to
grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
.. code-block:: yaml .. code-block:: yaml
@ -1330,22 +1363,23 @@ def makedirs_(path,
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}} {'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and deny_perms (dict):
permissions to deny along with the ``applies_to`` setting. Use the same A dictionary containing the user/group and permissions to deny along
format used for the ``grant_perms`` parameter. Remember, deny with the ``applies_to`` setting. Use the same format used for the
permissions supersede grant permissions. ``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
inheritance (bool): If True the object will inherit permissions from the inheritance (bool):
parent, if False, inheritance will be disabled. Inheritance setting will If True the object will inherit permissions from the parent, if
not apply to parent directories if they must be created False, inheritance will be disabled. Inheritance setting will not
apply to parent directories if they must be created.
.. note:: reset (bool):
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
The path must end with a trailing slash otherwise the directory(s) will .. versionadded:: Oxygen
be created up to the parent directory. For example if path is
``C:\\temp\\test``, then it would be treated as ``C:\\temp\\`` but if
the path ends with a trailing slash like ``C:\\temp\\test\\``, then it
would be treated as ``C:\\temp\\test\\``.
Returns: Returns:
bool: True if successful bool: True if successful
@ -1405,7 +1439,13 @@ def makedirs_(path,
for directory_to_create in directories_to_create: for directory_to_create in directories_to_create:
# all directories have the user, group and mode set!! # all directories have the user, group and mode set!!
log.debug('Creating directory: %s', directory_to_create) log.debug('Creating directory: %s', directory_to_create)
mkdir(directory_to_create, owner, grant_perms, deny_perms, inheritance) mkdir(
path=directory_to_create,
owner=owner,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=inheritance,
reset=reset)
return True return True
@ -1414,22 +1454,26 @@ def makedirs_perms(path,
owner=None, owner=None,
grant_perms=None, grant_perms=None,
deny_perms=None, deny_perms=None,
inheritance=True): inheritance=True,
reset=True):
''' '''
Set owner and permissions for each directory created. Set owner and permissions for each directory created.
Args: Args:
path (str): The full path to the directory. path (str):
The full path to the directory.
owner (str): The owner of the directory. If not passed, it will be the owner (str):
account that created the directory, likely SYSTEM The owner of the directory. If not passed, it will be the account
that created the directory, likely SYSTEM
grant_perms (dict): A dictionary containing the user/group and the basic grant_perms (dict):
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``. A dictionary containing the user/group and the basic permissions to
You can also set the ``applies_to`` setting here. The default is grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
``this_folder_subfolders_files``. Specify another ``applies_to`` setting set the ``applies_to`` setting here. The default is
like this: ``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
.. code-block:: yaml .. code-block:: yaml
@ -1441,15 +1485,24 @@ def makedirs_perms(path,
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}} {'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and deny_perms (dict):
permissions to deny along with the ``applies_to`` setting. Use the same A dictionary containing the user/group and permissions to deny along
format used for the ``grant_perms`` parameter. Remember, deny with the ``applies_to`` setting. Use the same format used for the
permissions supersede grant permissions. ``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
inheritance (bool): If True the object will inherit permissions from the inheritance (bool):
parent, if False, inheritance will be disabled. Inheritance setting will If ``True`` the object will inherit permissions from the parent, if
``False``, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created not apply to parent directories if they must be created
reset (bool):
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Returns: Returns:
bool: True if successful, otherwise raise an error bool: True if successful, otherwise raise an error
@ -1482,8 +1535,15 @@ def makedirs_perms(path,
try: try:
# Create the directory here, set inherited True because this is a # Create the directory here, set inherited True because this is a
# parent directory, the inheritance setting will only apply to the # parent directory, the inheritance setting will only apply to the
# child directory # target directory. Reset will be False as we only want to reset
makedirs_perms(head, owner, grant_perms, deny_perms, True) # the permissions on the target directory
makedirs_perms(
path=head,
owner=owner,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=True,
reset=False)
except OSError as exc: except OSError as exc:
# be happy if someone already created the path # be happy if someone already created the path
if exc.errno != errno.EEXIST: if exc.errno != errno.EEXIST:
@ -1492,7 +1552,13 @@ def makedirs_perms(path,
return {} return {}
# Make the directory # Make the directory
mkdir(path, owner, grant_perms, deny_perms, inheritance) mkdir(
path=path,
owner=owner,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=inheritance,
reset=reset)
return True return True
@ -1502,66 +1568,64 @@ def check_perms(path,
owner=None, owner=None,
grant_perms=None, grant_perms=None,
deny_perms=None, deny_perms=None,
inheritance=True): inheritance=True,
reset=False):
''' '''
Set owner and permissions for each directory created. Check owner and permissions for the passed directory. This function checks
the permissions and sets them, returning the changes made.
Args: Args:
path (str): The full path to the directory. path (str):
The full path to the directory.
ret (dict): A dictionary to append changes to and return. If not passed, ret (dict):
will create a new dictionary to return. A dictionary to append changes to and return. If not passed, will
create a new dictionary to return.
owner (str): The owner of the directory. If not passed, it will be the owner (str):
account that created the directory, likely SYSTEM The owner to set for the directory.
grant_perms (dict): A dictionary containing the user/group and the basic grant_perms (dict):
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``. A dictionary containing the user/group and the basic permissions to
You can also set the ``applies_to`` setting here. The default is check/grant, ie: ``{'user': {'perms': 'basic_permission'}}``.
``this_folder_subfolders_files``. Specify another ``applies_to`` setting Default is ``None``.
like this:
.. code-block:: yaml deny_perms (dict):
A dictionary containing the user/group and permissions to
check/deny. Default is ``None``.
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}} inheritance (bool):
``True will check if inheritance is enabled and enable it. ``False``
will check if inheritance is disabled and disable it. Defaultl is
``True``.
To set advanced permissions use a list for the ``perms`` parameter, ie: reset (bool):
``True`` wil show what permisisons will be removed by resetting the
.. code-block:: yaml DACL. ``False`` will do nothing. Default is ``False``.
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and
permissions to deny along with the ``applies_to`` setting. Use the same
format used for the ``grant_perms`` parameter. Remember, deny
permissions supersede grant permissions.
inheritance (bool): If True the object will inherit permissions from the
parent, if False, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created
Returns: Returns:
bool: True if successful, otherwise raise an error dict: A dictionary of changes that have been made
CLI Example: CLI Example:
.. code-block:: bash .. code-block:: bash
# To grant the 'Users' group 'read & execute' permissions. # To see changes to ``C:\\Temp`` if the 'Users' group is given 'read & execute' permissions.
salt '*' file.check_perms C:\\Temp\\ Administrators "{'Users': {'perms': 'read_execute'}}" salt '*' file.check_perms C:\\Temp\\ {} Administrators "{'Users': {'perms': 'read_execute'}}"
# Locally using salt call # Locally using salt call
salt-call file.check_perms C:\\Temp\\ Administrators "{'Users': {'perms': 'read_execute', 'applies_to': 'this_folder_only'}}" salt-call file.check_perms C:\\Temp\\ {} Administrators "{'Users': {'perms': 'read_execute', 'applies_to': 'this_folder_only'}}"
# Specify advanced attributes with a list # Specify advanced attributes with a list
salt '*' file.check_perms C:\\Temp\\ Administrators "{'jsnuffy': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'files_only'}}" salt '*' file.check_perms C:\\Temp\\ {} Administrators "{'jsnuffy': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'files_only'}}"
''' '''
path = os.path.expanduser(path) path = os.path.expanduser(path)
if not ret: if not ret:
ret = {'name': path, ret = {'name': path,
'changes': {}, 'changes': {},
'pchanges': {},
'comment': [], 'comment': [],
'result': True} 'result': True}
orig_comment = '' orig_comment = ''
@ -1571,14 +1635,16 @@ def check_perms(path,
# Check owner # Check owner
if owner: if owner:
owner = salt.utils.win_dacl.get_name(owner) owner = salt.utils.win_dacl.get_name(principal=owner)
current_owner = salt.utils.win_dacl.get_owner(path) current_owner = salt.utils.win_dacl.get_owner(obj_name=path)
if owner != current_owner: if owner != current_owner:
if __opts__['test'] is True: if __opts__['test'] is True:
ret['pchanges']['owner'] = owner ret['pchanges']['owner'] = owner
else: else:
try: try:
salt.utils.win_dacl.set_owner(path, owner) salt.utils.win_dacl.set_owner(
obj_name=path,
principal=owner)
ret['changes']['owner'] = owner ret['changes']['owner'] = owner
except CommandExecutionError: except CommandExecutionError:
ret['result'] = False ret['result'] = False
@ -1586,7 +1652,7 @@ def check_perms(path,
'Failed to change owner to "{0}"'.format(owner)) 'Failed to change owner to "{0}"'.format(owner))
# Check permissions # Check permissions
cur_perms = salt.utils.win_dacl.get_permissions(path) cur_perms = salt.utils.win_dacl.get_permissions(obj_name=path)
# Verify Deny Permissions # Verify Deny Permissions
changes = {} changes = {}
@ -1594,7 +1660,7 @@ def check_perms(path,
for user in deny_perms: for user in deny_perms:
# Check that user exists: # Check that user exists:
try: try:
user_name = salt.utils.win_dacl.get_name(user) user_name = salt.utils.win_dacl.get_name(principal=user)
except CommandExecutionError: except CommandExecutionError:
ret['comment'].append( ret['comment'].append(
'Deny Perms: User "{0}" missing from Target System'.format(user)) 'Deny Perms: User "{0}" missing from Target System'.format(user))
@ -1619,7 +1685,11 @@ def check_perms(path,
# Check Perms # Check Perms
if isinstance(deny_perms[user]['perms'], six.string_types): if isinstance(deny_perms[user]['perms'], six.string_types):
if not salt.utils.win_dacl.has_permission( if not salt.utils.win_dacl.has_permission(
path, user, deny_perms[user]['perms'], 'deny'): obj_name=path,
principal=user,
permission=deny_perms[user]['perms'],
access_mode='deny',
exact=False):
changes[user] = {'perms': deny_perms[user]['perms']} changes[user] = {'perms': deny_perms[user]['perms']}
else: else:
for perm in deny_perms[user]['perms']: for perm in deny_perms[user]['perms']:
@ -1640,9 +1710,10 @@ def check_perms(path,
changes[user]['applies_to'] = applies_to changes[user]['applies_to'] = applies_to
if changes: if changes:
ret['pchanges']['deny_perms'] = {}
ret['changes']['deny_perms'] = {} ret['changes']['deny_perms'] = {}
for user in changes: for user in changes:
user_name = salt.utils.win_dacl.get_name(user) user_name = salt.utils.win_dacl.get_name(principal=user)
if __opts__['test'] is True: if __opts__['test'] is True:
ret['pchanges']['deny_perms'][user] = changes[user] ret['pchanges']['deny_perms'][user] = changes[user]
@ -1689,7 +1760,11 @@ def check_perms(path,
try: try:
salt.utils.win_dacl.set_permissions( salt.utils.win_dacl.set_permissions(
path, user, perms, 'deny', applies_to) obj_name=path,
principal=user,
permissions=perms,
access_mode='deny',
applies_to=applies_to)
ret['changes']['deny_perms'][user] = changes[user] ret['changes']['deny_perms'][user] = changes[user]
except CommandExecutionError: except CommandExecutionError:
ret['result'] = False ret['result'] = False
@ -1703,7 +1778,7 @@ def check_perms(path,
for user in grant_perms: for user in grant_perms:
# Check that user exists: # Check that user exists:
try: try:
user_name = salt.utils.win_dacl.get_name(user) user_name = salt.utils.win_dacl.get_name(principal=user)
except CommandExecutionError: except CommandExecutionError:
ret['comment'].append( ret['comment'].append(
'Grant Perms: User "{0}" missing from Target System'.format(user)) 'Grant Perms: User "{0}" missing from Target System'.format(user))
@ -1729,12 +1804,19 @@ def check_perms(path,
# Check Perms # Check Perms
if isinstance(grant_perms[user]['perms'], six.string_types): if isinstance(grant_perms[user]['perms'], six.string_types):
if not salt.utils.win_dacl.has_permission( if not salt.utils.win_dacl.has_permission(
path, user, grant_perms[user]['perms']): obj_name=path,
principal=user,
permission=grant_perms[user]['perms'],
access_mode='grant'):
changes[user] = {'perms': grant_perms[user]['perms']} changes[user] = {'perms': grant_perms[user]['perms']}
else: else:
for perm in grant_perms[user]['perms']: for perm in grant_perms[user]['perms']:
if not salt.utils.win_dacl.has_permission( if not salt.utils.win_dacl.has_permission(
path, user, perm, exact=False): obj_name=path,
principal=user,
permission=perm,
access_mode='grant',
exact=False):
if user not in changes: if user not in changes:
changes[user] = {'perms': []} changes[user] = {'perms': []}
changes[user]['perms'].append(grant_perms[user]['perms']) changes[user]['perms'].append(grant_perms[user]['perms'])
@ -1750,11 +1832,12 @@ def check_perms(path,
changes[user]['applies_to'] = applies_to changes[user]['applies_to'] = applies_to
if changes: if changes:
ret['pchanges']['grant_perms'] = {}
ret['changes']['grant_perms'] = {} ret['changes']['grant_perms'] = {}
for user in changes: for user in changes:
user_name = salt.utils.win_dacl.get_name(user) user_name = salt.utils.win_dacl.get_name(principal=user)
if __opts__['test'] is True: if __opts__['test'] is True:
ret['changes']['grant_perms'][user] = changes[user] ret['pchanges']['grant_perms'][user] = changes[user]
else: else:
applies_to = None applies_to = None
if 'applies_to' not in changes[user]: if 'applies_to' not in changes[user]:
@ -1796,7 +1879,11 @@ def check_perms(path,
try: try:
salt.utils.win_dacl.set_permissions( salt.utils.win_dacl.set_permissions(
path, user, perms, 'grant', applies_to) obj_name=path,
principal=user,
permissions=perms,
access_mode='grant',
applies_to=applies_to)
ret['changes']['grant_perms'][user] = changes[user] ret['changes']['grant_perms'][user] = changes[user]
except CommandExecutionError: except CommandExecutionError:
ret['result'] = False ret['result'] = False
@ -1806,12 +1893,14 @@ def check_perms(path,
# Check inheritance # Check inheritance
if inheritance is not None: if inheritance is not None:
if not inheritance == salt.utils.win_dacl.get_inheritance(path): if not inheritance == salt.utils.win_dacl.get_inheritance(obj_name=path):
if __opts__['test'] is True: if __opts__['test'] is True:
ret['changes']['inheritance'] = inheritance ret['pchanges']['inheritance'] = inheritance
else: else:
try: try:
salt.utils.win_dacl.set_inheritance(path, inheritance) salt.utils.win_dacl.set_inheritance(
obj_name=path,
enabled=inheritance)
ret['changes']['inheritance'] = inheritance ret['changes']['inheritance'] = inheritance
except CommandExecutionError: except CommandExecutionError:
ret['result'] = False ret['result'] = False
@ -1819,6 +1908,45 @@ def check_perms(path,
'Failed to set inheritance for "{0}" to ' 'Failed to set inheritance for "{0}" to '
'{1}'.format(path, inheritance)) '{1}'.format(path, inheritance))
# Check reset
# If reset=True, which users will be removed as a result
if reset:
for user_name in cur_perms:
if user_name not in grant_perms:
if 'grant' in cur_perms[user_name] and not \
cur_perms[user_name]['grant']['inherited']:
if __opts__['test'] is True:
if 'remove_perms' not in ret['pchanges']:
ret['pchanges']['remove_perms'] = {}
ret['pchanges']['remove_perms'].update(
{user_name: cur_perms[user_name]})
else:
if 'remove_perms' not in ret['changes']:
ret['changes']['remove_perms'] = {}
salt.utils.win_dacl.rm_permissions(
obj_name=path,
principal=user_name,
ace_type='grant')
ret['changes']['remove_perms'].update(
{user_name: cur_perms[user_name]})
if user_name not in deny_perms:
if 'deny' in cur_perms[user_name] and not \
cur_perms[user_name]['deny']['inherited']:
if __opts__['test'] is True:
if 'remove_perms' not in ret['pchanges']:
ret['pchanges']['remove_perms'] = {}
ret['pchanges']['remove_perms'].update(
{user_name: cur_perms[user_name]})
else:
if 'remove_perms' not in ret['changes']:
ret['changes']['remove_perms'] = {}
salt.utils.win_dacl.rm_permissions(
obj_name=path,
principal=user_name,
ace_type='deny')
ret['changes']['remove_perms'].update(
{user_name: cur_perms[user_name]})
# Re-add the Original Comment if defined # Re-add the Original Comment if defined
if isinstance(orig_comment, six.string_types): if isinstance(orig_comment, six.string_types):
if orig_comment: if orig_comment:
@ -1830,25 +1958,30 @@ def check_perms(path,
ret['comment'] = '\n'.join(ret['comment']) ret['comment'] = '\n'.join(ret['comment'])
# Set result for test = True # Set result for test = True
if __opts__['test'] is True and ret['changes']: if __opts__['test'] and (ret['changes'] or ret['pchanges']):
ret['result'] = None ret['result'] = None
return ret return ret
def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True): def set_perms(path,
grant_perms=None,
deny_perms=None,
inheritance=True,
reset=False):
''' '''
Set permissions for the given path Set permissions for the given path
Args: Args:
path (str): The full path to the directory. path (str):
The full path to the directory.
grant_perms (dict): grant_perms (dict):
A dictionary containing the user/group and the basic permissions to A dictionary containing the user/group and the basic permissions to
grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
set the ``applies_to`` setting here. The default is set the ``applies_to`` setting here. The default for ``applise_to``
``this_folder_subfolders_files``. Specify another ``applies_to`` is ``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this: setting like this:
.. code-block:: yaml .. code-block:: yaml
@ -1863,7 +1996,10 @@ def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}} {'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
To see a list of available attributes and applies to settings see To see a list of available attributes and applies to settings see
the documentation for salt.utils.win_dacl the documentation for salt.utils.win_dacl.
A value of ``None`` will make no changes to the ``grant`` portion of
the DACL. Default is ``None``.
deny_perms (dict): deny_perms (dict):
A dictionary containing the user/group and permissions to deny along A dictionary containing the user/group and permissions to deny along
@ -1871,13 +2007,27 @@ def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
``grant_perms`` parameter. Remember, deny permissions supersede ``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions. grant permissions.
A value of ``None`` will make no changes to the ``deny`` portion of
the DACL. Default is ``None``.
inheritance (bool): inheritance (bool):
If True the object will inherit permissions from the parent, if If ``True`` the object will inherit permissions from the parent, if
False, inheritance will be disabled. Inheritance setting will not ``False``, inheritance will be disabled. Inheritance setting will
apply to parent directories if they must be created not apply to parent directories if they must be created. Default is
``False``.
reset (bool):
If ``True`` the existing DCL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded: Oxygen
Returns: Returns:
bool: True if successful, otherwise raise an error bool: True if successful
Raises:
CommandExecutionError: If unsuccessful
CLI Example: CLI Example:
@ -1894,6 +2044,14 @@ def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
''' '''
ret = {} ret = {}
if reset:
# Get an empty DACL
dacl = salt.utils.win_dacl.dacl()
# Get an empty perms dict
cur_perms = {}
else:
# Get the DACL for the directory # Get the DACL for the directory
dacl = salt.utils.win_dacl.dacl(path) dacl = salt.utils.win_dacl.dacl(path)

14
salt/modules/yumpkg.py
View File

@ -279,11 +279,23 @@ def _get_extra_options(**kwargs):
''' '''
ret = [] ret = []
kwargs = salt.utils.args.clean_kwargs(**kwargs) kwargs = salt.utils.args.clean_kwargs(**kwargs)
# Remove already handled options from kwargs
fromrepo = kwargs.pop('fromrepo', '')
repo = kwargs.pop('repo', '')
disablerepo = kwargs.pop('disablerepo', '')
enablerepo = kwargs.pop('enablerepo', '')
disable_excludes = kwargs.pop('disableexcludes', '')
branch = kwargs.pop('branch', '')
for key, value in six.iteritems(kwargs): for key, value in six.iteritems(kwargs):
if isinstance(key, six.string_types): if isinstance(value, six.string_types):
log.info('Adding extra option --%s=\'%s\'', key, value)
ret.append('--{0}=\'{1}\''.format(key, value)) ret.append('--{0}=\'{1}\''.format(key, value))
elif value is True: elif value is True:
log.info('Adding extra option --%s', key)
ret.append('--{0}'.format(key)) ret.append('--{0}'.format(key))
log.info('Adding extra options %s', ret)
return ret return ret

4
salt/returners/local_cache.py
View File

@ -294,9 +294,11 @@ def get_load(jid):
if not os.path.exists(jid_dir) or not os.path.exists(load_fn): if not os.path.exists(jid_dir) or not os.path.exists(load_fn):
return {} return {}
serial = salt.payload.Serial(__opts__) serial = salt.payload.Serial(__opts__)
ret = {}
with salt.utils.files.fopen(os.path.join(jid_dir, LOAD_P), 'rb') as rfh: with salt.utils.files.fopen(os.path.join(jid_dir, LOAD_P), 'rb') as rfh:
ret = serial.load(rfh) ret = serial.load(rfh)
if ret is None:
ret = {}
minions_cache = [os.path.join(jid_dir, MINIONS_P)] minions_cache = [os.path.join(jid_dir, MINIONS_P)]
minions_cache.extend( minions_cache.extend(
glob.glob(os.path.join(jid_dir, SYNDIC_MINIONS_P.format('*'))) glob.glob(os.path.join(jid_dir, SYNDIC_MINIONS_P.format('*')))

7
salt/runner.py
View File

@ -43,6 +43,7 @@ class RunnerClient(mixins.SyncClientMixin, mixins.AsyncClientMixin, object):
def __init__(self, opts): def __init__(self, opts):
self.opts = opts self.opts = opts
self.context = {}
@property @property
def functions(self): def functions(self):
@ -51,11 +52,13 @@ class RunnerClient(mixins.SyncClientMixin, mixins.AsyncClientMixin, object):
self.utils = salt.loader.utils(self.opts) self.utils = salt.loader.utils(self.opts)
# Must be self.functions for mixin to work correctly :-/ # Must be self.functions for mixin to work correctly :-/
try: try:
self._functions = salt.loader.runner(self.opts, utils=self.utils) self._functions = salt.loader.runner(
self.opts, utils=self.utils, context=self.context)
except AttributeError: except AttributeError:
# Just in case self.utils is still not present (perhaps due to # Just in case self.utils is still not present (perhaps due to
# problems with the loader), load the runner funcs without them # problems with the loader), load the runner funcs without them
self._functions = salt.loader.runner(self.opts) self._functions = salt.loader.runner(
self.opts, context=self.context)
return self._functions return self._functions

37
salt/runners/saltutil.py
View File

@ -52,6 +52,7 @@ def sync_all(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
ret['runners'] = sync_runners(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist) ret['runners'] = sync_runners(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['wheel'] = sync_wheel(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist) ret['wheel'] = sync_wheel(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['engines'] = sync_engines(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist) ret['engines'] = sync_engines(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['thorium'] = sync_thorium(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['queues'] = sync_queues(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist) ret['queues'] = sync_queues(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['pillar'] = sync_pillar(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist) ret['pillar'] = sync_pillar(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['utils'] = sync_utils(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist) ret['utils'] = sync_utils(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
@ -303,6 +304,32 @@ def sync_engines(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
extmod_blacklist=extmod_blacklist)[0] extmod_blacklist=extmod_blacklist)[0]
def sync_thorium(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: Oxygen
Sync Thorium from ``salt://_thorium`` to the master
saltenv: ``base``
The fileserver environment from which to sync. To sync from more than
one environment, pass a comma-separated list.
extmod_whitelist
comma-seperated list of modules to sync
extmod_blacklist
comma-seperated list of modules to blacklist based on type
CLI Example:
.. code-block:: bash
salt-run saltutil.sync_thorium
'''
return salt.utils.extmods.sync(__opts__, 'thorium', saltenv=saltenv, extmod_whitelist=extmod_whitelist,
extmod_blacklist=extmod_blacklist)[0]
def sync_queues(saltenv='base', extmod_whitelist=None, extmod_blacklist=None): def sync_queues(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
''' '''
Sync queue modules from ``salt://_queues`` to the master Sync queue modules from ``salt://_queues`` to the master
@ -381,7 +408,7 @@ def sync_sdb(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
''' '''
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
Sync utils modules from ``salt://_sdb`` to the master Sync sdb modules from ``salt://_sdb`` to the master
saltenv : base saltenv : base
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than
@ -427,7 +454,7 @@ def sync_cache(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
''' '''
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
Sync utils modules from ``salt://_cache`` to the master Sync cache modules from ``salt://_cache`` to the master
saltenv : base saltenv : base
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than
@ -453,7 +480,7 @@ def sync_fileserver(saltenv='base', extmod_whitelist=None, extmod_blacklist=None
''' '''
.. versionadded:: Oxygen .. versionadded:: Oxygen
Sync utils modules from ``salt://_fileserver`` to the master Sync fileserver modules from ``salt://_fileserver`` to the master
saltenv : base saltenv : base
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than
@ -479,7 +506,7 @@ def sync_clouds(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
''' '''
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
Sync utils modules from ``salt://_clouds`` to the master Sync cloud modules from ``salt://_clouds`` to the master
saltenv : base saltenv : base
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than
@ -505,7 +532,7 @@ def sync_roster(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
''' '''
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
Sync utils modules from ``salt://_roster`` to the master Sync roster modules from ``salt://_roster`` to the master
saltenv : base saltenv : base
The fileserver environment from which to sync. To sync from more than The fileserver environment from which to sync. To sync from more than

6
salt/runners/sdb.py
View File

@ -22,7 +22,7 @@ def get(uri):
.. code-block:: bash .. code-block:: bash
salt '*' sdb.get sdb://mymemcached/foo salt-run sdb.get sdb://mymemcached/foo
''' '''
return salt.utils.sdb.sdb_get(uri, __opts__, __utils__) return salt.utils.sdb.sdb_get(uri, __opts__, __utils__)
@ -37,7 +37,7 @@ def set_(uri, value):
.. code-block:: bash .. code-block:: bash
salt '*' sdb.set sdb://mymemcached/foo bar salt-run sdb.set sdb://mymemcached/foo bar
''' '''
return salt.utils.sdb.sdb_set(uri, value, __opts__, __utils__) return salt.utils.sdb.sdb_set(uri, value, __opts__, __utils__)
@ -52,7 +52,7 @@ def delete(uri):
.. code-block:: bash .. code-block:: bash
salt '*' sdb.delete sdb://mymemcached/foo salt-run sdb.delete sdb://mymemcached/foo
''' '''
return salt.utils.sdb.sdb_delete(uri, __opts__, __utils__) return salt.utils.sdb.sdb_delete(uri, __opts__, __utils__)

18
salt/runners/state.py
View File

@ -15,6 +15,24 @@ from salt.exceptions import SaltInvocationError
LOGGER = logging.getLogger(__name__) LOGGER = logging.getLogger(__name__)
def set_pause(jid, state_id, duration=None):
'''
Set up a state id pause, this instructs a running state to pause at a given
state id. This needs to pass in the jid of the running state and can
optionally pass in a duration in seconds.
'''
minion = salt.minion.MasterMinion(__opts__)
minion['state.set_pause'](jid, state_id, duration)
def rm_pause(jid, state_id, duration=None):
'''
Remove a pause from a jid, allowing it to continue
'''
minion = salt.minion.MasterMinion(__opts__)
minion['state.rm_pause'](jid, state_id)
def orchestrate(mods, def orchestrate(mods,
saltenv='base', saltenv='base',
test=None, test=None,

52
salt/state.py
View File

@ -1918,6 +1918,8 @@ class State(object):
if self.mocked: if self.mocked:
ret = mock_ret(cdata) ret = mock_ret(cdata)
else: else:
# Check if this low chunk is paused
self.check_pause(low)
# Execute the state function # Execute the state function
if not low.get(u'__prereq__') and low.get(u'parallel'): if not low.get(u'__prereq__') and low.get(u'parallel'):
# run the state call in parallel, but only if not in a prereq # run the state call in parallel, but only if not in a prereq
@ -2127,6 +2129,48 @@ class State(object):
return not running[tag][u'result'] return not running[tag][u'result']
return False return False
def check_pause(self, low):
'''
Check to see if this low chunk has been paused
'''
if not self.jid:
# Can't pause on salt-ssh since we can't track continuous state
return
pause_path = os.path.join(self.opts[u'cachedir'], 'state_pause', self.jid)
start = time.time()
if os.path.isfile(pause_path):
try:
while True:
tries = 0
with salt.utils.files.fopen(pause_path, 'rb') as fp_:
try:
pdat = msgpack.loads(fp_.read())
except msgpack.UnpackValueError:
# Reading race condition
if tries > 10:
# Break out if there are a ton of read errors
return
tries += 1
time.sleep(1)
continue
id_ = low[u'__id__']
key = u''
if id_ in pdat:
key = id_
elif u'__all__' in pdat:
key = u'__all__'
if key:
if u'duration' in pdat[key]:
now = time.time()
if now - start > pdat[key][u'duration']:
return
else:
return
time.sleep(1)
except Exception as exc:
log.error('Failed to read in pause data for file located at: %s', pause_path)
return
def reconcile_procs(self, running): def reconcile_procs(self, running):
''' '''
Check the running dict for processes and resolve them Check the running dict for processes and resolve them
@ -2682,6 +2726,14 @@ class State(object):
except OSError: except OSError:
log.debug(u'File %s does not exist, no need to cleanup', accum_data_path) log.debug(u'File %s does not exist, no need to cleanup', accum_data_path)
_cleanup_accumulator_data() _cleanup_accumulator_data()
if self.jid is not None:
pause_path = os.path.join(self.opts[u'cachedir'], u'state_pause', self.jid)
if os.path.isfile(pause_path):
try:
os.remove(pause_path)
except OSError:
# File is not present, all is well
pass
return ret return ret

127
salt/states/file.py
View File

@ -761,7 +761,8 @@ def _check_directory_win(name,
win_owner, win_owner,
win_perms=None, win_perms=None,
win_deny_perms=None, win_deny_perms=None,
win_inheritance=None): win_inheritance=None,
win_perms_reset=None):
''' '''
Check what changes need to be made on a directory Check what changes need to be made on a directory
''' '''
@ -879,6 +880,20 @@ def _check_directory_win(name,
if not win_inheritance == salt.utils.win_dacl.get_inheritance(name): if not win_inheritance == salt.utils.win_dacl.get_inheritance(name):
changes['inheritance'] = win_inheritance changes['inheritance'] = win_inheritance
# Check reset
if win_perms_reset:
for user_name in perms:
if user_name not in win_perms:
if 'grant' in perms[user_name] and not perms[user_name]['grant']['inherited']:
if 'remove_perms' not in changes:
changes['remove_perms'] = {}
changes['remove_perms'].update({user_name: perms[user_name]})
if user_name not in win_deny_perms:
if 'deny' in perms[user_name] and not perms[user_name]['deny']['inherited']:
if 'remove_perms' not in changes:
changes['remove_perms'] = {}
changes['remove_perms'].update({user_name: perms[user_name]})
if changes: if changes:
return None, 'The directory "{0}" will be changed'.format(name), changes return None, 'The directory "{0}" will be changed'.format(name), changes
@ -1566,6 +1581,7 @@ def managed(name,
win_perms=None, win_perms=None,
win_deny_perms=None, win_deny_perms=None,
win_inheritance=True, win_inheritance=True,
win_perms_reset=False,
**kwargs): **kwargs):
r''' r'''
Manage a given file, this function allows for a file to be downloaded from Manage a given file, this function allows for a file to be downloaded from
@ -2072,6 +2088,13 @@ def managed(name,
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
win_perms_reset : False
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Here's an example using the above ``win_*`` parameters: Here's an example using the above ``win_*`` parameters:
.. code-block:: yaml .. code-block:: yaml
@ -2314,8 +2337,13 @@ def managed(name,
# Check and set the permissions if necessary # Check and set the permissions if necessary
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms']( ret = __salt__['file.check_perms'](
name, ret, win_owner, win_perms, win_deny_perms, None, path=name,
win_inheritance) ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else: else:
ret, _ = __salt__['file.check_perms']( ret, _ = __salt__['file.check_perms'](
name, ret, user, group, mode, attrs, follow_symlinks) name, ret, user, group, mode, attrs, follow_symlinks)
@ -2356,8 +2384,13 @@ def managed(name,
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms']( ret = __salt__['file.check_perms'](
name, ret, win_owner, win_perms, win_deny_perms, None, path=name,
win_inheritance) ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
if isinstance(ret['pchanges'], tuple): if isinstance(ret['pchanges'], tuple):
ret['result'], ret['comment'] = ret['pchanges'] ret['result'], ret['comment'] = ret['pchanges']
@ -2448,6 +2481,7 @@ def managed(name,
win_perms=win_perms, win_perms=win_perms,
win_deny_perms=win_deny_perms, win_deny_perms=win_deny_perms,
win_inheritance=win_inheritance, win_inheritance=win_inheritance,
win_perms_reset=win_perms_reset,
encoding=encoding, encoding=encoding,
encoding_errors=encoding_errors, encoding_errors=encoding_errors,
**kwargs) **kwargs)
@ -2517,6 +2551,7 @@ def managed(name,
win_perms=win_perms, win_perms=win_perms,
win_deny_perms=win_deny_perms, win_deny_perms=win_deny_perms,
win_inheritance=win_inheritance, win_inheritance=win_inheritance,
win_perms_reset=win_perms_reset,
encoding=encoding, encoding=encoding,
encoding_errors=encoding_errors, encoding_errors=encoding_errors,
**kwargs) **kwargs)
@ -2590,6 +2625,7 @@ def directory(name,
win_perms=None, win_perms=None,
win_deny_perms=None, win_deny_perms=None,
win_inheritance=True, win_inheritance=True,
win_perms_reset=False,
**kwargs): **kwargs):
r''' r'''
Ensure that a named directory is present and has the right perms Ensure that a named directory is present and has the right perms
@ -2751,6 +2787,13 @@ def directory(name,
.. versionadded:: 2017.7.0 .. versionadded:: 2017.7.0
win_perms_reset : False
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Here's an example using the above ``win_*`` parameters: Here's an example using the above ``win_*`` parameters:
.. code-block:: yaml .. code-block:: yaml
@ -2855,13 +2898,23 @@ def directory(name,
elif force: elif force:
# Remove whatever is in the way # Remove whatever is in the way
if os.path.isfile(name): if os.path.isfile(name):
if __opts__['test']:
ret['pchanges']['forced'] = 'File was forcibly replaced'
else:
os.remove(name) os.remove(name)
ret['changes']['forced'] = 'File was forcibly replaced' ret['changes']['forced'] = 'File was forcibly replaced'
elif __salt__['file.is_link'](name): elif __salt__['file.is_link'](name):
if __opts__['test']:
ret['pchanges']['forced'] = 'Symlink was forcibly replaced'
else:
__salt__['file.remove'](name) __salt__['file.remove'](name)
ret['changes']['forced'] = 'Symlink was forcibly replaced' ret['changes']['forced'] = 'Symlink was forcibly replaced'
else:
if __opts__['test']:
ret['pchanges']['forced'] = 'Directory was forcibly replaced'
else: else:
__salt__['file.remove'](name) __salt__['file.remove'](name)
ret['changes']['forced'] = 'Directory was forcibly replaced'
else: else:
if os.path.isfile(name): if os.path.isfile(name):
return _error( return _error(
@ -2874,17 +2927,26 @@ def directory(name,
# Check directory? # Check directory?
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
presult, pcomment, ret['pchanges'] = _check_directory_win( presult, pcomment, pchanges = _check_directory_win(
name, win_owner, win_perms, win_deny_perms, win_inheritance) name=name,
win_owner=win_owner,
win_perms=win_perms,
win_deny_perms=win_deny_perms,
win_inheritance=win_inheritance,
win_perms_reset=win_perms_reset)
else: else:
presult, pcomment, ret['pchanges'] = _check_directory( presult, pcomment, pchanges = _check_directory(
name, user, group, recurse or [], dir_mode, clean, require, name, user, group, recurse or [], dir_mode, clean, require,
exclude_pat, max_depth, follow_symlinks) exclude_pat, max_depth, follow_symlinks)
if __opts__['test']: if pchanges:
ret['pchanges'].update(pchanges)
# Don't run through the reset of the function if there are no changes to be
# made
if not ret['pchanges'] or __opts__['test']:
ret['result'] = presult ret['result'] = presult
ret['comment'] = pcomment ret['comment'] = pcomment
ret['changes'] = ret['pchanges']
return ret return ret
if not os.path.isdir(name): if not os.path.isdir(name):
@ -2900,8 +2962,13 @@ def directory(name,
if not os.path.isdir(drive): if not os.path.isdir(drive):
return _error( return _error(
ret, 'Drive {0} is not mapped'.format(drive)) ret, 'Drive {0} is not mapped'.format(drive))
__salt__['file.makedirs'](name, win_owner, win_perms, __salt__['file.makedirs'](
win_deny_perms, win_inheritance) path=name,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else: else:
__salt__['file.makedirs'](name, user=user, group=group, __salt__['file.makedirs'](name, user=user, group=group,
mode=dir_mode) mode=dir_mode)
@ -2910,8 +2977,13 @@ def directory(name,
ret, 'No directory to create {0} in'.format(name)) ret, 'No directory to create {0} in'.format(name))
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
__salt__['file.mkdir'](name, win_owner, win_perms, win_deny_perms, __salt__['file.mkdir'](
win_inheritance) path=name,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else: else:
__salt__['file.mkdir'](name, user=user, group=group, mode=dir_mode) __salt__['file.mkdir'](name, user=user, group=group, mode=dir_mode)
@ -2925,7 +2997,13 @@ def directory(name,
if not children_only: if not children_only:
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms']( ret = __salt__['file.check_perms'](
name, ret, win_owner, win_perms, win_deny_perms, None, win_inheritance) path=name,
ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else: else:
ret, perms = __salt__['file.check_perms']( ret, perms = __salt__['file.check_perms'](
name, ret, user, group, dir_mode, None, follow_symlinks) name, ret, user, group, dir_mode, None, follow_symlinks)
@ -2996,8 +3074,13 @@ def directory(name,
try: try:
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms']( ret = __salt__['file.check_perms'](
full, ret, win_owner, win_perms, win_deny_perms, None, path=full,
win_inheritance) ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else: else:
ret, _ = __salt__['file.check_perms']( ret, _ = __salt__['file.check_perms'](
full, ret, user, group, file_mode, None, follow_symlinks) full, ret, user, group, file_mode, None, follow_symlinks)
@ -3011,8 +3094,13 @@ def directory(name,
try: try:
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms']( ret = __salt__['file.check_perms'](
full, ret, win_owner, win_perms, win_deny_perms, None, path=full,
win_inheritance) ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else: else:
ret, _ = __salt__['file.check_perms']( ret, _ = __salt__['file.check_perms'](
full, ret, user, group, dir_mode, None, follow_symlinks) full, ret, user, group, dir_mode, None, follow_symlinks)
@ -3034,6 +3122,7 @@ def directory(name,
if children_only: if children_only:
ret['comment'] = u'Directory {0}/* updated'.format(name) ret['comment'] = u'Directory {0}/* updated'.format(name)
else: else:
if ret['changes']:
ret['comment'] = u'Directory {0} updated'.format(name) ret['comment'] = u'Directory {0} updated'.format(name)
if __opts__['test']: if __opts__['test']:

47
salt/states/saltmod.py
View File

@ -787,28 +787,15 @@ def runner(name, **kwargs):
runner_return = out.get('return') runner_return = out.get('return')
if isinstance(runner_return, dict) and 'Error' in runner_return: if isinstance(runner_return, dict) and 'Error' in runner_return:
out['success'] = False out['success'] = False
if not out.get('success', True):
cmt = "Runner function '{0}' failed{1}.".format( success = out.get('success', True)
ret = {'name': name,
'changes': {'return': runner_return},
'result': success}
ret['comment'] = "Runner function '{0}' {1}.".format(
name, name,
' with return {0}'.format(runner_return) if runner_return else '', 'executed' if success else 'failed',
) )
ret = {
'name': name,
'result': False,
'changes': {},
'comment': cmt,
}
else:
cmt = "Runner function '{0}' executed{1}.".format(
name,
' with return {0}'.format(runner_return) if runner_return else '',
)
ret = {
'name': name,
'result': True,
'changes': {},
'comment': cmt,
}
ret['__orchestration__'] = True ret['__orchestration__'] = True
if 'jid' in out: if 'jid' in out:
@ -1039,15 +1026,21 @@ def wheel(name, **kwargs):
__env__=__env__, __env__=__env__,
**kwargs) **kwargs)
ret['result'] = True wheel_return = out.get('return')
if isinstance(wheel_return, dict) and 'Error' in wheel_return:
out['success'] = False
success = out.get('success', True)
ret = {'name': name,
'changes': {'return': wheel_return},
'result': success}
ret['comment'] = "Wheel function '{0}' {1}.".format(
name,
'executed' if success else 'failed',
)
ret['__orchestration__'] = True ret['__orchestration__'] = True
if 'jid' in out: if 'jid' in out:
ret['__jid__'] = out['jid'] ret['__jid__'] = out['jid']
runner_return = out.get('return')
ret['comment'] = "Wheel function '{0}' executed{1}.".format(
name,
' with return {0}'.format(runner_return) if runner_return else '',
)
return ret return ret

6
salt/utils/args.py
View File

@ -451,10 +451,10 @@ def format_call(fun,
continue continue
extra[key] = copy.deepcopy(value) extra[key] = copy.deepcopy(value)
# We'll be showing errors to the users until Salt Oxygen comes out, after # We'll be showing errors to the users until Salt Fluorine comes out, after
# which, errors will be raised instead. # which, errors will be raised instead.
salt.utils.versions.warn_until( salt.utils.versions.warn_until(
'Oxygen', 'Fluorine',
'It\'s time to start raising `SaltInvocationError` instead of ' 'It\'s time to start raising `SaltInvocationError` instead of '
'returning warnings', 'returning warnings',
# Let's not show the deprecation warning on the console, there's no # Let's not show the deprecation warning on the console, there's no
@ -491,7 +491,7 @@ def format_call(fun,
'{0}. If you were trying to pass additional data to be used ' '{0}. If you were trying to pass additional data to be used '
'in a template context, please populate \'context\' with ' 'in a template context, please populate \'context\' with '
'\'key: value\' pairs. Your approach will work until Salt ' '\'key: value\' pairs. Your approach will work until Salt '
'Oxygen is out.{1}'.format( 'Fluorine is out.{1}'.format(
msg, msg,
'' if 'full' not in ret else ' Please update your state files.' '' if 'full' not in ret else ' Please update your state files.'
) )

184
salt/utils/schedule.py
View File

@ -89,6 +89,28 @@ localtime.
This will schedule the command: ``state.sls httpd test=True`` at 5:00 PM on This will schedule the command: ``state.sls httpd test=True`` at 5:00 PM on
Monday, Wednesday and Friday, and 3:00 PM on Tuesday and Thursday. Monday, Wednesday and Friday, and 3:00 PM on Tuesday and Thursday.
.. code-block:: yaml
schedule:
job1:
function: state.sls
args:
- httpd
kwargs:
test: True
when:
- 'tea time'
.. code-block:: yaml
whens:
tea time: 1:40pm
deployment time: Friday 5:00pm
The Salt scheduler also allows custom phrases to be used for the `when`
parameter. These `whens` can be stored as either pillar values or
grain values.
.. code-block:: yaml .. code-block:: yaml
schedule: schedule:
@ -333,7 +355,6 @@ import logging
import errno import errno
import random import random
import yaml import yaml
import copy
# Import Salt libs # Import Salt libs
import salt.config import salt.config
@ -409,6 +430,7 @@ class Schedule(object):
self.proxy = proxy self.proxy = proxy
self.functions = functions self.functions = functions
self.standalone = standalone self.standalone = standalone
self.skip_function = None
if isinstance(intervals, dict): if isinstance(intervals, dict):
self.intervals = intervals self.intervals = intervals
else: else:
@ -745,6 +767,69 @@ class Schedule(object):
evt.fire_event({'complete': True}, evt.fire_event({'complete': True},
tag='/salt/minion/minion_schedule_saved') tag='/salt/minion/minion_schedule_saved')
def postpone_job(self, name, data):
'''
Postpone a job in the scheduler.
Ignores jobs from pillar
'''
time = data['time']
new_time = data['new_time']
# ensure job exists, then disable it
if name in self.opts['schedule']:
if 'skip_explicit' not in self.opts['schedule'][name]:
self.opts['schedule'][name]['skip_explicit'] = []
self.opts['schedule'][name]['skip_explicit'].append(time)
if 'run_explicit' not in self.opts['schedule'][name]:
self.opts['schedule'][name]['run_explicit'] = []
self.opts['schedule'][name]['run_explicit'].append(new_time)
elif name in self._get_schedule(include_opts=False):
log.warning('Cannot modify job {0}, '
'it`s in the pillar!'.format(name))
# Fire the complete event back along with updated list of schedule
evt = salt.utils.event.get_event('minion', opts=self.opts, listen=False)
evt.fire_event({'complete': True, 'schedule': self._get_schedule()},
tag='/salt/minion/minion_schedule_postpone_job_complete')
def skip_job(self, name, data):
'''
Skip a job at a specific time in the scheduler.
Ignores jobs from pillar
'''
time = data['time']
# ensure job exists, then disable it
if name in self.opts['schedule']:
if 'skip_explicit' not in self.opts['schedule'][name]:
self.opts['schedule'][name]['skip_explicit'] = []
self.opts['schedule'][name]['skip_explicit'].append(time)
elif name in self._get_schedule(include_opts=False):
log.warning('Cannot modify job {0}, '
'it`s in the pillar!'.format(name))
# Fire the complete event back along with updated list of schedule
evt = salt.utils.event.get_event('minion', opts=self.opts, listen=False)
evt.fire_event({'complete': True, 'schedule': self._get_schedule()},
tag='/salt/minion/minion_schedule_skip_job_complete')
def get_next_fire_time(self, name):
'''
Disable a job in the scheduler. Ignores jobs from pillar
'''
schedule = self._get_schedule()
if schedule:
_next_fire_time = schedule[name]['_next_fire_time']
# Fire the complete event back along with updated list of schedule
evt = salt.utils.event.get_event('minion', opts=self.opts, listen=False)
evt.fire_event({'complete': True, 'next_fire_time': _next_fire_time},
tag='/salt/minion/minion_schedule_next_fire_time_complete')
def handle_func(self, multiprocessing_enabled, func, data): def handle_func(self, multiprocessing_enabled, func, data):
''' '''
Execute this method in a multiprocess or thread Execute this method in a multiprocess or thread
@ -948,11 +1033,16 @@ class Schedule(object):
# Let's make sure we exit the process! # Let's make sure we exit the process!
sys.exit(salt.defaults.exitcodes.EX_GENERIC) sys.exit(salt.defaults.exitcodes.EX_GENERIC)
def eval(self): def eval(self, now=None):
''' '''
Evaluate and execute the schedule Evaluate and execute the schedule
:param int now: Override current time with a Unix timestamp``
''' '''
log.trace('==== evaluating schedule =====')
def _splay(splaytime): def _splay(splaytime):
''' '''
Calculate splaytime Calculate splaytime
@ -974,9 +1064,13 @@ class Schedule(object):
raise ValueError('Schedule must be of type dict.') raise ValueError('Schedule must be of type dict.')
if 'enabled' in schedule and not schedule['enabled']: if 'enabled' in schedule and not schedule['enabled']:
return return
if 'skip_function' in schedule:
self.skip_function = schedule['skip_function']
for job, data in six.iteritems(schedule): for job, data in six.iteritems(schedule):
if job == 'enabled' or not data: if job == 'enabled' or not data:
continue continue
if job == 'skip_function' or not data:
continue
if not isinstance(data, dict): if not isinstance(data, dict):
log.error('Scheduled job "{0}" should have a dict value, not {1}'.format(job, type(data))) log.error('Scheduled job "{0}" should have a dict value, not {1}'.format(job, type(data)))
continue continue
@ -1011,6 +1105,7 @@ class Schedule(object):
'_run_on_start' not in data: '_run_on_start' not in data:
data['_run_on_start'] = True data['_run_on_start'] = True
if not now:
now = int(time.time()) now = int(time.time())
if 'until' in data: if 'until' in data:
@ -1065,6 +1160,23 @@ class Schedule(object):
'", "'.join(scheduling_elements))) '", "'.join(scheduling_elements)))
continue continue
if 'run_explicit' in data:
_run_explicit = data['run_explicit']
if isinstance(_run_explicit, six.string_types):
_run_explicit = [_run_explicit]
# Copy the list so we can loop through it
for i in copy.deepcopy(_run_explicit):
if len(_run_explicit) > 1:
if i < now - self.opts['loop_interval']:
_run_explicit.remove(i)
if _run_explicit:
if _run_explicit[0] <= now < (_run_explicit[0] + self.opts['loop_interval']):
run = True
data['_next_fire_time'] = _run_explicit[0]
if True in [True for item in time_elements if item in data]: if True in [True for item in time_elements if item in data]:
if '_seconds' not in data: if '_seconds' not in data:
interval = int(data.get('seconds', 0)) interval = int(data.get('seconds', 0))
@ -1153,7 +1265,8 @@ class Schedule(object):
# Copy the list so we can loop through it # Copy the list so we can loop through it
for i in copy.deepcopy(_when): for i in copy.deepcopy(_when):
if i < now and len(_when) > 1: if len(_when) > 1:
if i < now - self.opts['loop_interval']:
# Remove all missed schedules except the latest one. # Remove all missed schedules except the latest one.
# We need it to detect if it was triggered previously. # We need it to detect if it was triggered previously.
_when.remove(i) _when.remove(i)
@ -1258,15 +1371,17 @@ class Schedule(object):
seconds = data['_next_fire_time'] - now seconds = data['_next_fire_time'] - now
if data['_splay']: if data['_splay']:
seconds = data['_splay'] - now seconds = data['_splay'] - now
if seconds <= 0:
if '_seconds' in data: if '_seconds' in data:
if seconds <= 0:
run = True run = True
elif 'when' in data and data['_run']: elif 'when' in data and data['_run']:
if data['_next_fire_time'] <= now <= (data['_next_fire_time'] + self.opts['loop_interval']):
data['_run'] = False data['_run'] = False
run = True run = True
elif 'cron' in data: elif 'cron' in data:
# Reset next scheduled time because it is in the past now, # Reset next scheduled time because it is in the past now,
# and we should trigger the job run, then wait for the next one. # and we should trigger the job run, then wait for the next one.
if seconds <= 0:
data['_next_fire_time'] = None data['_next_fire_time'] = None
run = True run = True
elif seconds == 0: elif seconds == 0:
@ -1311,6 +1426,10 @@ class Schedule(object):
else: else:
if start <= now <= end: if start <= now <= end:
run = True run = True
else:
if self.skip_function:
run = True
func = self.skip_function
else: else:
run = False run = False
else: else:
@ -1322,6 +1441,62 @@ class Schedule(object):
Ignoring job {0}.'.format(job)) Ignoring job {0}.'.format(job))
continue continue
if 'skip_during_range' in data:
if not _RANGE_SUPPORTED:
log.error('Missing python-dateutil. Ignoring job {0}'.format(job))
continue
else:
if isinstance(data['skip_during_range'], dict):
try:
start = int(time.mktime(dateutil_parser.parse(data['skip_during_range']['start']).timetuple()))
except ValueError:
log.error('Invalid date string for start in skip_during_range. Ignoring job {0}.'.format(job))
continue
try:
end = int(time.mktime(dateutil_parser.parse(data['skip_during_range']['end']).timetuple()))
except ValueError:
log.error('Invalid date string for end in skip_during_range. Ignoring job {0}.'.format(job))
log.error(data)
continue
if end > start:
if start <= now <= end:
if self.skip_function:
run = True
func = self.skip_function
else:
run = False
else:
run = True
else:
log.error('schedule.handle_func: Invalid range, end must be larger than start. \
Ignoring job {0}.'.format(job))
continue
else:
log.error('schedule.handle_func: Invalid, range must be specified as a dictionary. \
Ignoring job {0}.'.format(job))
continue
if 'skip_explicit' in data:
_skip_explicit = data['skip_explicit']
if isinstance(_skip_explicit, six.string_types):
_skip_explicit = [_skip_explicit]
# Copy the list so we can loop through it
for i in copy.deepcopy(_skip_explicit):
if i < now - self.opts['loop_interval']:
_skip_explicit.remove(i)
if _skip_explicit:
if _skip_explicit[0] <= now <= (_skip_explicit[0] + self.opts['loop_interval']):
if self.skip_function:
run = True
func = self.skip_function
else:
run = False
else:
run = True
if not run: if not run:
continue continue
@ -1374,6 +1549,7 @@ class Schedule(object):
finally: finally:
if '_seconds' in data: if '_seconds' in data:
data['_next_fire_time'] = now + data['_seconds'] data['_next_fire_time'] = now + data['_seconds']
data['_last_run'] = now
data['_splay'] = None data['_splay'] = None
if salt.utils.platform.is_windows(): if salt.utils.platform.is_windows():
# Restore our function references. # Restore our function references.

3
salt/wheel/__init__.py
View File

@ -43,7 +43,8 @@ class WheelClient(salt.client.mixins.SyncClientMixin,
def __init__(self, opts=None): def __init__(self, opts=None):
self.opts = opts self.opts = opts
self.functions = salt.loader.wheels(opts) self.context = {}
self.functions = salt.loader.wheels(opts, context=self.context)
# TODO: remove/deprecate # TODO: remove/deprecate
def call_func(self, fun, **kwargs): def call_func(self, fun, **kwargs):

16
tests/integration/files/file/base/_runners/runtests_helpers.py Normal file
View File

@ -0,0 +1,16 @@
# -*- coding: utf-8 -*-
'''
Runner functions for integration tests
'''
# Import python libs
from __future__ import absolute_import
def failure():
__context__['retcode'] = 1
return False
def success():
return True

16
tests/integration/files/file/base/_wheel/runtests_helpers.py Normal file
View File

@ -0,0 +1,16 @@
# -*- coding: utf-8 -*-
'''
Wheel functions for integration tests
'''
# Import python libs
from __future__ import absolute_import
def failure():
__context__['retcode'] = 1
return False
def success():
return True

15
tests/integration/files/file/base/orch/retcode.sls Normal file
View File

@ -0,0 +1,15 @@
test_runner_success:
salt.runner:
- name: runtests_helpers.success
test_runner_failure:
salt.runner:
- name: runtests_helpers.failure
test_wheel_success:
salt.wheel:
- name: runtests_helpers.success
test_wheel_failure:
salt.wheel:
- name: runtests_helpers.failure

12
tests/integration/modules/test_saltutil.py
View File

@ -93,7 +93,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [], 'states': [],
'sdb': [], 'sdb': [],
'proxymodules': [], 'proxymodules': [],
'output': []} 'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all') ret = self.run_function('saltutil.sync_all')
self.assertEqual(ret, expected_return) self.assertEqual(ret, expected_return)
@ -113,7 +114,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [], 'states': [],
'sdb': [], 'sdb': [],
'proxymodules': [], 'proxymodules': [],
'output': []} 'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all', extmod_whitelist={'modules': ['salttest']}) ret = self.run_function('saltutil.sync_all', extmod_whitelist={'modules': ['salttest']})
self.assertEqual(ret, expected_return) self.assertEqual(ret, expected_return)
@ -135,7 +137,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [], 'states': [],
'sdb': [], 'sdb': [],
'proxymodules': [], 'proxymodules': [],
'output': []} 'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all', extmod_blacklist={'modules': ['runtests_decorators']}) ret = self.run_function('saltutil.sync_all', extmod_blacklist={'modules': ['runtests_decorators']})
self.assertEqual(ret, expected_return) self.assertEqual(ret, expected_return)
@ -155,7 +158,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [], 'states': [],
'sdb': [], 'sdb': [],
'proxymodules': [], 'proxymodules': [],
'output': []} 'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all', extmod_whitelist={'modules': ['runtests_decorators']}, ret = self.run_function('saltutil.sync_all', extmod_whitelist={'modules': ['runtests_decorators']},
extmod_blacklist={'modules': ['runtests_decorators']}) extmod_blacklist={'modules': ['runtests_decorators']})
self.assertEqual(ret, expected_return) self.assertEqual(ret, expected_return)

29
tests/integration/runners/test_state.py
View File

@ -106,6 +106,35 @@ class StateRunnerTest(ShellCase):
for item in out: for item in out:
self.assertIn(item, ret) self.assertIn(item, ret)
def test_orchestrate_retcode(self):
'''
Test orchestration with nonzero retcode set in __context__
'''
self.run_run('saltutil.sync_runners')
self.run_run('saltutil.sync_wheel')
ret = '\n'.join(self.run_run('state.orchestrate orch.retcode'))
for result in (' ID: test_runner_success\n'
' Function: salt.runner\n'
' Name: runtests_helpers.success\n'
' Result: True',
' ID: test_runner_failure\n'
' Function: salt.runner\n'
' Name: runtests_helpers.failure\n'
' Result: False',
' ID: test_wheel_success\n'
' Function: salt.wheel\n'
' Name: runtests_helpers.success\n'
' Result: True',
' ID: test_wheel_failure\n'
' Function: salt.wheel\n'
' Name: runtests_helpers.failure\n'
' Result: False'):
self.assertIn(result, ret)
def test_orchestrate_target_doesnt_exists(self): def test_orchestrate_target_doesnt_exists(self):
''' '''
test orchestration when target doesnt exist test orchestration when target doesnt exist

23
tests/unit/modules/test_ansiblegate.py
View File

@ -126,3 +126,26 @@ description:
patch('salt.modules.ansiblegate.importlib.import_module', lambda x: x): patch('salt.modules.ansiblegate.importlib.import_module', lambda x: x):
with pytest.raises(LoaderError) as loader_error: with pytest.raises(LoaderError) as loader_error:
self.resolver.load_module('something.strange') self.resolver.load_module('something.strange')
def test_virtual_function_no_ansible_installed(self):
'''
Test Ansible module __virtual__ when ansible is not installed on the minion.
:return:
'''
with patch('salt.modules.ansiblegate.ansible', None):
assert ansible.__virtual__() == (False, 'Ansible is not installed on this system')
@patch('salt.modules.ansiblegate.ansible', MagicMock())
@patch('salt.modules.ansiblegate.list', MagicMock())
@patch('salt.modules.ansiblegate._set_callables', MagicMock())
@patch('salt.modules.ansiblegate.AnsibleModuleCaller', MagicMock())
def test_virtual_function_ansible_is_installed(self):
'''
Test Ansible module __virtual__ when ansible is installed on the minion.
:return:
'''
resolver = MagicMock()
resolver.resolve = MagicMock()
resolver.resolve.install = MagicMock()
with patch('salt.modules.ansiblegate.AnsibleModuleResolver', resolver):
assert ansible.__virtual__() == (True, None)

7
tests/unit/states/test_file.py
View File

@ -815,7 +815,7 @@ class TestFileState(TestCase, LoaderModuleMockMixin):
'comment': comt, 'comment': comt,
'result': None, 'result': None,
'pchanges': p_chg, 'pchanges': p_chg,
'changes': {'/etc/grub.conf': {'directory': 'new'}} 'changes': {}
}) })
self.assertDictEqual(filestate.directory(name, self.assertDictEqual(filestate.directory(name,
user=user, user=user,
@ -841,6 +841,11 @@ class TestFileState(TestCase, LoaderModuleMockMixin):
ret) ret)
recurse = ['ignore_files', 'ignore_dirs'] recurse = ['ignore_files', 'ignore_dirs']
ret.update({'comment': 'Must not specify "recurse" '
'options "ignore_files" and '
'"ignore_dirs" at the same '
'time.',
'pchanges': {}})
with patch.object(os.path, 'isdir', mock_t): with patch.object(os.path, 'isdir', mock_t):
self.assertDictEqual(filestate.directory self.assertDictEqual(filestate.directory
(name, user=user, (name, user=user,

8
tests/unit/states/test_saltmod.py
View File

@ -258,8 +258,8 @@ class SaltmodTestCase(TestCase, LoaderModuleMockMixin):
''' '''
name = 'state' name = 'state'
ret = {'changes': {}, 'name': 'state', 'result': True, ret = {'changes': {'return': True}, 'name': 'state', 'result': True,
'comment': 'Runner function \'state\' executed with return True.', 'comment': 'Runner function \'state\' executed.',
'__orchestration__': True} '__orchestration__': True}
runner_mock = MagicMock(return_value={'return': True}) runner_mock = MagicMock(return_value={'return': True})
@ -274,8 +274,8 @@ class SaltmodTestCase(TestCase, LoaderModuleMockMixin):
''' '''
name = 'state' name = 'state'
ret = {'changes': {}, 'name': 'state', 'result': True, ret = {'changes': {'return': True}, 'name': 'state', 'result': True,
'comment': 'Wheel function \'state\' executed with return True.', 'comment': 'Wheel function \'state\' executed.',
'__orchestration__': True} '__orchestration__': True}
wheel_mock = MagicMock(return_value={'return': True}) wheel_mock = MagicMock(return_value={'return': True})