valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-08 01:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'develop' into alternate_composefile

Browse Source
This commit is contained in:
Sjuul Janssen 2017-12-04 10:31:10 +01:00 committed by GitHub
commit 0d81aa83e2
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
39 changed files with 1487 additions and 335 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
conf/master
View File

@ -297,6 +297,11 @@
#batch_safe_limit: 100
#batch_safe_size: 8
# Master stats enables stats events to be fired from the master at close
# to the defined interval
#master_stats: False
#master_stats_event_iter: 60
##### Security settings #####
##########################################

23
doc/ref/configuration/master.rst
View File

@ -868,6 +868,29 @@ what you are doing! Transports are explained in :ref:`Salt Transports
ret_port: 4606
zeromq: []
.. conf_master:: master_stats
``master_stats``
----------------
Default: False
Turning on the master stats enables runtime throughput and statistics events
to be fired from the master event bus. These events will report on what
functions have been run on the master and how long these runs have, on
average, taken over a given period of time.
.. conf_master:: master_stats_event_iter
``master_stats_event_iter``
---------------------------
Default: 60
The time in seconds to fire master_stats events. This will only fire in
conjunction with receiving a request to the master, idle masters will not
fire these events.
.. conf_master:: sock_pool_size
``sock_pool_size``

2
doc/ref/states/writing.rst
View File

@ -111,6 +111,8 @@ This code will call the `managed` function in the :mod:`file
<salt.states.file>` state module and pass the arguments ``name`` and ``source``
to it.
.. _state-return-data:
Return Data
===========

32
doc/topics/orchestrate/orchestrate_runner.rst
View File

@ -5,10 +5,10 @@ Orchestrate Runner
==================
Executing states or highstate on a minion is perfect when you want to ensure that
minion configured and running the way you want. Sometimes however you want to
minion configured and running the way you want. Sometimes however you want to
configure a set of minions all at once.
For example, if you want to set up a load balancer in front of a cluster of web
For example, if you want to set up a load balancer in front of a cluster of web
servers you can ensure the load balancer is set up first, and then the same
matching configuration is applied consistently across the whole cluster.
@ -222,6 +222,34 @@ To execute with pillar data.
"master": "mymaster"}'
Return Codes in Runner/Wheel Jobs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. versionadded:: Oxygen
State (``salt.state``) jobs are able to report failure via the :ref:`state
return dictionary <state-return-data>`. Remote execution (``salt.function``)
jobs are able to report failure by setting a ``retcode`` key in the
``__context__`` dictionary. However, runner (``salt.runner``) and wheel
(``salt.wheel``) jobs would only report a ``False`` result when the
runner/wheel function raised an exception. As of the Oxygen release, it is now
possible to set a retcode in runner and wheel functions just as you can do in
remote execution functions. Here is some example pseudocode:
.. code-block:: python
def myrunner():
...
do stuff
...
if some_error_condition:
__context__['retcode'] = 1
return result
This allows a custom runner/wheel function to report its failure so that
requisites can accurately tell that a job has failed.
More Complex Orchestration
~~~~~~~~~~~~~~~~~~~~~~~~~~

19
doc/topics/releases/oxygen.rst
View File

@ -25,6 +25,25 @@ by any master tops matches that are not matched via a top file.
To make master tops matches execute first, followed by top file matches, set
the new :conf_minion:`master_tops_first` minion config option to ``True``.
Return Codes for Runner/Wheel Functions
---------------------------------------
When using :ref:`orchestration <orchestrate-runner>`, runner and wheel
functions used to report a ``True`` result if the function ran to completion
without raising an exception. It is now possible to set a return code in the
``__context__`` dictionary, allowing runner and wheel functions to report that
they failed. Here's some example pseudocode:
.. code-block:: python
def myrunner():
...
do stuff
...
if some_error_condition:
__context__['retcode'] = 1
return result
LDAP via External Authentication Changes
----------------------------------------
In this release of Salt, if LDAP Bind Credentials are supplied, then

160
doc/topics/tutorials/salt_bootstrap.rst
View File

@ -78,7 +78,7 @@ UNIX systems
**BSD**:
- OpenBSD (``pip`` installation)
- OpenBSD
- FreeBSD 9/10/11
**SunOS**:
@ -272,66 +272,118 @@ Here's a summary of the command line options:
$ sh bootstrap-salt.sh -h
Usage : bootstrap-salt.sh [options] <install-type> <install-type-args>
Installation types:
- stable (default)
- stable [version] (ubuntu specific)
- daily (ubuntu specific)
- testing (redhat specific)
- git
- stable Install latest stable release. This is the default
install type
- stable [branch] Install latest version on a branch. Only supported
for packages available at repo.saltstack.com
- stable [version] Install a specific version. Only supported for
packages available at repo.saltstack.com
- daily Ubuntu specific: configure SaltStack Daily PPA
- testing RHEL-family specific: configure EPEL testing repo
- git Install from the head of the develop branch
- git [ref] Install from any git ref (such as a branch, tag, or
commit)
Examples:
- bootstrap-salt.sh
- bootstrap-salt.sh stable
- bootstrap-salt.sh stable 2014.7
- bootstrap-salt.sh stable 2017.7
- bootstrap-salt.sh stable 2017.7.2
- bootstrap-salt.sh daily
- bootstrap-salt.sh testing
- bootstrap-salt.sh git
- bootstrap-salt.sh git develop
- bootstrap-salt.sh git v0.17.0
- bootstrap-salt.sh git 8c3fadf15ec183e5ce8c63739850d543617e4357
- bootstrap-salt.sh git 2017.7
- bootstrap-salt.sh git v2017.7.2
- bootstrap-salt.sh git 06f249901a2e2f1ed310d58ea3921a129f214358
Options:
-h Display this message
-v Display script version
-n No colours.
-D Show debug output.
-c Temporary configuration directory
-g Salt repository URL. (default: git://github.com/saltstack/salt.git)
-G Instead of cloning from git://github.com/saltstack/salt.git, clone from https://github.com/saltstack/salt.git (Usually necessary on systems which have the regular git protocol port blocked, where https usually is not)
-k Temporary directory holding the minion keys which will pre-seed
the master.
-s Sleep time used when waiting for daemons to start, restart and when checking
for the services running. Default: 3
-M Also install salt-master
-S Also install salt-syndic
-N Do not install salt-minion
-X Do not start daemons after installation
-C Only run the configuration function. This option automatically
bypasses any installation.
-P Allow pip based installations. On some distributions the required salt
packages or its dependencies are not available as a package for that
distribution. Using this flag allows the script to use pip as a last
resort method. NOTE: This only works for functions which actually
implement pip based installations.
-F Allow copied files to overwrite existing(config, init.d, etc)
-U If set, fully upgrade the system prior to bootstrapping salt
-K If set, keep the temporary files in the temporary directories specified
with -c and -k.
-I If set, allow insecure connections while downloading any files. For
example, pass '--no-check-certificate' to 'wget' or '--insecure' to 'curl'
-A Pass the salt-master DNS name or IP. This will be stored under
${BS_SALT_ETC_DIR}/minion.d/99-master-address.conf
-i Pass the salt-minion id. This will be stored under
${BS_SALT_ETC_DIR}/minion_id
-L Install the Apache Libcloud package if possible(required for salt-cloud)
-p Extra-package to install while installing salt dependencies. One package
per -p flag. You're responsible for providing the proper package name.
-d Disable check_service functions. Setting this flag disables the
'install_<distro>_check_services' checks. You can also do this by
touching /tmp/disable_salt_checks on the target host. Defaults ${BS_FALSE}
-H Use the specified http proxy for the installation
-Z Enable external software source for newer ZeroMQ(Only available for RHEL/CentOS/Fedora/Ubuntu based distributions)
-b Assume that dependencies are already installed and software sources are set up.
If git is selected, git tree is still checked out as dependency step.
-h Display this message
-v Display script version
-n No colours
-D Show debug output
-c Temporary configuration directory
-g Salt Git repository URL. Default: https://github.com/saltstack/salt.git
-w Install packages from downstream package repository rather than
upstream, saltstack package repository. This is currently only
implemented for SUSE.
-k Temporary directory holding the minion keys which will pre-seed
the master.
-s Sleep time used when waiting for daemons to start, restart and when
checking for the services running. Default: 3
-L Also install salt-cloud and required python-libcloud package
-M Also install salt-master
-S Also install salt-syndic
-N Do not install salt-minion
-X Do not start daemons after installation
-d Disables checking if Salt services are enabled to start on system boot.
You can also do this by touching /tmp/disable_salt_checks on the target
host. Default: ${BS_FALSE}
-P Allow pip based installations. On some distributions the required salt
packages or its dependencies are not available as a package for that
distribution. Using this flag allows the script to use pip as a last
resort method. NOTE: This only works for functions which actually
implement pip based installations.
-U If set, fully upgrade the system prior to bootstrapping Salt
-I If set, allow insecure connections while downloading any files. For
example, pass '--no-check-certificate' to 'wget' or '--insecure' to
'curl'. On Debian and Ubuntu, using this option with -U allows to obtain
GnuPG archive keys insecurely if distro has changed release signatures.
-F Allow copied files to overwrite existing (config, init.d, etc)
-K If set, keep the temporary files in the temporary directories specified
with -c and -k
-C Only run the configuration function. Implies -F (forced overwrite).
To overwrite Master or Syndic configs, -M or -S, respectively, must
also be specified. Salt installation will be ommitted, but some of the
dependencies could be installed to write configuration with -j or -J.
-A Pass the salt-master DNS name or IP. This will be stored under
${BS_SALT_ETC_DIR}/minion.d/99-master-address.conf
-i Pass the salt-minion id. This will be stored under
${BS_SALT_ETC_DIR}/minion_id
-p Extra-package to install while installing Salt dependencies. One package
per -p flag. You're responsible for providing the proper package name.
-H Use the specified HTTP proxy for all download URLs (including https://).
For example: http://myproxy.example.com:3128
-Z Enable additional package repository for newer ZeroMQ
(only available for RHEL/CentOS/Fedora/Ubuntu based distributions)
-b Assume that dependencies are already installed and software sources are
set up. If git is selected, git tree is still checked out as dependency
step.
-f Force shallow cloning for git installations.
This may result in an "n/a" in the version number.
-l Disable ssl checks. When passed, switches "https" calls to "http" where
possible.
-V Install Salt into virtualenv
(only available for Ubuntu based distributions)
-a Pip install all Python pkg dependencies for Salt. Requires -V to install
all pip pkgs into the virtualenv.
(Only available for Ubuntu based distributions)
-r Disable all repository configuration performed by this script. This
option assumes all necessary repository configuration is already present
on the system.
-R Specify a custom repository URL. Assumes the custom repository URL
points to a repository that mirrors Salt packages located at
repo.saltstack.com. The option passed with -R replaces the
"repo.saltstack.com". If -R is passed, -r is also set. Currently only
works on CentOS/RHEL and Debian based distributions.
-J Replace the Master config file with data passed in as a JSON string. If
a Master config file is found, a reasonable effort will be made to save
the file with a ".bak" extension. If used in conjunction with -C or -F,
no ".bak" file will be created as either of those options will force
a complete overwrite of the file.
-j Replace the Minion config file with data passed in as a JSON string. If
a Minion config file is found, a reasonable effort will be made to save
the file with a ".bak" extension. If used in conjunction with -C or -F,
no ".bak" file will be created as either of those options will force
a complete overwrite of the file.
-q Quiet salt installation from git (setup.py install -q)
-x Changes the python version used to install a git version of salt. Currently
this is considered experimental and has only been tested on Centos 6. This
only works for git installations.
-y Installs a different python version on host. Currently this has only been
tested with Centos 6 and is considered experimental. This will install the
ius repo on the box if disable repo is false. This must be used in conjunction
with -x <pythonversion>. For example:
sh bootstrap.sh -P -y -x python2.7 git v2016.11.3
The above will install python27 and install the git version of salt using the
python2.7 executable. This only works for git and pip installations.

4
salt/cli/daemons.py
View File

@ -161,7 +161,6 @@ class Master(salt.utils.parsers.MasterOptionParser, DaemonsMixin): # pylint: di
v_dirs,
self.config['user'],
permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir'], self.config['key_dir']],
)
@ -283,7 +282,6 @@ class Minion(salt.utils.parsers.MinionOptionParser, DaemonsMixin): # pylint: di
v_dirs,
self.config['user'],
permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir']],
)
@ -472,7 +470,6 @@ class ProxyMinion(salt.utils.parsers.ProxyMinionOptionParser, DaemonsMixin): #
v_dirs,
self.config['user'],
permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir']],
)
@ -582,7 +579,6 @@ class Syndic(salt.utils.parsers.SyndicOptionParser, DaemonsMixin): # pylint: di
],
self.config['user'],
permissive=self.config['permissive_pki_access'],
pki_dir=self.config['pki_dir'],
root_dir=self.config['root_dir'],
sensitive_dirs=[self.config['pki_dir']],
)

6
salt/client/mixins.py
View File

@ -385,7 +385,11 @@ class SyncClientMixin(object):
# Initialize a context for executing the method.
with tornado.stack_context.StackContext(self.functions.context_dict.clone):
data[u'return'] = self.functions[fun](*args, **kwargs)
data[u'success'] = True
try:
data[u'success'] = self.context.get(u'retcode', 0) == 0
except AttributeError:
# Assume a True result if no context attribute
data[u'success'] = True
if isinstance(data[u'return'], dict) and u'data' in data[u'return']:
# some functions can return boolean values
data[u'success'] = salt.utils.state.check_result(data[u'return'][u'data'])

6
salt/config/__init__.py
View File

@ -165,6 +165,10 @@ VALID_OPTS = {
# The master_pubkey_signature must also be set for this.
'master_use_pubkey_signature': bool,
# Enable master stats eveents to be fired, these events will contain information about
# what commands the master is processing and what the rates are of the executions
'master_stats': bool,
'master_stats_event_iter': int,
# The key fingerprint of the higher-level master for the syndic to verify it is talking to the
# intended master
'syndic_finger': str,
@ -1515,6 +1519,8 @@ DEFAULT_MASTER_OPTS = {
'svnfs_saltenv_whitelist': [],
'svnfs_saltenv_blacklist': [],
'max_event_size': 1048576,
'master_stats': False,
'master_stats_event_iter': 60,
'minionfs_env': 'base',
'minionfs_mountpoint': '',
'minionfs_whitelist': [],

11
salt/loader.py
View File

@ -372,15 +372,18 @@ def tops(opts):
return FilterDictWrapper(ret, u'.top')
def wheels(opts, whitelist=None):
def wheels(opts, whitelist=None, context=None):
'''
Returns the wheels modules
'''
if context is None:
context = {}
return LazyLoader(
_module_dirs(opts, u'wheel'),
opts,
tag=u'wheel',
whitelist=whitelist,
pack={u'__context__': context},
)
@ -836,17 +839,19 @@ def call(fun, **kwargs):
return funcs[fun](*args)
def runner(opts, utils=None):
def runner(opts, utils=None, context=None):
'''
Directly call a function inside a loader directory
'''
if utils is None:
utils = {}
if context is None:
context = {}
ret = LazyLoader(
_module_dirs(opts, u'runners', u'runner', ext_type_dirs=u'runner_dirs'),
opts,
tag=u'runners',
pack={u'__utils__': utils},
pack={u'__utils__': utils, u'__context__': context},
)
# TODO: change from __salt__ to something else, we overload __salt__ too much
ret.pack[u'__salt__'] = ret

39
salt/master.py
View File

@ -16,6 +16,7 @@ import errno
import signal
import stat
import logging
import collections
import multiprocessing
import salt.serializers.msgpack
@ -797,6 +798,7 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
:return: Master worker
'''
kwargs[u'name'] = name
self.name = name
super(MWorker, self).__init__(**kwargs)
self.opts = opts
self.req_channels = req_channels
@ -804,6 +806,8 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
self.mkey = mkey
self.key = key
self.k_mtime = 0
self.stats = collections.defaultdict(lambda: {'mean': 0, 'runs': 0})
self.stat_clock = time.time()
# We need __setstate__ and __getstate__ to also pickle 'SMaster.secrets'.
# Otherwise, 'SMaster.secrets' won't be copied over to the spawned process
@ -879,6 +883,19 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
u'clear': self._handle_clear}[key](load)
raise tornado.gen.Return(ret)
def _post_stats(self, start, cmd):
'''
Calculate the master stats and fire events with stat info
'''
end = time.time()
duration = end - start
self.stats[cmd][u'mean'] = (self.stats[cmd][u'mean'] * (self.stats[cmd][u'runs'] - 1) + duration) / self.stats[cmd][u'runs']
if end - self.stat_clock > self.opts[u'master_stats_event_iter']:
# Fire the event with the stats and wipe the tracker
self.aes_funcs.event.fire_event({u'time': end - self.stat_clock, u'worker': self.name, u'stats': self.stats}, tagify(self.name, u'stats'))
self.stats = collections.defaultdict(lambda: {'mean': 0, 'runs': 0})
self.stat_clock = end
def _handle_clear(self, load):
'''
Process a cleartext command
@ -888,9 +905,16 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
the command specified in the load's 'cmd' key.
'''
log.trace(u'Clear payload received with command %s', load[u'cmd'])
if load[u'cmd'].startswith(u'__'):
cmd = load[u'cmd']
if cmd.startswith(u'__'):
return False
return getattr(self.clear_funcs, load[u'cmd'])(load), {u'fun': u'send_clear'}
if self.opts[u'master_stats']:
start = time.time()
self.stats[cmd][u'runs'] += 1
ret = getattr(self.clear_funcs, cmd)(load), {u'fun': u'send_clear'}
if self.opts[u'master_stats']:
self._post_stats(start, cmd)
return ret
def _handle_aes(self, data):
'''
@ -903,10 +927,17 @@ class MWorker(salt.utils.process.SignalHandlingMultiprocessingProcess):
if u'cmd' not in data:
log.error(u'Received malformed command %s', data)
return {}
cmd = data[u'cmd']
log.trace(u'AES payload received with command %s', data[u'cmd'])
if data[u'cmd'].startswith(u'__'):
if cmd.startswith(u'__'):
return False
return self.aes_funcs.run_func(data[u'cmd'], data)
if self.opts[u'master_stats']:
start = time.time()
self.stats[cmd][u'runs'] += 1
ret = self.aes_funcs.run_func(data[u'cmd'], data)
if self.opts[u'master_stats']:
self._post_stats(start, cmd)
return ret
def run(self):
'''

4
salt/minion.py
View File

@ -2067,12 +2067,16 @@ class Minion(MinionBase):
self.schedule.run_job(name)
elif func == u'disable_job':
self.schedule.disable_job(name, persist)
elif func == u'postpone_job':
self.schedule.postpone_job(name, data)
elif func == u'reload':
self.schedule.reload(schedule)
elif func == u'list':
self.schedule.list(where)
elif func == u'save_schedule':
self.schedule.save_schedule()
elif func == u'get_next_fire_time':
self.schedule.get_next_fire_time(name)
def manage_beacons(self, tag, data):
'''

6
salt/modules/ansiblegate.py
View File

@ -214,14 +214,12 @@ def __virtual__():
'''
ret = ansible is not None
msg = not ret and "Ansible is not installed on this system" or None
if msg:
log.warning(msg)
else:
if ret:
global _resolver
global _caller
_resolver = AnsibleModuleResolver(__opts__).resolve().install()
_caller = AnsibleModuleCaller(_resolver)
_set_callables(list())
_set_callables(list())
return ret, msg

58
salt/modules/file.py
View File

@ -552,11 +552,11 @@ def lsattr(path):
raise SaltInvocationError("File or directory does not exist.")
cmd = ['lsattr', path]
result = __salt__['cmd.run'](cmd, python_shell=False)
result = __salt__['cmd.run'](cmd, ignore_retcode=True, python_shell=False)
results = {}
for line in result.splitlines():
if not line.startswith('lsattr'):
if not line.startswith('lsattr: '):
vals = line.split(None, 1)
results[vals[1]] = re.findall(r"[acdijstuADST]", vals[0])
@ -5203,13 +5203,18 @@ def manage_file(name,
'Replace symbolic link with regular file'
if salt.utils.platform.is_windows():
ret = check_perms(name,
ret,
kwargs.get('win_owner'),
kwargs.get('win_perms'),
kwargs.get('win_deny_perms'),
None,
kwargs.get('win_inheritance'))
# This function resides in win_file.py and will be available
# on Windows. The local function will be overridden
# pylint: disable=E1120,E1121,E1123
ret = check_perms(
path=name,
ret=ret,
owner=kwargs.get('win_owner'),
grant_perms=kwargs.get('win_perms'),
deny_perms=kwargs.get('win_deny_perms'),
inheritance=kwargs.get('win_inheritance', True),
reset=kwargs.get('win_perms_reset', False))
# pylint: enable=E1120,E1121,E1123
else:
ret, _ = check_perms(name, ret, user, group, mode, attrs, follow_symlinks)
@ -5250,13 +5255,15 @@ def manage_file(name,
if salt.utils.platform.is_windows():
# This function resides in win_file.py and will be available
# on Windows. The local function will be overridden
# pylint: disable=E1121
makedirs_(name,
kwargs.get('win_owner'),
kwargs.get('win_perms'),
kwargs.get('win_deny_perms'),
kwargs.get('win_inheritance'))
# pylint: enable=E1121
# pylint: disable=E1120,E1121,E1123
makedirs_(
path=name,
owner=kwargs.get('win_owner'),
grant_perms=kwargs.get('win_perms'),
deny_perms=kwargs.get('win_deny_perms'),
inheritance=kwargs.get('win_inheritance', True),
reset=kwargs.get('win_perms_reset', False))
# pylint: enable=E1120,E1121,E1123
else:
makedirs_(name, user=user, group=group, mode=dir_mode)
@ -5369,13 +5376,18 @@ def manage_file(name,
mode = oct((0o777 ^ mask) & 0o666)
if salt.utils.platform.is_windows():
ret = check_perms(name,
ret,
kwargs.get('win_owner'),
kwargs.get('win_perms'),
kwargs.get('win_deny_perms'),
None,
kwargs.get('win_inheritance'))
# This function resides in win_file.py and will be available
# on Windows. The local function will be overridden
# pylint: disable=E1120,E1121,E1123
ret = check_perms(
path=name,
ret=ret,
owner=kwargs.get('win_owner'),
grant_perms=kwargs.get('win_perms'),
deny_perms=kwargs.get('win_deny_perms'),
inheritance=kwargs.get('win_inheritance', True),
reset=kwargs.get('win_perms_reset', False))
# pylint: enable=E1120,E1121,E1123
else:
ret, _ = check_perms(name, ret, user, group, mode, attrs)

10
salt/modules/rh_ip.py
View File

@ -643,12 +643,18 @@ def _parse_settings_eth(opts, iface_type, enabled, iface):
result[opt] = opts[opt]
if iface_type not in ['bond', 'vlan', 'bridge', 'ipip']:
auto_addr = False
if 'addr' in opts:
if salt.utils.validate.net.mac(opts['addr']):
result['addr'] = opts['addr']
else:
_raise_error_iface(iface, opts['addr'], ['AA:BB:CC:DD:EE:FF'])
elif opts['addr'] == 'auto':
auto_addr = True
elif opts['addr'] != 'none':
_raise_error_iface(iface, opts['addr'], ['AA:BB:CC:DD:EE:FF', 'auto', 'none'])
else:
auto_addr = True
if auto_addr:
# If interface type is slave for bond, not setting hwaddr
if iface_type != 'slave':
ifaces = __salt__['network.interfaces']()

43
salt/modules/saltutil.py
View File

@ -474,7 +474,7 @@ def sync_returners(saltenv=None, refresh=True, extmod_whitelist=None, extmod_bla
'''
.. versionadded:: 0.10.0
Sync beacons from ``salt://_returners`` to the minion
Sync returners from ``salt://_returners`` to the minion
saltenv
The fileserver environment from which to sync. To sync from more than
@ -585,6 +585,44 @@ def sync_engines(saltenv=None, refresh=False, extmod_whitelist=None, extmod_blac
return ret
def sync_thorium(saltenv=None, refresh=False, extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: Oxygen
Sync Thorium modules from ``salt://_thorium`` to the minion
saltenv
The fileserver environment from which to sync. To sync from more than
one environment, pass a comma-separated list.
If not passed, then all environments configured in the :ref:`top files
<states-top>` will be checked for engines to sync. If no top files are
found, then the ``base`` environment will be synced.
refresh: ``True``
If ``True``, refresh the available execution modules on the minion.
This refresh will be performed even if no new Thorium modules are synced.
Set to ``False`` to prevent this refresh.
extmod_whitelist
comma-seperated list of modules to sync
extmod_blacklist
comma-seperated list of modules to blacklist based on type
CLI Examples:
.. code-block:: bash
salt '*' saltutil.sync_thorium
salt '*' saltutil.sync_thorium saltenv=base,dev
'''
ret = _sync('thorium', saltenv, extmod_whitelist, extmod_blacklist)
if refresh:
refresh_modules()
return ret
def sync_output(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blacklist=None):
'''
Sync outputters from ``salt://_output`` to the minion
@ -628,7 +666,7 @@ def sync_clouds(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blackl
'''
.. versionadded:: 2017.7.0
Sync utility modules from ``salt://_cloud`` to the minion
Sync cloud modules from ``salt://_cloud`` to the minion
saltenv : base
The fileserver environment from which to sync. To sync from more than
@ -864,6 +902,7 @@ def sync_all(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blacklist
ret['log_handlers'] = sync_log_handlers(saltenv, False, extmod_whitelist, extmod_blacklist)
ret['proxymodules'] = sync_proxymodules(saltenv, False, extmod_whitelist, extmod_blacklist)
ret['engines'] = sync_engines(saltenv, False, extmod_whitelist, extmod_blacklist)
ret['thorium'] = sync_thorium(saltenv, False, extmod_whitelist, extmod_blacklist)
if __opts__['file_client'] == 'local':
ret['pillar'] = sync_pillar(saltenv, False, extmod_whitelist, extmod_blacklist)
if refresh:

193
salt/modules/schedule.py
View File

@ -10,6 +10,7 @@ Module for managing the Salt schedule on a minion
from __future__ import absolute_import
import copy as pycopy
import difflib
import logging
import os
import yaml
@ -23,7 +24,6 @@ from salt.ext import six
__proxyenabled__ = ['*']
import logging
log = logging.getLogger(__name__)
__func_alias__ = {
@ -58,6 +58,7 @@ SCHEDULE_CONF = [
'return_config',
'return_kwargs',
'run_on_start'
'skip_during_range',
]
@ -353,7 +354,7 @@ def build_schedule_item(name, **kwargs):
for item in ['range', 'when', 'once', 'once_fmt', 'cron',
'returner', 'after', 'return_config', 'return_kwargs',
'until', 'run_on_start']:
'until', 'run_on_start', 'skip_during_range']:
if item in kwargs:
schedule[name][item] = kwargs[item]
@ -951,3 +952,191 @@ def copy(name, target, **kwargs):
ret['minions'] = minions
return ret
return ret
def postpone_job(name, current_time, new_time, **kwargs):
'''
Postpone a job in the minion's schedule
Current time and new time should be specified as Unix timestamps
.. versionadded:: Oxygen
CLI Example:
.. code-block:: bash
salt '*' schedule.postpone_job job current_time new_time
'''
ret = {'comment': [],
'result': True}
if not name:
ret['comment'] = 'Job name is required.'
ret['result'] = False
return ret
if not current_time:
ret['comment'] = 'Job current time is required.'
ret['result'] = False
return ret
else:
if not isinstance(current_time, six.integer_types):
ret['comment'] = 'Job current time must be an integer.'
ret['result'] = False
return ret
if not new_time:
ret['comment'] = 'Job new_time is required.'
ret['result'] = False
return ret
else:
if not isinstance(new_time, six.integer_types):
ret['comment'] = 'Job new time must be an integer.'
ret['result'] = False
return ret
if 'test' in __opts__ and __opts__['test']:
ret['comment'] = 'Job: {0} would be postponed in schedule.'.format(name)
else:
if name in list_(show_all=True, where='opts', return_yaml=False):
event_data = {'name': name,
'time': current_time,
'new_time': new_time,
'func': 'postpone_job'}
elif name in list_(show_all=True, where='pillar', return_yaml=False):
event_data = {'name': name,
'time': current_time,
'new_time': new_time,
'where': 'pillar',
'func': 'postpone_job'}
else:
ret['comment'] = 'Job {0} does not exist.'.format(name)
ret['result'] = False
return ret
try:
eventer = salt.utils.event.get_event('minion', opts=__opts__)
res = __salt__['event.fire'](event_data, 'manage_schedule')
if res:
event_ret = eventer.get_event(tag='/salt/minion/minion_schedule_postpone_job_complete', wait=30)
if event_ret and event_ret['complete']:
schedule = event_ret['schedule']
# check item exists in schedule and is enabled
if name in schedule and schedule[name]['enabled']:
ret['result'] = True
ret['comment'] = 'Postponed Job {0} in schedule.'.format(name)
else:
ret['result'] = False
ret['comment'] = 'Failed to postpone job {0} in schedule.'.format(name)
return ret
except KeyError:
# Effectively a no-op, since we can't really return without an event system
ret['comment'] = 'Event module not available. Schedule postpone job failed.'
return ret
def skip_job(name, time, **kwargs):
'''
Skip a job in the minion's schedule at specified time.
Time to skip should be specified as Unix timestamps
.. versionadded:: Oxygen
CLI Example:
.. code-block:: bash
salt '*' schedule.skip_job job time
'''
ret = {'comment': [],
'result': True}
if not name:
ret['comment'] = 'Job name is required.'
ret['result'] = False
if not time:
ret['comment'] = 'Job time is required.'
ret['result'] = False
if 'test' in __opts__ and __opts__['test']:
ret['comment'] = 'Job: {0} would be skipped in schedule.'.format(name)
else:
if name in list_(show_all=True, where='opts', return_yaml=False):
event_data = {'name': name,
'time': time,
'func': 'skip_job'}
elif name in list_(show_all=True, where='pillar', return_yaml=False):
event_data = {'name': name,
'time': time,
'where': 'pillar',
'func': 'skip_job'}
else:
ret['comment'] = 'Job {0} does not exist.'.format(name)
ret['result'] = False
return ret
try:
eventer = salt.utils.event.get_event('minion', opts=__opts__)
res = __salt__['event.fire'](event_data, 'manage_schedule')
if res:
event_ret = eventer.get_event(tag='/salt/minion/minion_schedule_skip_job_complete', wait=30)
if event_ret and event_ret['complete']:
schedule = event_ret['schedule']
# check item exists in schedule and is enabled
if name in schedule and schedule[name]['enabled']:
ret['result'] = True
ret['comment'] = 'Added Skip Job {0} in schedule.'.format(name)
else:
ret['result'] = False
ret['comment'] = 'Failed to skip job {0} in schedule.'.format(name)
return ret
except KeyError:
# Effectively a no-op, since we can't really return without an event system
ret['comment'] = 'Event module not available. Schedule skip job failed.'
return ret
def show_next_fire_time(name, **kwargs):
'''
Show the next fire time for scheduled job
.. versionadded:: Oxygen
CLI Example:
.. code-block:: bash
salt '*' schedule.show_next_fire_time job_name
'''
ret = {'comment': [],
'result': True}
if not name:
ret['comment'] = 'Job name is required.'
ret['result'] = False
try:
event_data = {'name': name, 'func': 'get_next_fire_time'}
eventer = salt.utils.event.get_event('minion', opts=__opts__)
res = __salt__['event.fire'](event_data,
'manage_schedule')
if res:
event_ret = eventer.get_event(tag='/salt/minion/minion_schedule_next_fire_time_complete', wait=30)
except KeyError:
# Effectively a no-op, since we can't really return without an event system
ret = {}
ret['comment'] = 'Event module not available. Schedule show next fire time failed.'
ret['result'] = True
log.debug(ret['comment'])
return ret
return event_ret

94
salt/modules/state.py
View File

@ -43,6 +43,7 @@ from salt.runners.state import orchestrate as _orchestrate
# Import 3rd-party libs
from salt.ext import six
import msgpack
__proxyenabled__ = ['*']
@ -165,6 +166,99 @@ def _snapper_post(opts, jid, pre_num):
log.error('Failed to create snapper pre snapshot for jid: {0}'.format(jid))
def pause(jid, state_id=None, duration=None):
'''
Set up a state id pause, this instructs a running state to pause at a given
state id. This needs to pass in the jid of the running state and can
optionally pass in a duration in seconds. If a state_id is not passed then
the jid referenced will be paused at the begining of the next state run.
The given state id is the id got a given state execution, so given a state
that looks like this:
.. code-block:: yaml
vim:
pkg.installed: []
The state_id to pass to `pause` is `vim`
CLI Examples:
.. code-block:: bash
salt '*' state.pause 20171130110407769519
salt '*' state.pause 20171130110407769519 vim
salt '*' state.pause 20171130110407769519 vim 20
'''
jid = str(jid)
if state_id is None:
state_id = '__all__'
pause_dir = os.path.join(__opts__[u'cachedir'], 'state_pause')
pause_path = os.path.join(pause_dir, jid)
if not os.path.exists(pause_dir):
try:
os.makedirs(pause_dir)
except OSError:
# File created in the gap
pass
data = {}
if os.path.exists(pause_path):
with salt.utils.files.fopen(pause_path, 'rb') as fp_:
data = msgpack.loads(fp_.read())
if state_id not in data:
data[state_id] = {}
if duration:
data[state_id]['duration'] = int(duration)
with salt.utils.files.fopen(pause_path, 'wb') as fp_:
fp_.write(msgpack.dumps(data))
def resume(jid, state_id=None):
'''
Remove a pause from a jid, allowing it to continue. If the state_id is
not specified then the a general pause will be resumed.
The given state_id is the id got a given state execution, so given a state
that looks like this:
.. code-block:: yaml
vim:
pkg.installed: []
The state_id to pass to `rm_pause` is `vim`
CLI Examples:
.. code-block:: bash
salt '*' state.resume 20171130110407769519
salt '*' state.resume 20171130110407769519 vim
'''
jid = str(jid)
if state_id is None:
state_id = '__all__'
pause_dir = os.path.join(__opts__[u'cachedir'], 'state_pause')
pause_path = os.path.join(pause_dir, jid)
if not os.path.exists(pause_dir):
try:
os.makedirs(pause_dir)
except OSError:
# File created in the gap
pass
data = {}
if os.path.exists(pause_path):
with salt.utils.files.fopen(pause_path, 'rb') as fp_:
data = msgpack.loads(fp_.read())
else:
return True
if state_id in data:
data.pop(state_id)
with salt.utils.files.fopen(pause_path, 'wb') as fp_:
fp_.write(msgpack.dumps(data))
def orchestrate(mods,
saltenv='base',
test=None,

436
salt/modules/win_file.py
View File

@ -1218,41 +1218,55 @@ def mkdir(path,
owner=None,
grant_perms=None,
deny_perms=None,
inheritance=True):
inheritance=True,
reset=False):
'''
Ensure that the directory is available and permissions are set.
Args:
path (str): The full path to the directory.
path (str):
The full path to the directory.
owner (str): The owner of the directory. If not passed, it will be the
account that created the directory, likely SYSTEM
owner (str):
The owner of the directory. If not passed, it will be the account
that created the directory, likely SYSTEM
grant_perms (dict): A dictionary containing the user/group and the basic
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``.
You can also set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to`` setting
like this:
grant_perms (dict):
A dictionary containing the user/group and the basic permissions to
grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
.. code-block:: yaml
.. code-block:: yaml
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
To set advanced permissions use a list for the ``perms`` parameter, ie:
To set advanced permissions use a list for the ``perms`` parameter,
ie:
.. code-block:: yaml
.. code-block:: yaml
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and
permissions to deny along with the ``applies_to`` setting. Use the same
format used for the ``grant_perms`` parameter. Remember, deny
permissions supersede grant permissions.
deny_perms (dict):
A dictionary containing the user/group and permissions to deny along
with the ``applies_to`` setting. Use the same format used for the
``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
inheritance (bool): If True the object will inherit permissions from the
parent, if False, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created
inheritance (bool):
If True the object will inherit permissions from the parent, if
``False``, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created.
reset (bool):
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Returns:
bool: True if successful
@ -1289,10 +1303,16 @@ def mkdir(path,
# Set owner
if owner:
salt.utils.win_dacl.set_owner(path, owner)
salt.utils.win_dacl.set_owner(obj_name=path, principal=owner)
# Set permissions
set_perms(path, grant_perms, deny_perms, inheritance)
set_perms(
path=path,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=inheritance,
reset=reset)
except WindowsError as exc:
raise CommandExecutionError(exc)
@ -1303,49 +1323,63 @@ def makedirs_(path,
owner=None,
grant_perms=None,
deny_perms=None,
inheritance=True):
inheritance=True,
reset=False):
'''
Ensure that the parent directory containing this path is available.
Args:
path (str): The full path to the directory.
path (str):
The full path to the directory.
owner (str): The owner of the directory. If not passed, it will be the
account that created the directly, likely SYSTEM
.. note::
grant_perms (dict): A dictionary containing the user/group and the basic
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``.
You can also set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to`` setting
like this:
The path must end with a trailing slash otherwise the
directory(s) will be created up to the parent directory. For
example if path is ``C:\\temp\\test``, then it would be treated
as ``C:\\temp\\`` but if the path ends with a trailing slash
like ``C:\\temp\\test\\``, then it would be treated as
``C:\\temp\\test\\``.
.. code-block:: yaml
owner (str):
The owner of the directory. If not passed, it will be the account
that created the directly, likely SYSTEM
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
grant_perms (dict):
A dictionary containing the user/group and the basic permissions to
grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
To set advanced permissions use a list for the ``perms`` parameter, ie:
.. code-block:: yaml
.. code-block:: yaml
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
To set advanced permissions use a list for the ``perms`` parameter, ie:
deny_perms (dict): A dictionary containing the user/group and
permissions to deny along with the ``applies_to`` setting. Use the same
format used for the ``grant_perms`` parameter. Remember, deny
permissions supersede grant permissions.
.. code-block:: yaml
inheritance (bool): If True the object will inherit permissions from the
parent, if False, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
.. note::
deny_perms (dict):
A dictionary containing the user/group and permissions to deny along
with the ``applies_to`` setting. Use the same format used for the
``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
The path must end with a trailing slash otherwise the directory(s) will
be created up to the parent directory. For example if path is
``C:\\temp\\test``, then it would be treated as ``C:\\temp\\`` but if
the path ends with a trailing slash like ``C:\\temp\\test\\``, then it
would be treated as ``C:\\temp\\test\\``.
inheritance (bool):
If True the object will inherit permissions from the parent, if
False, inheritance will be disabled. Inheritance setting will not
apply to parent directories if they must be created.
reset (bool):
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Returns:
bool: True if successful
@ -1405,7 +1439,13 @@ def makedirs_(path,
for directory_to_create in directories_to_create:
# all directories have the user, group and mode set!!
log.debug('Creating directory: %s', directory_to_create)
mkdir(directory_to_create, owner, grant_perms, deny_perms, inheritance)
mkdir(
path=directory_to_create,
owner=owner,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=inheritance,
reset=reset)
return True
@ -1414,41 +1454,54 @@ def makedirs_perms(path,
owner=None,
grant_perms=None,
deny_perms=None,
inheritance=True):
inheritance=True,
reset=True):
'''
Set owner and permissions for each directory created.
Args:
path (str): The full path to the directory.
path (str):
The full path to the directory.
owner (str): The owner of the directory. If not passed, it will be the
account that created the directory, likely SYSTEM
owner (str):
The owner of the directory. If not passed, it will be the account
that created the directory, likely SYSTEM
grant_perms (dict): A dictionary containing the user/group and the basic
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``.
You can also set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to`` setting
like this:
grant_perms (dict):
A dictionary containing the user/group and the basic permissions to
grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
.. code-block:: yaml
.. code-block:: yaml
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
To set advanced permissions use a list for the ``perms`` parameter, ie:
To set advanced permissions use a list for the ``perms`` parameter, ie:
.. code-block:: yaml
.. code-block:: yaml
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and
permissions to deny along with the ``applies_to`` setting. Use the same
format used for the ``grant_perms`` parameter. Remember, deny
permissions supersede grant permissions.
deny_perms (dict):
A dictionary containing the user/group and permissions to deny along
with the ``applies_to`` setting. Use the same format used for the
``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
inheritance (bool): If True the object will inherit permissions from the
parent, if False, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created
inheritance (bool):
If ``True`` the object will inherit permissions from the parent, if
``False``, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created
reset (bool):
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Returns:
bool: True if successful, otherwise raise an error
@ -1482,8 +1535,15 @@ def makedirs_perms(path,
try:
# Create the directory here, set inherited True because this is a
# parent directory, the inheritance setting will only apply to the
# child directory
makedirs_perms(head, owner, grant_perms, deny_perms, True)
# target directory. Reset will be False as we only want to reset
# the permissions on the target directory
makedirs_perms(
path=head,
owner=owner,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=True,
reset=False)
except OSError as exc:
# be happy if someone already created the path
if exc.errno != errno.EEXIST:
@ -1492,7 +1552,13 @@ def makedirs_perms(path,
return {}
# Make the directory
mkdir(path, owner, grant_perms, deny_perms, inheritance)
mkdir(
path=path,
owner=owner,
grant_perms=grant_perms,
deny_perms=deny_perms,
inheritance=inheritance,
reset=reset)
return True
@ -1502,66 +1568,64 @@ def check_perms(path,
owner=None,
grant_perms=None,
deny_perms=None,
inheritance=True):
inheritance=True,
reset=False):
'''
Set owner and permissions for each directory created.
Check owner and permissions for the passed directory. This function checks
the permissions and sets them, returning the changes made.
Args:
path (str): The full path to the directory.
path (str):
The full path to the directory.
ret (dict): A dictionary to append changes to and return. If not passed,
will create a new dictionary to return.
ret (dict):
A dictionary to append changes to and return. If not passed, will
create a new dictionary to return.
owner (str): The owner of the directory. If not passed, it will be the
account that created the directory, likely SYSTEM
owner (str):
The owner to set for the directory.
grant_perms (dict): A dictionary containing the user/group and the basic
permissions to grant, ie: ``{'user': {'perms': 'basic_permission'}}``.
You can also set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to`` setting
like this:
grant_perms (dict):
A dictionary containing the user/group and the basic permissions to
check/grant, ie: ``{'user': {'perms': 'basic_permission'}}``.
Default is ``None``.
.. code-block:: yaml
deny_perms (dict):
A dictionary containing the user/group and permissions to
check/deny. Default is ``None``.
{'user': {'perms': 'full_control', 'applies_to': 'this_folder'}}
inheritance (bool):
``True will check if inheritance is enabled and enable it. ``False``
will check if inheritance is disabled and disable it. Defaultl is
``True``.
To set advanced permissions use a list for the ``perms`` parameter, ie:
.. code-block:: yaml
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
deny_perms (dict): A dictionary containing the user/group and
permissions to deny along with the ``applies_to`` setting. Use the same
format used for the ``grant_perms`` parameter. Remember, deny
permissions supersede grant permissions.
inheritance (bool): If True the object will inherit permissions from the
parent, if False, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created
reset (bool):
``True`` wil show what permisisons will be removed by resetting the
DACL. ``False`` will do nothing. Default is ``False``.
Returns:
bool: True if successful, otherwise raise an error
dict: A dictionary of changes that have been made
CLI Example:
.. code-block:: bash
# To grant the 'Users' group 'read & execute' permissions.
salt '*' file.check_perms C:\\Temp\\ Administrators "{'Users': {'perms': 'read_execute'}}"
# To see changes to ``C:\\Temp`` if the 'Users' group is given 'read & execute' permissions.
salt '*' file.check_perms C:\\Temp\\ {} Administrators "{'Users': {'perms': 'read_execute'}}"
# Locally using salt call
salt-call file.check_perms C:\\Temp\\ Administrators "{'Users': {'perms': 'read_execute', 'applies_to': 'this_folder_only'}}"
salt-call file.check_perms C:\\Temp\\ {} Administrators "{'Users': {'perms': 'read_execute', 'applies_to': 'this_folder_only'}}"
# Specify advanced attributes with a list
salt '*' file.check_perms C:\\Temp\\ Administrators "{'jsnuffy': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'files_only'}}"
salt '*' file.check_perms C:\\Temp\\ {} Administrators "{'jsnuffy': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'files_only'}}"
'''
path = os.path.expanduser(path)
if not ret:
ret = {'name': path,
'changes': {},
'pchanges': {},
'comment': [],
'result': True}
orig_comment = ''
@ -1571,14 +1635,16 @@ def check_perms(path,
# Check owner
if owner:
owner = salt.utils.win_dacl.get_name(owner)
current_owner = salt.utils.win_dacl.get_owner(path)
owner = salt.utils.win_dacl.get_name(principal=owner)
current_owner = salt.utils.win_dacl.get_owner(obj_name=path)
if owner != current_owner:
if __opts__['test'] is True:
ret['pchanges']['owner'] = owner
else:
try:
salt.utils.win_dacl.set_owner(path, owner)
salt.utils.win_dacl.set_owner(
obj_name=path,
principal=owner)
ret['changes']['owner'] = owner
except CommandExecutionError:
ret['result'] = False
@ -1586,7 +1652,7 @@ def check_perms(path,
'Failed to change owner to "{0}"'.format(owner))
# Check permissions
cur_perms = salt.utils.win_dacl.get_permissions(path)
cur_perms = salt.utils.win_dacl.get_permissions(obj_name=path)
# Verify Deny Permissions
changes = {}
@ -1594,7 +1660,7 @@ def check_perms(path,
for user in deny_perms:
# Check that user exists:
try:
user_name = salt.utils.win_dacl.get_name(user)
user_name = salt.utils.win_dacl.get_name(principal=user)
except CommandExecutionError:
ret['comment'].append(
'Deny Perms: User "{0}" missing from Target System'.format(user))
@ -1619,7 +1685,11 @@ def check_perms(path,
# Check Perms
if isinstance(deny_perms[user]['perms'], six.string_types):
if not salt.utils.win_dacl.has_permission(
path, user, deny_perms[user]['perms'], 'deny'):
obj_name=path,
principal=user,
permission=deny_perms[user]['perms'],
access_mode='deny',
exact=False):
changes[user] = {'perms': deny_perms[user]['perms']}
else:
for perm in deny_perms[user]['perms']:
@ -1640,9 +1710,10 @@ def check_perms(path,
changes[user]['applies_to'] = applies_to
if changes:
ret['pchanges']['deny_perms'] = {}
ret['changes']['deny_perms'] = {}
for user in changes:
user_name = salt.utils.win_dacl.get_name(user)
user_name = salt.utils.win_dacl.get_name(principal=user)
if __opts__['test'] is True:
ret['pchanges']['deny_perms'][user] = changes[user]
@ -1689,7 +1760,11 @@ def check_perms(path,
try:
salt.utils.win_dacl.set_permissions(
path, user, perms, 'deny', applies_to)
obj_name=path,
principal=user,
permissions=perms,
access_mode='deny',
applies_to=applies_to)
ret['changes']['deny_perms'][user] = changes[user]
except CommandExecutionError:
ret['result'] = False
@ -1703,7 +1778,7 @@ def check_perms(path,
for user in grant_perms:
# Check that user exists:
try:
user_name = salt.utils.win_dacl.get_name(user)
user_name = salt.utils.win_dacl.get_name(principal=user)
except CommandExecutionError:
ret['comment'].append(
'Grant Perms: User "{0}" missing from Target System'.format(user))
@ -1729,12 +1804,19 @@ def check_perms(path,
# Check Perms
if isinstance(grant_perms[user]['perms'], six.string_types):
if not salt.utils.win_dacl.has_permission(
path, user, grant_perms[user]['perms']):
obj_name=path,
principal=user,
permission=grant_perms[user]['perms'],
access_mode='grant'):
changes[user] = {'perms': grant_perms[user]['perms']}
else:
for perm in grant_perms[user]['perms']:
if not salt.utils.win_dacl.has_permission(
path, user, perm, exact=False):
obj_name=path,
principal=user,
permission=perm,
access_mode='grant',
exact=False):
if user not in changes:
changes[user] = {'perms': []}
changes[user]['perms'].append(grant_perms[user]['perms'])
@ -1750,11 +1832,12 @@ def check_perms(path,
changes[user]['applies_to'] = applies_to
if changes:
ret['pchanges']['grant_perms'] = {}
ret['changes']['grant_perms'] = {}
for user in changes:
user_name = salt.utils.win_dacl.get_name(user)
user_name = salt.utils.win_dacl.get_name(principal=user)
if __opts__['test'] is True:
ret['changes']['grant_perms'][user] = changes[user]
ret['pchanges']['grant_perms'][user] = changes[user]
else:
applies_to = None
if 'applies_to' not in changes[user]:
@ -1796,7 +1879,11 @@ def check_perms(path,
try:
salt.utils.win_dacl.set_permissions(
path, user, perms, 'grant', applies_to)
obj_name=path,
principal=user,
permissions=perms,
access_mode='grant',
applies_to=applies_to)
ret['changes']['grant_perms'][user] = changes[user]
except CommandExecutionError:
ret['result'] = False
@ -1806,12 +1893,14 @@ def check_perms(path,
# Check inheritance
if inheritance is not None:
if not inheritance == salt.utils.win_dacl.get_inheritance(path):
if not inheritance == salt.utils.win_dacl.get_inheritance(obj_name=path):
if __opts__['test'] is True:
ret['changes']['inheritance'] = inheritance
ret['pchanges']['inheritance'] = inheritance
else:
try:
salt.utils.win_dacl.set_inheritance(path, inheritance)
salt.utils.win_dacl.set_inheritance(
obj_name=path,
enabled=inheritance)
ret['changes']['inheritance'] = inheritance
except CommandExecutionError:
ret['result'] = False
@ -1819,6 +1908,45 @@ def check_perms(path,
'Failed to set inheritance for "{0}" to '
'{1}'.format(path, inheritance))
# Check reset
# If reset=True, which users will be removed as a result
if reset:
for user_name in cur_perms:
if user_name not in grant_perms:
if 'grant' in cur_perms[user_name] and not \
cur_perms[user_name]['grant']['inherited']:
if __opts__['test'] is True:
if 'remove_perms' not in ret['pchanges']:
ret['pchanges']['remove_perms'] = {}
ret['pchanges']['remove_perms'].update(
{user_name: cur_perms[user_name]})
else:
if 'remove_perms' not in ret['changes']:
ret['changes']['remove_perms'] = {}
salt.utils.win_dacl.rm_permissions(
obj_name=path,
principal=user_name,
ace_type='grant')
ret['changes']['remove_perms'].update(
{user_name: cur_perms[user_name]})
if user_name not in deny_perms:
if 'deny' in cur_perms[user_name] and not \
cur_perms[user_name]['deny']['inherited']:
if __opts__['test'] is True:
if 'remove_perms' not in ret['pchanges']:
ret['pchanges']['remove_perms'] = {}
ret['pchanges']['remove_perms'].update(
{user_name: cur_perms[user_name]})
else:
if 'remove_perms' not in ret['changes']:
ret['changes']['remove_perms'] = {}
salt.utils.win_dacl.rm_permissions(
obj_name=path,
principal=user_name,
ace_type='deny')
ret['changes']['remove_perms'].update(
{user_name: cur_perms[user_name]})
# Re-add the Original Comment if defined
if isinstance(orig_comment, six.string_types):
if orig_comment:
@ -1830,25 +1958,30 @@ def check_perms(path,
ret['comment'] = '\n'.join(ret['comment'])
# Set result for test = True
if __opts__['test'] is True and ret['changes']:
if __opts__['test'] and (ret['changes'] or ret['pchanges']):
ret['result'] = None
return ret
def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
def set_perms(path,
grant_perms=None,
deny_perms=None,
inheritance=True,
reset=False):
'''
Set permissions for the given path
Args:
path (str): The full path to the directory.
path (str):
The full path to the directory.
grant_perms (dict):
A dictionary containing the user/group and the basic permissions to
grant, ie: ``{'user': {'perms': 'basic_permission'}}``. You can also
set the ``applies_to`` setting here. The default is
``this_folder_subfolders_files``. Specify another ``applies_to``
set the ``applies_to`` setting here. The default for ``applise_to``
is ``this_folder_subfolders_files``. Specify another ``applies_to``
setting like this:
.. code-block:: yaml
@ -1863,7 +1996,10 @@ def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
{'user': {'perms': ['read_attributes', 'read_ea'], 'applies_to': 'this_folder'}}
To see a list of available attributes and applies to settings see
the documentation for salt.utils.win_dacl
the documentation for salt.utils.win_dacl.
A value of ``None`` will make no changes to the ``grant`` portion of
the DACL. Default is ``None``.
deny_perms (dict):
A dictionary containing the user/group and permissions to deny along
@ -1871,13 +2007,27 @@ def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
``grant_perms`` parameter. Remember, deny permissions supersede
grant permissions.
A value of ``None`` will make no changes to the ``deny`` portion of
the DACL. Default is ``None``.
inheritance (bool):
If True the object will inherit permissions from the parent, if
False, inheritance will be disabled. Inheritance setting will not
apply to parent directories if they must be created
If ``True`` the object will inherit permissions from the parent, if
``False``, inheritance will be disabled. Inheritance setting will
not apply to parent directories if they must be created. Default is
``False``.
reset (bool):
If ``True`` the existing DCL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded: Oxygen
Returns:
bool: True if successful, otherwise raise an error
bool: True if successful
Raises:
CommandExecutionError: If unsuccessful
CLI Example:
@ -1894,11 +2044,19 @@ def set_perms(path, grant_perms=None, deny_perms=None, inheritance=True):
'''
ret = {}
# Get the DACL for the directory
dacl = salt.utils.win_dacl.dacl(path)
if reset:
# Get an empty DACL
dacl = salt.utils.win_dacl.dacl()
# Get current file/folder permissions
cur_perms = salt.utils.win_dacl.get_permissions(path)
# Get an empty perms dict
cur_perms = {}
else:
# Get the DACL for the directory
dacl = salt.utils.win_dacl.dacl(path)
# Get current file/folder permissions
cur_perms = salt.utils.win_dacl.get_permissions(path)
# Set 'deny' perms if any
if deny_perms is not None:

14
salt/modules/yumpkg.py
View File

@ -279,11 +279,23 @@ def _get_extra_options(**kwargs):
'''
ret = []
kwargs = salt.utils.args.clean_kwargs(**kwargs)
# Remove already handled options from kwargs
fromrepo = kwargs.pop('fromrepo', '')
repo = kwargs.pop('repo', '')
disablerepo = kwargs.pop('disablerepo', '')
enablerepo = kwargs.pop('enablerepo', '')
disable_excludes = kwargs.pop('disableexcludes', '')
branch = kwargs.pop('branch', '')
for key, value in six.iteritems(kwargs):
if isinstance(key, six.string_types):
if isinstance(value, six.string_types):
log.info('Adding extra option --%s=\'%s\'', key, value)
ret.append('--{0}=\'{1}\''.format(key, value))
elif value is True:
log.info('Adding extra option --%s', key)
ret.append('--{0}'.format(key))
log.info('Adding extra options %s', ret)
return ret

4
salt/returners/local_cache.py
View File

@ -294,9 +294,11 @@ def get_load(jid):
if not os.path.exists(jid_dir) or not os.path.exists(load_fn):
return {}
serial = salt.payload.Serial(__opts__)
ret = {}
with salt.utils.files.fopen(os.path.join(jid_dir, LOAD_P), 'rb') as rfh:
ret = serial.load(rfh)
if ret is None:
ret = {}
minions_cache = [os.path.join(jid_dir, MINIONS_P)]
minions_cache.extend(
glob.glob(os.path.join(jid_dir, SYNDIC_MINIONS_P.format('*')))

7
salt/runner.py
View File

@ -43,6 +43,7 @@ class RunnerClient(mixins.SyncClientMixin, mixins.AsyncClientMixin, object):
def __init__(self, opts):
self.opts = opts
self.context = {}
@property
def functions(self):
@ -51,11 +52,13 @@ class RunnerClient(mixins.SyncClientMixin, mixins.AsyncClientMixin, object):
self.utils = salt.loader.utils(self.opts)
# Must be self.functions for mixin to work correctly :-/
try:
self._functions = salt.loader.runner(self.opts, utils=self.utils)
self._functions = salt.loader.runner(
self.opts, utils=self.utils, context=self.context)
except AttributeError:
# Just in case self.utils is still not present (perhaps due to
# problems with the loader), load the runner funcs without them
self._functions = salt.loader.runner(self.opts)
self._functions = salt.loader.runner(
self.opts, context=self.context)
return self._functions

37
salt/runners/saltutil.py
View File

@ -52,6 +52,7 @@ def sync_all(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
ret['runners'] = sync_runners(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['wheel'] = sync_wheel(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['engines'] = sync_engines(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['thorium'] = sync_thorium(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['queues'] = sync_queues(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['pillar'] = sync_pillar(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
ret['utils'] = sync_utils(saltenv=saltenv, extmod_whitelist=extmod_whitelist, extmod_blacklist=extmod_blacklist)
@ -303,6 +304,32 @@ def sync_engines(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
extmod_blacklist=extmod_blacklist)[0]
def sync_thorium(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: Oxygen
Sync Thorium from ``salt://_thorium`` to the master
saltenv: ``base``
The fileserver environment from which to sync. To sync from more than
one environment, pass a comma-separated list.
extmod_whitelist
comma-seperated list of modules to sync
extmod_blacklist
comma-seperated list of modules to blacklist based on type
CLI Example:
.. code-block:: bash
salt-run saltutil.sync_thorium
'''
return salt.utils.extmods.sync(__opts__, 'thorium', saltenv=saltenv, extmod_whitelist=extmod_whitelist,
extmod_blacklist=extmod_blacklist)[0]
def sync_queues(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
Sync queue modules from ``salt://_queues`` to the master
@ -381,7 +408,7 @@ def sync_sdb(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: 2017.7.0
Sync utils modules from ``salt://_sdb`` to the master
Sync sdb modules from ``salt://_sdb`` to the master
saltenv : base
The fileserver environment from which to sync. To sync from more than
@ -427,7 +454,7 @@ def sync_cache(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: 2017.7.0
Sync utils modules from ``salt://_cache`` to the master
Sync cache modules from ``salt://_cache`` to the master
saltenv : base
The fileserver environment from which to sync. To sync from more than
@ -453,7 +480,7 @@ def sync_fileserver(saltenv='base', extmod_whitelist=None, extmod_blacklist=None
'''
.. versionadded:: Oxygen
Sync utils modules from ``salt://_fileserver`` to the master
Sync fileserver modules from ``salt://_fileserver`` to the master
saltenv : base
The fileserver environment from which to sync. To sync from more than
@ -479,7 +506,7 @@ def sync_clouds(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: 2017.7.0
Sync utils modules from ``salt://_clouds`` to the master
Sync cloud modules from ``salt://_clouds`` to the master
saltenv : base
The fileserver environment from which to sync. To sync from more than
@ -505,7 +532,7 @@ def sync_roster(saltenv='base', extmod_whitelist=None, extmod_blacklist=None):
'''
.. versionadded:: 2017.7.0
Sync utils modules from ``salt://_roster`` to the master
Sync roster modules from ``salt://_roster`` to the master
saltenv : base
The fileserver environment from which to sync. To sync from more than

6
salt/runners/sdb.py
View File

@ -22,7 +22,7 @@ def get(uri):
.. code-block:: bash
salt '*' sdb.get sdb://mymemcached/foo
salt-run sdb.get sdb://mymemcached/foo
'''
return salt.utils.sdb.sdb_get(uri, __opts__, __utils__)
@ -37,7 +37,7 @@ def set_(uri, value):
.. code-block:: bash
salt '*' sdb.set sdb://mymemcached/foo bar
salt-run sdb.set sdb://mymemcached/foo bar
'''
return salt.utils.sdb.sdb_set(uri, value, __opts__, __utils__)
@ -52,7 +52,7 @@ def delete(uri):
.. code-block:: bash
salt '*' sdb.delete sdb://mymemcached/foo
salt-run sdb.delete sdb://mymemcached/foo
'''
return salt.utils.sdb.sdb_delete(uri, __opts__, __utils__)

18
salt/runners/state.py
View File

@ -15,6 +15,24 @@ from salt.exceptions import SaltInvocationError
LOGGER = logging.getLogger(__name__)
def set_pause(jid, state_id, duration=None):
'''
Set up a state id pause, this instructs a running state to pause at a given
state id. This needs to pass in the jid of the running state and can
optionally pass in a duration in seconds.
'''
minion = salt.minion.MasterMinion(__opts__)
minion['state.set_pause'](jid, state_id, duration)
def rm_pause(jid, state_id, duration=None):
'''
Remove a pause from a jid, allowing it to continue
'''
minion = salt.minion.MasterMinion(__opts__)
minion['state.rm_pause'](jid, state_id)
def orchestrate(mods,
saltenv='base',
test=None,

52
salt/state.py
View File

@ -1918,6 +1918,8 @@ class State(object):
if self.mocked:
ret = mock_ret(cdata)
else:
# Check if this low chunk is paused
self.check_pause(low)
# Execute the state function
if not low.get(u'__prereq__') and low.get(u'parallel'):
# run the state call in parallel, but only if not in a prereq
@ -2127,6 +2129,48 @@ class State(object):
return not running[tag][u'result']
return False
def check_pause(self, low):
'''
Check to see if this low chunk has been paused
'''
if not self.jid:
# Can't pause on salt-ssh since we can't track continuous state
return
pause_path = os.path.join(self.opts[u'cachedir'], 'state_pause', self.jid)
start = time.time()
if os.path.isfile(pause_path):
try:
while True:
tries = 0
with salt.utils.files.fopen(pause_path, 'rb') as fp_:
try:
pdat = msgpack.loads(fp_.read())
except msgpack.UnpackValueError:
# Reading race condition
if tries > 10:
# Break out if there are a ton of read errors
return
tries += 1
time.sleep(1)
continue
id_ = low[u'__id__']
key = u''
if id_ in pdat:
key = id_
elif u'__all__' in pdat:
key = u'__all__'
if key:
if u'duration' in pdat[key]:
now = time.time()
if now - start > pdat[key][u'duration']:
return
else:
return
time.sleep(1)
except Exception as exc:
log.error('Failed to read in pause data for file located at: %s', pause_path)
return
def reconcile_procs(self, running):
'''
Check the running dict for processes and resolve them
@ -2682,6 +2726,14 @@ class State(object):
except OSError:
log.debug(u'File %s does not exist, no need to cleanup', accum_data_path)
_cleanup_accumulator_data()
if self.jid is not None:
pause_path = os.path.join(self.opts[u'cachedir'], u'state_pause', self.jid)
if os.path.isfile(pause_path):
try:
os.remove(pause_path)
except OSError:
# File is not present, all is well
pass
return ret

139
salt/states/file.py
View File

@ -761,7 +761,8 @@ def _check_directory_win(name,
win_owner,
win_perms=None,
win_deny_perms=None,
win_inheritance=None):
win_inheritance=None,
win_perms_reset=None):
'''
Check what changes need to be made on a directory
'''
@ -879,6 +880,20 @@ def _check_directory_win(name,
if not win_inheritance == salt.utils.win_dacl.get_inheritance(name):
changes['inheritance'] = win_inheritance
# Check reset
if win_perms_reset:
for user_name in perms:
if user_name not in win_perms:
if 'grant' in perms[user_name] and not perms[user_name]['grant']['inherited']:
if 'remove_perms' not in changes:
changes['remove_perms'] = {}
changes['remove_perms'].update({user_name: perms[user_name]})
if user_name not in win_deny_perms:
if 'deny' in perms[user_name] and not perms[user_name]['deny']['inherited']:
if 'remove_perms' not in changes:
changes['remove_perms'] = {}
changes['remove_perms'].update({user_name: perms[user_name]})
if changes:
return None, 'The directory "{0}" will be changed'.format(name), changes
@ -1566,6 +1581,7 @@ def managed(name,
win_perms=None,
win_deny_perms=None,
win_inheritance=True,
win_perms_reset=False,
**kwargs):
r'''
Manage a given file, this function allows for a file to be downloaded from
@ -2072,6 +2088,13 @@ def managed(name,
.. versionadded:: 2017.7.0
win_perms_reset : False
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Here's an example using the above ``win_*`` parameters:
.. code-block:: yaml
@ -2314,8 +2337,13 @@ def managed(name,
# Check and set the permissions if necessary
if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms'](
name, ret, win_owner, win_perms, win_deny_perms, None,
win_inheritance)
path=name,
ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else:
ret, _ = __salt__['file.check_perms'](
name, ret, user, group, mode, attrs, follow_symlinks)
@ -2356,8 +2384,13 @@ def managed(name,
if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms'](
name, ret, win_owner, win_perms, win_deny_perms, None,
win_inheritance)
path=name,
ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
if isinstance(ret['pchanges'], tuple):
ret['result'], ret['comment'] = ret['pchanges']
@ -2448,6 +2481,7 @@ def managed(name,
win_perms=win_perms,
win_deny_perms=win_deny_perms,
win_inheritance=win_inheritance,
win_perms_reset=win_perms_reset,
encoding=encoding,
encoding_errors=encoding_errors,
**kwargs)
@ -2517,6 +2551,7 @@ def managed(name,
win_perms=win_perms,
win_deny_perms=win_deny_perms,
win_inheritance=win_inheritance,
win_perms_reset=win_perms_reset,
encoding=encoding,
encoding_errors=encoding_errors,
**kwargs)
@ -2590,6 +2625,7 @@ def directory(name,
win_perms=None,
win_deny_perms=None,
win_inheritance=True,
win_perms_reset=False,
**kwargs):
r'''
Ensure that a named directory is present and has the right perms
@ -2751,6 +2787,13 @@ def directory(name,
.. versionadded:: 2017.7.0
win_perms_reset : False
If ``True`` the existing DACL will be cleared and replaced with the
settings defined in this function. If ``False``, new entries will be
appended to the existing DACL. Default is ``False``.
.. versionadded:: Oxygen
Here's an example using the above ``win_*`` parameters:
.. code-block:: yaml
@ -2855,13 +2898,23 @@ def directory(name,
elif force:
# Remove whatever is in the way
if os.path.isfile(name):
os.remove(name)
ret['changes']['forced'] = 'File was forcibly replaced'
if __opts__['test']:
ret['pchanges']['forced'] = 'File was forcibly replaced'
else:
os.remove(name)
ret['changes']['forced'] = 'File was forcibly replaced'
elif __salt__['file.is_link'](name):
__salt__['file.remove'](name)
ret['changes']['forced'] = 'Symlink was forcibly replaced'
if __opts__['test']:
ret['pchanges']['forced'] = 'Symlink was forcibly replaced'
else:
__salt__['file.remove'](name)
ret['changes']['forced'] = 'Symlink was forcibly replaced'
else:
__salt__['file.remove'](name)
if __opts__['test']:
ret['pchanges']['forced'] = 'Directory was forcibly replaced'
else:
__salt__['file.remove'](name)
ret['changes']['forced'] = 'Directory was forcibly replaced'
else:
if os.path.isfile(name):
return _error(
@ -2874,17 +2927,26 @@ def directory(name,
# Check directory?
if salt.utils.platform.is_windows():
presult, pcomment, ret['pchanges'] = _check_directory_win(
name, win_owner, win_perms, win_deny_perms, win_inheritance)
presult, pcomment, pchanges = _check_directory_win(
name=name,
win_owner=win_owner,
win_perms=win_perms,
win_deny_perms=win_deny_perms,
win_inheritance=win_inheritance,
win_perms_reset=win_perms_reset)
else:
presult, pcomment, ret['pchanges'] = _check_directory(
presult, pcomment, pchanges = _check_directory(
name, user, group, recurse or [], dir_mode, clean, require,
exclude_pat, max_depth, follow_symlinks)
if __opts__['test']:
if pchanges:
ret['pchanges'].update(pchanges)
# Don't run through the reset of the function if there are no changes to be
# made
if not ret['pchanges'] or __opts__['test']:
ret['result'] = presult
ret['comment'] = pcomment
ret['changes'] = ret['pchanges']
return ret
if not os.path.isdir(name):
@ -2900,8 +2962,13 @@ def directory(name,
if not os.path.isdir(drive):
return _error(
ret, 'Drive {0} is not mapped'.format(drive))
__salt__['file.makedirs'](name, win_owner, win_perms,
win_deny_perms, win_inheritance)
__salt__['file.makedirs'](
path=name,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else:
__salt__['file.makedirs'](name, user=user, group=group,
mode=dir_mode)
@ -2910,8 +2977,13 @@ def directory(name,
ret, 'No directory to create {0} in'.format(name))
if salt.utils.platform.is_windows():
__salt__['file.mkdir'](name, win_owner, win_perms, win_deny_perms,
win_inheritance)
__salt__['file.mkdir'](
path=name,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else:
__salt__['file.mkdir'](name, user=user, group=group, mode=dir_mode)
@ -2925,7 +2997,13 @@ def directory(name,
if not children_only:
if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms'](
name, ret, win_owner, win_perms, win_deny_perms, None, win_inheritance)
path=name,
ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else:
ret, perms = __salt__['file.check_perms'](
name, ret, user, group, dir_mode, None, follow_symlinks)
@ -2996,8 +3074,13 @@ def directory(name,
try:
if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms'](
full, ret, win_owner, win_perms, win_deny_perms, None,
win_inheritance)
path=full,
ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else:
ret, _ = __salt__['file.check_perms'](
full, ret, user, group, file_mode, None, follow_symlinks)
@ -3011,8 +3094,13 @@ def directory(name,
try:
if salt.utils.platform.is_windows():
ret = __salt__['file.check_perms'](
full, ret, win_owner, win_perms, win_deny_perms, None,
win_inheritance)
path=full,
ret=ret,
owner=win_owner,
grant_perms=win_perms,
deny_perms=win_deny_perms,
inheritance=win_inheritance,
reset=win_perms_reset)
else:
ret, _ = __salt__['file.check_perms'](
full, ret, user, group, dir_mode, None, follow_symlinks)
@ -3034,7 +3122,8 @@ def directory(name,
if children_only:
ret['comment'] = u'Directory {0}/* updated'.format(name)
else:
ret['comment'] = u'Directory {0} updated'.format(name)
if ret['changes']:
ret['comment'] = u'Directory {0} updated'.format(name)
if __opts__['test']:
ret['comment'] = 'Directory {0} not updated'.format(name)

51
salt/states/saltmod.py
View File

@ -787,28 +787,15 @@ def runner(name, **kwargs):
runner_return = out.get('return')
if isinstance(runner_return, dict) and 'Error' in runner_return:
out['success'] = False
if not out.get('success', True):
cmt = "Runner function '{0}' failed{1}.".format(
name,
' with return {0}'.format(runner_return) if runner_return else '',
)
ret = {
'name': name,
'result': False,
'changes': {},
'comment': cmt,
}
else:
cmt = "Runner function '{0}' executed{1}.".format(
name,
' with return {0}'.format(runner_return) if runner_return else '',
)
ret = {
'name': name,
'result': True,
'changes': {},
'comment': cmt,
}
success = out.get('success', True)
ret = {'name': name,
'changes': {'return': runner_return},
'result': success}
ret['comment'] = "Runner function '{0}' {1}.".format(
name,
'executed' if success else 'failed',
)
ret['__orchestration__'] = True
if 'jid' in out:
@ -1039,15 +1026,21 @@ def wheel(name, **kwargs):
__env__=__env__,
**kwargs)
ret['result'] = True
wheel_return = out.get('return')
if isinstance(wheel_return, dict) and 'Error' in wheel_return:
out['success'] = False
success = out.get('success', True)
ret = {'name': name,
'changes': {'return': wheel_return},
'result': success}
ret['comment'] = "Wheel function '{0}' {1}.".format(
name,
'executed' if success else 'failed',
)
ret['__orchestration__'] = True
if 'jid' in out:
ret['__jid__'] = out['jid']
runner_return = out.get('return')
ret['comment'] = "Wheel function '{0}' executed{1}.".format(
name,
' with return {0}'.format(runner_return) if runner_return else '',
)
return ret

6
salt/utils/args.py
View File

@ -451,10 +451,10 @@ def format_call(fun,
continue
extra[key] = copy.deepcopy(value)
# We'll be showing errors to the users until Salt Oxygen comes out, after
# We'll be showing errors to the users until Salt Fluorine comes out, after
# which, errors will be raised instead.
salt.utils.versions.warn_until(
'Oxygen',
'Fluorine',
'It\'s time to start raising `SaltInvocationError` instead of '
'returning warnings',
# Let's not show the deprecation warning on the console, there's no
@ -491,7 +491,7 @@ def format_call(fun,
'{0}. If you were trying to pass additional data to be used '
'in a template context, please populate \'context\' with '
'\'key: value\' pairs. Your approach will work until Salt '
'Oxygen is out.{1}'.format(
'Fluorine is out.{1}'.format(
msg,
'' if 'full' not in ret else ' Please update your state files.'
)

208
salt/utils/schedule.py
View File

@ -89,6 +89,28 @@ localtime.
This will schedule the command: ``state.sls httpd test=True`` at 5:00 PM on
Monday, Wednesday and Friday, and 3:00 PM on Tuesday and Thursday.
.. code-block:: yaml
schedule:
job1:
function: state.sls
args:
- httpd
kwargs:
test: True
when:
- 'tea time'
.. code-block:: yaml
whens:
tea time: 1:40pm
deployment time: Friday 5:00pm
The Salt scheduler also allows custom phrases to be used for the `when`
parameter. These `whens` can be stored as either pillar values or
grain values.
.. code-block:: yaml
schedule:
@ -333,7 +355,6 @@ import logging
import errno
import random
import yaml
import copy
# Import Salt libs
import salt.config
@ -409,6 +430,7 @@ class Schedule(object):
self.proxy = proxy
self.functions = functions
self.standalone = standalone
self.skip_function = None
if isinstance(intervals, dict):
self.intervals = intervals
else:
@ -745,6 +767,69 @@ class Schedule(object):
evt.fire_event({'complete': True},
tag='/salt/minion/minion_schedule_saved')
def postpone_job(self, name, data):
'''
Postpone a job in the scheduler.
Ignores jobs from pillar
'''
time = data['time']
new_time = data['new_time']
# ensure job exists, then disable it
if name in self.opts['schedule']:
if 'skip_explicit' not in self.opts['schedule'][name]:
self.opts['schedule'][name]['skip_explicit'] = []
self.opts['schedule'][name]['skip_explicit'].append(time)
if 'run_explicit' not in self.opts['schedule'][name]:
self.opts['schedule'][name]['run_explicit'] = []
self.opts['schedule'][name]['run_explicit'].append(new_time)
elif name in self._get_schedule(include_opts=False):
log.warning('Cannot modify job {0}, '
'it`s in the pillar!'.format(name))
# Fire the complete event back along with updated list of schedule
evt = salt.utils.event.get_event('minion', opts=self.opts, listen=False)
evt.fire_event({'complete': True, 'schedule': self._get_schedule()},
tag='/salt/minion/minion_schedule_postpone_job_complete')
def skip_job(self, name, data):
'''
Skip a job at a specific time in the scheduler.
Ignores jobs from pillar
'''
time = data['time']
# ensure job exists, then disable it
if name in self.opts['schedule']:
if 'skip_explicit' not in self.opts['schedule'][name]:
self.opts['schedule'][name]['skip_explicit'] = []
self.opts['schedule'][name]['skip_explicit'].append(time)
elif name in self._get_schedule(include_opts=False):
log.warning('Cannot modify job {0}, '
'it`s in the pillar!'.format(name))
# Fire the complete event back along with updated list of schedule
evt = salt.utils.event.get_event('minion', opts=self.opts, listen=False)
evt.fire_event({'complete': True, 'schedule': self._get_schedule()},
tag='/salt/minion/minion_schedule_skip_job_complete')
def get_next_fire_time(self, name):
'''
Disable a job in the scheduler. Ignores jobs from pillar
'''
schedule = self._get_schedule()
if schedule:
_next_fire_time = schedule[name]['_next_fire_time']
# Fire the complete event back along with updated list of schedule
evt = salt.utils.event.get_event('minion', opts=self.opts, listen=False)
evt.fire_event({'complete': True, 'next_fire_time': _next_fire_time},
tag='/salt/minion/minion_schedule_next_fire_time_complete')
def handle_func(self, multiprocessing_enabled, func, data):
'''
Execute this method in a multiprocess or thread
@ -948,11 +1033,16 @@ class Schedule(object):
# Let's make sure we exit the process!
sys.exit(salt.defaults.exitcodes.EX_GENERIC)
def eval(self):
def eval(self, now=None):
'''
Evaluate and execute the schedule
:param int now: Override current time with a Unix timestamp``
'''
log.trace('==== evaluating schedule =====')
def _splay(splaytime):
'''
Calculate splaytime
@ -974,9 +1064,13 @@ class Schedule(object):
raise ValueError('Schedule must be of type dict.')
if 'enabled' in schedule and not schedule['enabled']:
return
if 'skip_function' in schedule:
self.skip_function = schedule['skip_function']
for job, data in six.iteritems(schedule):
if job == 'enabled' or not data:
continue
if job == 'skip_function' or not data:
continue
if not isinstance(data, dict):
log.error('Scheduled job "{0}" should have a dict value, not {1}'.format(job, type(data)))
continue
@ -1011,7 +1105,8 @@ class Schedule(object):
'_run_on_start' not in data:
data['_run_on_start'] = True
now = int(time.time())
if not now:
now = int(time.time())
if 'until' in data:
if not _WHEN_SUPPORTED:
@ -1065,6 +1160,23 @@ class Schedule(object):
'", "'.join(scheduling_elements)))
continue
if 'run_explicit' in data:
_run_explicit = data['run_explicit']
if isinstance(_run_explicit, six.string_types):
_run_explicit = [_run_explicit]
# Copy the list so we can loop through it
for i in copy.deepcopy(_run_explicit):
if len(_run_explicit) > 1:
if i < now - self.opts['loop_interval']:
_run_explicit.remove(i)
if _run_explicit:
if _run_explicit[0] <= now < (_run_explicit[0] + self.opts['loop_interval']):
run = True
data['_next_fire_time'] = _run_explicit[0]
if True in [True for item in time_elements if item in data]:
if '_seconds' not in data:
interval = int(data.get('seconds', 0))
@ -1153,10 +1265,11 @@ class Schedule(object):
# Copy the list so we can loop through it
for i in copy.deepcopy(_when):
if i < now and len(_when) > 1:
# Remove all missed schedules except the latest one.
# We need it to detect if it was triggered previously.
_when.remove(i)
if len(_when) > 1:
if i < now - self.opts['loop_interval']:
# Remove all missed schedules except the latest one.
# We need it to detect if it was triggered previously.
_when.remove(i)
if _when:
# Grab the first element, which is the next run time or
@ -1258,19 +1371,21 @@ class Schedule(object):
seconds = data['_next_fire_time'] - now
if data['_splay']:
seconds = data['_splay'] - now
if seconds <= 0:
if '_seconds' in data:
if '_seconds' in data:
if seconds <= 0:
run = True
elif 'when' in data and data['_run']:
elif 'when' in data and data['_run']:
if data['_next_fire_time'] <= now <= (data['_next_fire_time'] + self.opts['loop_interval']):
data['_run'] = False
run = True
elif 'cron' in data:
# Reset next scheduled time because it is in the past now,
# and we should trigger the job run, then wait for the next one.
elif 'cron' in data:
# Reset next scheduled time because it is in the past now,
# and we should trigger the job run, then wait for the next one.
if seconds <= 0:
data['_next_fire_time'] = None
run = True
elif seconds == 0:
run = True
elif seconds == 0:
run = True
if '_run_on_start' in data and data['_run_on_start']:
run = True
@ -1312,7 +1427,11 @@ class Schedule(object):
if start <= now <= end:
run = True
else:
run = False
if self.skip_function:
run = True
func = self.skip_function
else:
run = False
else:
log.error('schedule.handle_func: Invalid range, end must be larger than start. \
Ignoring job {0}.'.format(job))
@ -1322,6 +1441,62 @@ class Schedule(object):
Ignoring job {0}.'.format(job))
continue
if 'skip_during_range' in data:
if not _RANGE_SUPPORTED:
log.error('Missing python-dateutil. Ignoring job {0}'.format(job))
continue
else:
if isinstance(data['skip_during_range'], dict):
try:
start = int(time.mktime(dateutil_parser.parse(data['skip_during_range']['start']).timetuple()))
except ValueError:
log.error('Invalid date string for start in skip_during_range. Ignoring job {0}.'.format(job))
continue
try:
end = int(time.mktime(dateutil_parser.parse(data['skip_during_range']['end']).timetuple()))
except ValueError:
log.error('Invalid date string for end in skip_during_range. Ignoring job {0}.'.format(job))
log.error(data)
continue
if end > start:
if start <= now <= end:
if self.skip_function:
run = True
func = self.skip_function
else:
run = False
else:
run = True
else:
log.error('schedule.handle_func: Invalid range, end must be larger than start. \
Ignoring job {0}.'.format(job))
continue
else:
log.error('schedule.handle_func: Invalid, range must be specified as a dictionary. \
Ignoring job {0}.'.format(job))
continue
if 'skip_explicit' in data:
_skip_explicit = data['skip_explicit']
if isinstance(_skip_explicit, six.string_types):
_skip_explicit = [_skip_explicit]
# Copy the list so we can loop through it
for i in copy.deepcopy(_skip_explicit):
if i < now - self.opts['loop_interval']:
_skip_explicit.remove(i)
if _skip_explicit:
if _skip_explicit[0] <= now <= (_skip_explicit[0] + self.opts['loop_interval']):
if self.skip_function:
run = True
func = self.skip_function
else:
run = False
else:
run = True
if not run:
continue
@ -1374,6 +1549,7 @@ class Schedule(object):
finally:
if '_seconds' in data:
data['_next_fire_time'] = now + data['_seconds']
data['_last_run'] = now
data['_splay'] = None
if salt.utils.platform.is_windows():
# Restore our function references.

3
salt/wheel/__init__.py
View File

@ -43,7 +43,8 @@ class WheelClient(salt.client.mixins.SyncClientMixin,
def __init__(self, opts=None):
self.opts = opts
self.functions = salt.loader.wheels(opts)
self.context = {}
self.functions = salt.loader.wheels(opts, context=self.context)
# TODO: remove/deprecate
def call_func(self, fun, **kwargs):

16
tests/integration/files/file/base/_runners/runtests_helpers.py Normal file
View File

@ -0,0 +1,16 @@
# -*- coding: utf-8 -*-
'''
Runner functions for integration tests
'''
# Import python libs
from __future__ import absolute_import
def failure():
__context__['retcode'] = 1
return False
def success():
return True

16
tests/integration/files/file/base/_wheel/runtests_helpers.py Normal file
View File

@ -0,0 +1,16 @@
# -*- coding: utf-8 -*-
'''
Wheel functions for integration tests
'''
# Import python libs
from __future__ import absolute_import
def failure():
__context__['retcode'] = 1
return False
def success():
return True

15
tests/integration/files/file/base/orch/retcode.sls Normal file
View File

@ -0,0 +1,15 @@
test_runner_success:
salt.runner:
- name: runtests_helpers.success
test_runner_failure:
salt.runner:
- name: runtests_helpers.failure
test_wheel_success:
salt.wheel:
- name: runtests_helpers.success
test_wheel_failure:
salt.wheel:
- name: runtests_helpers.failure

12
tests/integration/modules/test_saltutil.py
View File

@ -93,7 +93,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [],
'sdb': [],
'proxymodules': [],
'output': []}
'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all')
self.assertEqual(ret, expected_return)
@ -113,7 +114,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [],
'sdb': [],
'proxymodules': [],
'output': []}
'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all', extmod_whitelist={'modules': ['salttest']})
self.assertEqual(ret, expected_return)
@ -135,7 +137,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [],
'sdb': [],
'proxymodules': [],
'output': []}
'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all', extmod_blacklist={'modules': ['runtests_decorators']})
self.assertEqual(ret, expected_return)
@ -155,7 +158,8 @@ class SaltUtilSyncModuleTest(ModuleCase):
'states': [],
'sdb': [],
'proxymodules': [],
'output': []}
'output': [],
'thorium': []}
ret = self.run_function('saltutil.sync_all', extmod_whitelist={'modules': ['runtests_decorators']},
extmod_blacklist={'modules': ['runtests_decorators']})
self.assertEqual(ret, expected_return)

29
tests/integration/runners/test_state.py
View File

@ -106,6 +106,35 @@ class StateRunnerTest(ShellCase):
for item in out:
self.assertIn(item, ret)
def test_orchestrate_retcode(self):
'''
Test orchestration with nonzero retcode set in __context__
'''
self.run_run('saltutil.sync_runners')
self.run_run('saltutil.sync_wheel')
ret = '\n'.join(self.run_run('state.orchestrate orch.retcode'))
for result in (' ID: test_runner_success\n'
' Function: salt.runner\n'
' Name: runtests_helpers.success\n'
' Result: True',
' ID: test_runner_failure\n'
' Function: salt.runner\n'
' Name: runtests_helpers.failure\n'
' Result: False',
' ID: test_wheel_success\n'
' Function: salt.wheel\n'
' Name: runtests_helpers.success\n'
' Result: True',
' ID: test_wheel_failure\n'
' Function: salt.wheel\n'
' Name: runtests_helpers.failure\n'
' Result: False'):
self.assertIn(result, ret)
def test_orchestrate_target_doesnt_exists(self):
'''
test orchestration when target doesnt exist

23
tests/unit/modules/test_ansiblegate.py
View File

@ -126,3 +126,26 @@ description:
patch('salt.modules.ansiblegate.importlib.import_module', lambda x: x):
with pytest.raises(LoaderError) as loader_error:
self.resolver.load_module('something.strange')
def test_virtual_function_no_ansible_installed(self):
'''
Test Ansible module __virtual__ when ansible is not installed on the minion.
:return:
'''
with patch('salt.modules.ansiblegate.ansible', None):
assert ansible.__virtual__() == (False, 'Ansible is not installed on this system')
@patch('salt.modules.ansiblegate.ansible', MagicMock())
@patch('salt.modules.ansiblegate.list', MagicMock())
@patch('salt.modules.ansiblegate._set_callables', MagicMock())
@patch('salt.modules.ansiblegate.AnsibleModuleCaller', MagicMock())
def test_virtual_function_ansible_is_installed(self):
'''
Test Ansible module __virtual__ when ansible is installed on the minion.
:return:
'''
resolver = MagicMock()
resolver.resolve = MagicMock()
resolver.resolve.install = MagicMock()
with patch('salt.modules.ansiblegate.AnsibleModuleResolver', resolver):
assert ansible.__virtual__() == (True, None)

7
tests/unit/states/test_file.py
View File

@ -815,7 +815,7 @@ class TestFileState(TestCase, LoaderModuleMockMixin):
'comment': comt,
'result': None,
'pchanges': p_chg,
'changes': {'/etc/grub.conf': {'directory': 'new'}}
'changes': {}
})
self.assertDictEqual(filestate.directory(name,
user=user,
@ -841,6 +841,11 @@ class TestFileState(TestCase, LoaderModuleMockMixin):
ret)
recurse = ['ignore_files', 'ignore_dirs']
ret.update({'comment': 'Must not specify "recurse" '
'options "ignore_files" and '
'"ignore_dirs" at the same '
'time.',
'pchanges': {}})
with patch.object(os.path, 'isdir', mock_t):
self.assertDictEqual(filestate.directory
(name, user=user,

8
tests/unit/states/test_saltmod.py
View File

@ -258,8 +258,8 @@ class SaltmodTestCase(TestCase, LoaderModuleMockMixin):
'''
name = 'state'
ret = {'changes': {}, 'name': 'state', 'result': True,
'comment': 'Runner function \'state\' executed with return True.',
ret = {'changes': {'return': True}, 'name': 'state', 'result': True,
'comment': 'Runner function \'state\' executed.',
'__orchestration__': True}
runner_mock = MagicMock(return_value={'return': True})
@ -274,8 +274,8 @@ class SaltmodTestCase(TestCase, LoaderModuleMockMixin):
'''
name = 'state'
ret = {'changes': {}, 'name': 'state', 'result': True,
'comment': 'Wheel function \'state\' executed with return True.',
ret = {'changes': {'return': True}, 'name': 'state', 'result': True,
'comment': 'Wheel function \'state\' executed.',
'__orchestration__': True}
wheel_mock = MagicMock(return_value={'return': True})