valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-06 08:35:21 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch '2018.3' into 'fluorine'

Browse Source 
Conflicts:
  - salt/utils/mac_utils.py
  - salt/utils/win_runas.py
  - tests/support/case.py
This commit is contained in:
Ch3LL 2018-11-14 12:38:16 -05:00
commit 8501581f27
No known key found for this signature in database
GPG Key ID: 132B55A7C13EFA73
26 changed files with 509 additions and 306 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.kitchen.yml
View File

@ -30,7 +30,7 @@ provisioner:
salt_install: bootstrap
salt_version: latest
salt_bootstrap_url: https://bootstrap.saltstack.com
salt_bootstrap_options: -X -p rsync stable <%= version %>
salt_bootstrap_options: -X -p rsync git <%= version %>
log_level: info
sudo: true
require_chef: false

3
Gemfile
View File

@ -2,7 +2,8 @@
source 'https://rubygems.org'
gem 'test-kitchen', '~>1.21'
# Point this back at the test-kitchen package after 1.23.3 is relased
gem 'test-kitchen', :git => 'https://github.com/dwoz/test-kitchen.git', :branch => 'winrm_opts'
gem 'kitchen-salt', '~>0.2'
gem 'kitchen-sync'
gem 'git'

2
doc/_themes/saltstack2/layout.html vendored
View File

@ -256,7 +256,7 @@
<!--
<a href="https://saltstack.com/saltstack-enterprise/" target="_blank"><img class="nolightbox footer-banner center" src="{{ pathto('_static/images/enterprise_ad.jpg', 1) }}"/></a>
-->
<a href="http://saltconf.com" target="_blank"><img class="nolightbox footer-banner center" src="{{ pathto('_static/images/DOCBANNER.jpg', 1) }}"/></a>
<a href="http://saltconf.com/saltconf18-speakers/" target="_blank"><img class="nolightbox footer-banner center" src="{{ pathto('_static/images/DOCBANNER.png', 1) }}"/></a>
</div>
{% endif %}
</div>

BIN
doc/_themes/saltstack2/static/images/DOCBANNER.jpg vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 497 KiB

BIN
doc/_themes/saltstack2/static/images/DOCBANNER.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 767 KiB

4
doc/ref/clients/index.rst
View File

@ -98,13 +98,13 @@ RunnerClient
------------
.. autoclass:: salt.runner.RunnerClient
:members: cmd, async, cmd_sync, cmd_async
:members: cmd, asynchronous, cmd_sync, cmd_async
WheelClient
-----------
.. autoclass:: salt.wheel.WheelClient
:members: cmd, async, cmd_sync, cmd_async
:members: cmd, asynchronous, cmd_sync, cmd_async
CloudClient
-----------

16
doc/topics/development/contributing.rst
View File

@ -526,6 +526,19 @@ GPG key with ``git`` locally, and linking the GPG key to your GitHub account.
Once these steps are completed, the commit signing verification will look like
the example in GitHub's `GPG Signature Verification feature announcement`_.
Bootstrap Script Changes
------------------------
Salt's Bootstrap Script, known as `bootstrap-salt.sh`_ in the Salt repo, has it's own
repository, contributing guidelines, and release cadence.
All changes to the Bootstrap Script should be made to `salt-bootstrap repo`_. Any
pull requests made to the `bootstrap-salt.sh`_ file in the Salt repository will be
automatically overwritten upon the next stable release of the Bootstrap Script.
For more information on the release process or how to contribute to the Bootstrap
Script, see the Bootstrap Script's `Contributing Guidelines`_.
.. _`saltstack/salt`: https://github.com/saltstack/salt
.. _`GitHub Fork a Repo Guide`: https://help.github.com/articles/fork-a-repo
.. _`GitHub issue tracker`: https://github.com/saltstack/salt/issues
@ -537,3 +550,6 @@ the example in GitHub's `GPG Signature Verification feature announcement`_.
.. _GPG Probot: https://probot.github.io/apps/gpg/
.. _help articles: https://help.github.com/articles/signing-commits-with-gpg/
.. _GPG Signature Verification feature announcement: https://github.com/blog/2144-gpg-signature-verification
.. _bootstrap-salt.sh: https://github.com/saltstack/salt/blob/develop/salt/cloud/deploy/bootstrap-salt.sh
.. _salt-bootstrap repo: https://github.com/saltstack/salt-bootstrap
.. _Contributing Guidelines: https://github.com/saltstack/salt-bootstrap/blob/develop/CONTRIBUTING.md

14
doc/topics/releases/2018.3.3.rst
View File

@ -92,7 +92,7 @@ Example:
cmd.run 'echo '\''h=\"baz\"'\''' runas=macuser
Changelog for v2018.3.2..v2018.3.3
=================================================================
==================================
*Generated at: 2018-09-21 17:45:27 UTC*
@ -507,7 +507,7 @@ Changelog for v2018.3.2..v2018.3.3
* 3d26affa10 Fix remaining file state integration tests (py3)
* **PR** `#49171`_: (`Ch3LL`_) [2018.3.3] cherry pick `#49103`_
* **PR** `#49171`_: (`Ch3LL`_) [2018.3.3] cherry pick `#49103`_
@ *2018-08-17 20:23:32 UTC*
* **PR** `#49103`_: (`dwoz`_) Install the launcher so we can execute py files (refs: `#49171`_)
@ -1630,7 +1630,7 @@ Changelog for v2018.3.2..v2018.3.3
* **ISSUE** `#46896`_: (`Poil`_) Proxy + file.managed => Comment: Failed to cache xxx invalid arguments to setopt (refs: `#48754`_)
* **PR** `#48754`_: (`lomeroe`_) send proxy/ca_cert parameters as strings (not unicode) to tornado httpclient
* **PR** `#48754`_: (`lomeroe`_) send proxy/ca_cert parameters as strings (not unicode) to tornado httpclient
@ *2018-07-25 14:55:42 UTC*
* 030c921914 Merge pull request `#48754`_ from lomeroe/fix-tornado-proxy
@ -3075,7 +3075,7 @@ Changelog for v2018.3.2..v2018.3.3
* dae65da256 Merge branch '2018.3.1' into '2018.3'
* **PR** `#48186`_: (`rallytime`_) Add autodoc module for saltcheck.py
* **PR** `#48186`_: (`rallytime`_) Add autodoc module for saltcheck.py
@ *2018-06-19 19:03:55 UTC*
* 5b4897f050 Merge pull request `#48186`_ from rallytime/saltcheck-docs
@ -3362,11 +3362,11 @@ Changelog for v2018.3.2..v2018.3.3
* **PR** `#48109`_: (`rallytime`_) Back-port `#47851`_ to 2018.3
@ *2018-06-14 13:09:04 UTC*
* **PR** `#47851`_: (`rares-pop`_) Fixup! add master.py:FileserverUpdate **kwargs (refs: `#48109`_)
* **PR** `#47851`_: (`rares-pop`_) Fixup! add master.py:FileserverUpdate \*\*kwargs (refs: `#48109`_)
* 2902ee0b14 Merge pull request `#48109`_ from rallytime/bp-47851
* e9dc30bf8e Fixup! add master.py:FileserverUpdate **kwargs
* e9dc30bf8e Fixup! add master.py:FileserverUpdate \*\*kwargs
* **ISSUE** `#47925`_: (`JonGriggs`_) GitFS looking for files in the master branch only (refs: `#47943`_)
@ -3377,7 +3377,7 @@ Changelog for v2018.3.2..v2018.3.3
* 534e1a7100 Merge branch '2018.3' into issue47925
* **PR** `#48089`_: (`rallytime`_) Update release versions for the 2018.3 branch
* **PR** `#48089`_: (`rallytime`_) Update release versions for the 2018.3 branch
@ *2018-06-13 14:03:44 UTC*
* 9e1d0040e4 Merge pull request `#48089`_ from rallytime/update_version_doc_2018.3

8
salt/modules/dockermod.py
View File

@ -948,6 +948,9 @@ def compare_containers(first, second, ignore=None):
if item == 'Ulimits':
val1 = _ulimit_sort(val1)
val2 = _ulimit_sort(val2)
if item == 'Env':
val1 = sorted(val1)
val2 = sorted(val2)
if val1 != val2:
ret.setdefault(conf_dict, {})[item] = {'old': val1, 'new': val2}
# Check for optionally-present items that were in the second container
@ -974,6 +977,9 @@ def compare_containers(first, second, ignore=None):
if item == 'Ulimits':
val1 = _ulimit_sort(val1)
val2 = _ulimit_sort(val2)
if item == 'Env':
val1 = sorted(val1)
val2 = sorted(val2)
if val1 != val2:
ret.setdefault(conf_dict, {})[item] = {'old': val1, 'new': val2}
return ret
@ -5644,6 +5650,7 @@ def pause(name):
.format(name))}
return _change_state(name, 'pause', 'paused')
freeze = salt.utils.functools.alias_function(pause, 'freeze')
@ -5846,6 +5853,7 @@ def unpause(name):
.format(name))}
return _change_state(name, 'unpause', 'running')
unfreeze = salt.utils.functools.alias_function(unpause, 'unfreeze')

14
salt/modules/win_lgpo.py
View File

@ -664,6 +664,20 @@ class _policy_info(object):
},
'Transform': self.enabled_one_disabled_zero_transform,
},
'RestrictRemoteSAM': {
'Policy': 'Network access: Restrict clients allowed to '
'make remote calls to SAM',
'lgpo_section': self.security_options_gpedit_path,
'Registry': {
'Hive': 'HKEY_LOCAL_MACHINE',
'Path': 'System\\CurrentControlSet\\Control\\Lsa',
'Value': 'RestrictRemoteSAM',
'Type': 'REG_SZ'
},
'Transform': {
'Put': '_string_put_transform'
}
},
'RestrictAnonymous': {
'Policy': 'Network access: Do not allow anonymous '
'enumeration of SAM accounts and shares',

25
salt/pillar/saltclass.py
View File

@ -29,19 +29,20 @@ def __virtual__():
def ext_pillar(minion_id, pillar, *args, **kwargs):
'''
Node definitions path will be retrieved from args - or set to default -
then added to 'salt_data' dict that is passed to the 'get_pillars' function.
'salt_data' dict is a convenient way to pass all the required datas to the function
It contains:
- __opts__
- __salt__
- __grains__
- __pillar__
- minion_id
- path
If successfull the function will return a pillar dict for minion_id
Compile pillar data
'''
# Node definitions path will be retrieved from args (or set to default),
# then added to 'salt_data' dict that is passed to the 'get_pillars'
# function. The dictionary contains:
# - __opts__
# - __salt__
# - __grains__
# - __pillar__
# - minion_id
# - path
#
# If successful, the function will return a pillar dict for minion_id.
# If path has not been set, make a default
for i in args:
if 'path' not in i:

2
salt/states/ansiblegate.py
View File

@ -16,8 +16,6 @@
# limitations under the License.
r'''
:codeauthor: :email:`Bo Maryniuk <bo@suse.de>`
Execution of Ansible modules from within states
===============================================

67
salt/states/git.py
View File

@ -284,8 +284,8 @@ def latest(name,
identity=None,
https_user=None,
https_pass=None,
onlyif=False,
unless=False,
onlyif=None,
unless=None,
refspec_branch='*',
refspec_tag='*',
output_encoding=None,
@ -2231,8 +2231,8 @@ def detached(name,
identity=None,
https_user=None,
https_pass=None,
onlyif=False,
unless=False,
onlyif=None,
unless=None,
output_encoding=None,
**kwargs):
'''
@ -3430,18 +3430,65 @@ def mod_run_check(cmd_kwargs, onlyif, unless):
Otherwise, returns ``True``
'''
cmd_kwargs = copy.deepcopy(cmd_kwargs)
cmd_kwargs['python_shell'] = True
if onlyif:
if __salt__['cmd.retcode'](onlyif, **cmd_kwargs) != 0:
cmd_kwargs.update({
'use_vt': False,
'bg': False,
'ignore_retcode': True,
'python_shell': True,
})
if onlyif is not None:
if not isinstance(onlyif, list):
onlyif = [onlyif]
for command in onlyif:
if not isinstance(command, six.string_types) and command:
# Boolean or some other non-string which resolves to True
continue
try:
if __salt__['cmd.retcode'](command, **cmd_kwargs) == 0:
# Command exited with a zero retcode
continue
except Exception as exc:
log.exception(
'The following onlyif command raised an error: %s',
command
)
return {
'comment': 'onlyif raised error ({0}), see log for '
'more details'.format(exc),
'result': False
}
return {'comment': 'onlyif condition is false',
'skip_watch': True,
'result': True}
if unless:
if __salt__['cmd.retcode'](unless, **cmd_kwargs) == 0:
if unless is not None:
if not isinstance(unless, list):
unless = [unless]
for command in unless:
if not isinstance(command, six.string_types) and not command:
# Boolean or some other non-string which resolves to False
break
try:
if __salt__['cmd.retcode'](command, **cmd_kwargs) != 0:
# Command exited with a non-zero retcode
break
except Exception as exc:
log.exception(
'The following unless command raised an error: %s',
command
)
return {
'comment': 'unless raised error ({0}), see log for '
'more details'.format(exc),
'result': False
}
else:
return {'comment': 'unless condition is true',
'skip_watch': True,
'result': True}
# No reason to stop, return True
return True

27
salt/tops/saltclass.py
View File

@ -108,7 +108,7 @@ Saltclass Examples
``<saltclass_path>/nodes/lausanne/qls.node1.yml``
.. code-block:: yaml
.. code-block:: jinja
environment: base
@ -228,19 +228,20 @@ def __virtual__():
def top(**kwargs):
'''
Node definitions path will be retrieved from __opts__ - or set to default -
then added to 'salt_data' dict that is passed to the 'get_tops' function.
'salt_data' dict is a convenient way to pass all the required datas to the function
It contains:
- __opts__
- empty __salt__
- __grains__
- empty __pillar__
- minion_id
- path
If successfull the function will return a top dict for minion_id
Compile tops
'''
# Node definitions path will be retrieved from args (or set to default),
# then added to 'salt_data' dict that is passed to the 'get_pillars'
# function. The dictionary contains:
# - __opts__
# - __salt__
# - __grains__
# - __pillar__
# - minion_id
# - path
#
# If successful, the function will return a pillar dict for minion_id.
# If path has not been set, make a default
_opts = __opts__['master_tops']['saltclass']
if 'path' not in _opts:

4
salt/utils/dictupdate.py
View File

@ -15,8 +15,7 @@ except ImportError:
# Import 3rd-party libs
import copy
import logging
from salt.ext import six
from salt.serializers.yamlex import merge_recursive as _yamlex_merge_recursive
import salt.ext.six as six
log = logging.getLogger(__name__)
@ -94,6 +93,7 @@ def merge_recurse(obj_a, obj_b, merge_lists=False):
def merge_aggregate(obj_a, obj_b):
from salt.serializers.yamlex import merge_recursive as _yamlex_merge_recursive
return _yamlex_merge_recursive(obj_a, obj_b, level=1)

10
salt/utils/job.py
View File

@ -103,10 +103,12 @@ def store_job(opts, load, event=None, mminion=None):
log.error(emsg)
raise KeyError(emsg)
try:
mminion.returners[savefstr](load['jid'], load)
except KeyError as e:
log.error("Load does not contain 'jid': %s", e)
if job_cache != 'local_cache':
try:
mminion.returners[savefstr](load['jid'], load)
except KeyError as e:
log.error("Load does not contain 'jid': %s", e)
mminion.returners[fstr](load)
if (opts.get('job_cache_store_endtime')

72
salt/utils/mac_utils.py
View File

@ -11,6 +11,7 @@ import subprocess
import os
import plistlib
import time
import xml.parsers.expat
try:
import pwd
except ImportError:
@ -45,6 +46,11 @@ __salt__ = {
'cmd.run': salt.modules.cmdmod._run_quiet,
}
if six.PY2:
class InvalidFileException(Exception):
pass
plistlib.InvalidFileException = InvalidFileException
def __virtual__():
'''
@ -306,6 +312,12 @@ def launchctl(sub_cmd, *args, **kwargs):
def _available_services(refresh=False):
'''
This is a helper function for getting the available macOS services.
The strategy is to look through the known system locations for
launchd plist files, parse them, and use their information for
populating the list of services. Services can run without a plist
file present, but normally services which have an automated startup
will have a plist file, so this is a minor compromise.
'''
try:
if __context__['available_services'] and not refresh:
@ -324,7 +336,7 @@ def _available_services(refresh=False):
try:
for user in os.listdir('/Users/'):
agent_path = '/Users/{}/Library/LaunchAgents/'.format(user)
agent_path = '/Users/{}/Library/LaunchAgents'.format(user)
if os.path.isdir(agent_path):
launchd_paths.append(agent_path)
except OSError:
@ -342,39 +354,59 @@ def _available_services(refresh=False):
# Follow symbolic links of files in _launchd_paths
file_path = os.path.join(root, file_name)
true_path = os.path.realpath(file_path)
log.trace('Gathering service info for %s', true_path)
# ignore broken symlinks
if not os.path.exists(true_path):
continue
try:
# This assumes most of the plist files
# will be already in XML format
plist = plistlib.readPlist(true_path)
if six.PY2:
# py2 plistlib can't read binary plists, and
# uses a different API than py3.
plist = plistlib.readPlist(true_path)
else:
with salt.utils.files.fopen(true_path, 'rb') as handle:
plist = plistlib.load(handle)
except Exception:
# If plistlib is unable to read the file we'll need to use
# the system provided plutil program to do the conversion
except plistlib.InvalidFileException:
# Raised in python3 if the file is not XML.
# There's nothing we can do; move on to the next one.
msg = 'Unable to parse "%s" as it is invalid XML: InvalidFileException.'
logging.warning(msg, true_path)
continue
except xml.parsers.expat.ExpatError:
# Raised by py2 for all errors.
# Raised by py3 if the file is XML, but with errors.
if six.PY3:
# There's an error in the XML, so move on.
msg = 'Unable to parse "%s" as it is invalid XML: xml.parsers.expat.ExpatError.'
logging.warning(msg, true_path)
continue
# Use the system provided plutil program to attempt
# conversion from binary.
cmd = '/usr/bin/plutil -convert xml1 -o - -- "{0}"'.format(
true_path)
plist_xml = __salt__['cmd.run'](cmd)
if six.PY2:
try:
plist_xml = __salt__['cmd.run'](cmd)
plist = plistlib.readPlistFromString(plist_xml)
else:
plist = plistlib.loads(
salt.utils.stringutils.to_bytes(plist_xml))
except xml.parsers.expat.ExpatError:
# There's still an error in the XML, so move on.
msg = 'Unable to parse "%s" as it is invalid XML: xml.parsers.expat.ExpatError.'
logging.warning(msg, true_path)
continue
try:
_available_services[plist.Label.lower()] = {
'file_name': file_name,
'file_path': true_path,
'plist': plist}
except AttributeError:
# Handle malformed plist files
_available_services[os.path.basename(file_name).lower()] = {
# not all launchd plists contain a Label key
_available_services[plist['Label'].lower()] = {
'file_name': file_name,
'file_path': true_path,
'plist': plist}
except KeyError:
log.debug('Service %s does not contain a'
' Label key. Skipping.', true_path)
continue
# put this in __context__ as this is a time consuming function.
# a fix for this issue. https://github.com/saltstack/salt/issues/48414

4
salt/utils/timed_subprocess.py
View File

@ -42,8 +42,8 @@ class TimedProc(object):
if self.timeout and not isinstance(self.timeout, (int, float)):
raise salt.exceptions.TimedProcTimeoutError('Error: timeout {0} must be a number'.format(self.timeout))
if six.PY2 and kwargs.get('shell', False):
args = salt.utils.stringutils.to_bytes(args)
if kwargs.get('shell', False):
args = salt.utils.data.decode(args, to_str=True)
try:
self.process = subprocess.Popen(args, **kwargs)

1
salt/utils/win_runas.py
View File

@ -233,7 +233,6 @@ def runas_unpriv(cmd, username, password, cwd=None):
'''
Runas that works for non-priviledged users
'''
# Create a pipe to set as stdout in the child. The write handle needs to be
# inheritable.
c2pread, c2pwrite = salt.platform.win.CreatePipe(

22
tests/integration/minion/test_timeout.py
View File

@ -5,6 +5,10 @@ Tests for various minion timeouts
# Import Python libs
from __future__ import absolute_import
import os
import sys
import salt.utils.platform
# Import Salt Testing libs
from tests.support.case import ShellCase
@ -21,8 +25,18 @@ class MinionTimeoutTestCase(ShellCase):
'''
# Launch the command
sleep_length = 30
ret = self.run_salt('minion test.sleep {0}'.format(sleep_length), timeout=45)
self.assertTrue(isinstance(ret, list), 'Return is not a list. Minion'
if salt.utils.platform.is_windows():
popen_kwargs = {'env': dict(os.environ, PYTHONPATH=';'.join(sys.path))}
else:
popen_kwargs = None
ret = self.run_salt(
'minion test.sleep {0}'.format(sleep_length),
timeout=45,
catch_stderr=True,
popen_kwargs=popen_kwargs,
)
self.assertTrue(isinstance(ret[0], list), 'Return is not a list. Minion'
' may have returned error: {0}'.format(ret))
self.assertTrue('True' in ret[1], 'Minion did not return True after '
'{0} seconds.'.format(sleep_length))
self.assertEqual(len(ret[0]), 2, 'Standard out wrong length {}'.format(ret))
self.assertTrue('True' in ret[0][1], 'Minion did not return True after '
'{0} seconds. ret={1}'.format(sleep_length, ret))

7
tests/integration/modules/test_system.py
View File

@ -367,6 +367,12 @@ class WinSystemModuleTest(ModuleCase):
'''
Validate the date/time functions in the win_system module
'''
@classmethod
def tearDownClass(cls):
if subprocess.call('w32tm /resync', shell=True) != 0:
log.error("Re-syncing time failed")
def test_get_computer_name(self):
'''
Test getting the computer name
@ -400,6 +406,7 @@ class WinSystemModuleTest(ModuleCase):
@flaky
@destructiveTest
@flaky
def test_set_system_time(self):
'''
Test setting the system time

37
tests/integration/runners/test_jobs.py
View File

@ -45,3 +45,40 @@ class ManageTest(ShellCase):
'''
ret = self.run_run_plus('jobs.list_jobs')
self.assertIsInstance(ret['return'], dict)
class LocalCacheTargetTest(ShellCase):
'''
Test that a job stored in the local_cache has target information
'''
def test_target_info(self):
'''
This is a test case for issue #48734
PR #43454 fixed an issue where "jobs.lookup_jid" was not working
correctly with external job caches. However, this fix for external
job caches broke some inner workings of job storage when using the
local_cache.
We need to preserve the previous behavior for the local_cache, but
keep the new behavior for other external job caches.
If "savefstr" is called in the local cache, the target data does not
get written to the local_cache, and the target-type gets listed as a
"list" type instead of "glob".
This is a regression test for fixing the local_cache behavior.
'''
self.run_salt('minion test.echo target_info_test')
ret = self.run_run_plus('jobs.list_jobs')
for item in ret['return'].values():
if item['Function'] == 'test.echo' and \
item['Arguments'][0] == 'target_info_test':
job_ret = item
tgt = job_ret['Target']
tgt_type = job_ret['Target-type']
assert tgt != 'unknown-target'
assert tgt in ['minion', 'sub_minion']
assert tgt_type == 'glob'

11
tests/support/case.py
View File

@ -251,6 +251,7 @@ class ShellTestCase(TestCase, AdaptedConfigurationTestCaseMixin):
# FIXME A timeout of zero or disabling timeouts may not return results!
timeout=15,
raw=False,
popen_kwargs=None,
log_output=None):
'''
Execute a script with the given argument string
@ -285,11 +286,12 @@ class ShellTestCase(TestCase, AdaptedConfigurationTestCaseMixin):
tmp_file = tempfile.SpooledTemporaryFile()
popen_kwargs = {
popen_kwargs = popen_kwargs or {}
popen_kwargs = dict({
'shell': True,
'stdout': tmp_file,
'universal_newlines': True,
}
}, **popen_kwargs)
if catch_stderr is True:
popen_kwargs['stderr'] = subprocess.PIPE
@ -485,7 +487,7 @@ class ShellCase(ShellTestCase, AdaptedConfigurationTestCaseMixin, ScriptPathMixi
except OSError:
os.chdir(INTEGRATION_TEST_DIR)
def run_salt(self, arg_str, with_retcode=False, catch_stderr=False, timeout=90): # pylint: disable=W0221
def run_salt(self, arg_str, with_retcode=False, catch_stderr=False, timeout=90, popen_kwargs=None): # pylint: disable=W0221
'''
Execute salt
'''
@ -494,7 +496,8 @@ class ShellCase(ShellTestCase, AdaptedConfigurationTestCaseMixin, ScriptPathMixi
arg_str,
with_retcode=with_retcode,
catch_stderr=catch_stderr,
timeout=timeout)
timeout=timeout,
popen_kwargs=popen_kwargs)
log.debug('Result of run_salt for command \'%s\': %s', arg_str, ret)
return ret

37
tests/unit/modules/test_dockermod.py
View File

@ -822,15 +822,44 @@ class DockerTestCase(TestCase, LoaderModuleMockMixin):
'container1': {'Config': {},
'HostConfig': {
'Ulimits': [
{u'Hard': -1, u'Soft': -1, u'Name': u'core'},
{u'Hard': 65536, u'Soft': 65536, u'Name': u'nofile'}
{'Hard': -1, 'Soft': -1, 'Name': 'core'},
{'Hard': 65536, 'Soft': 65536, 'Name': 'nofile'}
]
}},
'container2': {'Config': {},
'HostConfig': {
'Ulimits': [
{u'Hard': 65536, u'Soft': 65536, u'Name': u'nofile'},
{u'Hard': -1, u'Soft': -1, u'Name': u'core'}
{'Hard': 65536, 'Soft': 65536, 'Name': 'nofile'},
{'Hard': -1, 'Soft': -1, 'Name': 'core'}
]
}},
}[id_]
inspect_container_mock = MagicMock(side_effect=_inspect_container_effect)
with patch.object(docker_mod, 'inspect_container', inspect_container_mock):
ret = docker_mod.compare_container('container1', 'container2')
self.assertEqual(ret, {})
def test_compare_container_env_order(self):
'''
Test comparing two containers when the order of the Env HostConfig
values are different, but the values are the same.
'''
def _inspect_container_effect(id_):
return {
'container1': {'Config': {},
'HostConfig': {
'Env': [
'FOO=bar',
'HELLO=world',
]
}},
'container2': {'Config': {},
'HostConfig': {
'Env': [
'HELLO=world',
'FOO=bar',
]
}},
}[id_]

404
tests/unit/utils/test_mac_utils.py
View File

@ -5,11 +5,19 @@ mac_utils tests
# Import python libs
from __future__ import absolute_import, unicode_literals
import os
import plistlib
import xml.parsers.expat
# Import Salt Testing Libs
from tests.support.unit import TestCase, skipIf
from tests.support.mock import MagicMock, patch, NO_MOCK, NO_MOCK_REASON, call
from tests.support.mock import (
call,
MagicMock,
mock_open,
NO_MOCK,
NO_MOCK_REASON,
patch
)
from tests.support.mixins import LoaderModuleMockMixin
# Import Salt libs
@ -215,259 +223,223 @@ class MacUtilsTestCase(TestCase, LoaderModuleMockMixin):
@patch('salt.utils.path.os_walk')
@patch('os.path.exists')
@patch('plistlib.readPlist')
def test_available_services(self, mock_read_plist, mock_exists, mock_os_walk):
def test_available_services_result(self, mock_exists, mock_os_walk):
'''
test available_services
test available_services results are properly formed dicts.
'''
mock_os_walk.side_effect = [
[('/Library/LaunchAgents', [], ['com.apple.lla1.plist', 'com.apple.lla2.plist'])],
[('/Library/LaunchDaemons', [], ['com.apple.lld1.plist', 'com.apple.lld2.plist'])],
[('/System/Library/LaunchAgents', [], ['com.apple.slla1.plist', 'com.apple.slla2.plist'])],
[('/System/Library/LaunchDaemons', [], ['com.apple.slld1.plist', 'com.apple.slld2.plist'])],
]
mock_read_plist.side_effect = [
MagicMock(Label='com.apple.lla1'),
MagicMock(Label='com.apple.lla2'),
MagicMock(Label='com.apple.lld1'),
MagicMock(Label='com.apple.lld2'),
MagicMock(Label='com.apple.slla1'),
MagicMock(Label='com.apple.slla2'),
MagicMock(Label='com.apple.slld1'),
MagicMock(Label='com.apple.slld2'),
]
results = {'/Library/LaunchAgents': ['com.apple.lla1.plist']}
mock_os_walk.side_effect = _get_walk_side_effects(results)
mock_exists.return_value = True
ret = mac_utils._available_services()
# Make sure it's a dict with 8 items
self.assertTrue(isinstance(ret, dict))
self.assertEqual(len(ret), 8)
plists = [{'Label': 'com.apple.lla1'}]
ret = _run_available_services(plists)
self.assertEqual(
ret['com.apple.lla1']['file_name'],
'com.apple.lla1.plist')
self.assertEqual(
ret['com.apple.lla1']['file_path'],
os.path.realpath(
os.path.join('/Library/LaunchAgents', 'com.apple.lla1.plist')))
self.assertEqual(
ret['com.apple.slld2']['file_name'],
'com.apple.slld2.plist')
self.assertEqual(
ret['com.apple.slld2']['file_path'],
os.path.realpath(
os.path.join('/System/Library/LaunchDaemons', 'com.apple.slld2.plist')))
expected = {
'com.apple.lla1': {
'file_name': 'com.apple.lla1.plist',
'file_path': '/Library/LaunchAgents/com.apple.lla1.plist',
'plist': plists[0]}}
self.assertEqual(ret, expected)
@patch('salt.utils.path.os_walk')
@patch('os.path.exists')
@patch('plistlib.readPlist')
@patch('os.listdir')
@patch('os.path.isdir')
def test_available_services_dirs(self,
mock_isdir,
mock_listdir,
mock_exists,
mock_os_walk):
'''
test available_services checks all of the expected dirs.
'''
results = {
'/Library/LaunchAgents': ['com.apple.lla1.plist'],
'/Library/LaunchDaemons': ['com.apple.lld1.plist'],
'/System/Library/LaunchAgents': ['com.apple.slla1.plist'],
'/System/Library/LaunchDaemons': ['com.apple.slld1.plist'],
'/Users/saltymcsaltface/Library/LaunchAgents': [
'com.apple.uslla1.plist']}
mock_os_walk.side_effect = _get_walk_side_effects(results)
mock_listdir.return_value = ['saltymcsaltface']
mock_isdir.return_value = True
mock_exists.return_value = True
plists = [
{'Label': 'com.apple.lla1'},
{'Label': 'com.apple.lld1'},
{'Label': 'com.apple.slla1'},
{'Label': 'com.apple.slld1'},
{'Label': 'com.apple.uslla1'}]
ret = _run_available_services(plists)
self.assertEqual(len(ret), 5)
@patch('salt.utils.path.os_walk')
@patch('os.path.exists')
@patch('plistlib.readPlist' if six.PY2 else 'plistlib.load')
def test_available_services_broken_symlink(self, mock_read_plist, mock_exists, mock_os_walk):
'''
test available_services
test available_services when it encounters a broken symlink.
'''
mock_os_walk.side_effect = [
[('/Library/LaunchAgents', [], ['com.apple.lla1.plist', 'com.apple.lla2.plist'])],
[('/Library/LaunchDaemons', [], ['com.apple.lld1.plist', 'com.apple.lld2.plist'])],
[('/System/Library/LaunchAgents', [], ['com.apple.slla1.plist', 'com.apple.slla2.plist'])],
[('/System/Library/LaunchDaemons', [], ['com.apple.slld1.plist', 'com.apple.slld2.plist'])],
]
results = {'/Library/LaunchAgents': ['com.apple.lla1.plist', 'com.apple.lla2.plist']}
mock_os_walk.side_effect = _get_walk_side_effects(results)
mock_exists.side_effect = [True, False]
mock_read_plist.side_effect = [
MagicMock(Label='com.apple.lla1'),
MagicMock(Label='com.apple.lla2'),
MagicMock(Label='com.apple.lld1'),
MagicMock(Label='com.apple.lld2'),
MagicMock(Label='com.apple.slld1'),
MagicMock(Label='com.apple.slld2'),
]
plists = [{'Label': 'com.apple.lla1'}]
ret = _run_available_services(plists)
mock_exists.side_effect = [True, True, True, True, False, False, True, True]
ret = mac_utils._available_services()
# Make sure it's a dict with 6 items
self.assertTrue(isinstance(ret, dict))
self.assertEqual(len(ret), 6)
self.assertEqual(
ret['com.apple.lla1']['file_name'],
'com.apple.lla1.plist')
self.assertEqual(
ret['com.apple.lla1']['file_path'],
os.path.realpath(
os.path.join('/Library/LaunchAgents', 'com.apple.lla1.plist')))
self.assertEqual(
ret['com.apple.slld2']['file_name'],
'com.apple.slld2.plist')
self.assertEqual(
ret['com.apple.slld2']['file_path'],
os.path.realpath(
os.path.join('/System/Library/LaunchDaemons', 'com.apple.slld2.plist')))
expected = {
'com.apple.lla1': {
'file_name': 'com.apple.lla1.plist',
'file_path': '/Library/LaunchAgents/com.apple.lla1.plist',
'plist': plists[0]}}
self.assertEqual(ret, expected)
@patch('salt.utils.path.os_walk')
@patch('os.path.exists')
@patch('plistlib.readPlist')
@patch('salt.utils.mac_utils.__salt__')
@patch('plistlib.readPlistFromString' if six.PY2 else 'plistlib.loads')
def test_available_services_non_xml(self,
mock_read_plist_from_string,
mock_run,
mock_read_plist,
mock_exists,
mock_os_walk):
@patch('plistlib.readPlistFromString', create=True)
def test_available_services_binary_plist(self,
mock_read_plist_from_string,
mock_run,
mock_read_plist,
mock_exists,
mock_os_walk):
'''
test available_services
test available_services handles binary plist files.
'''
mock_os_walk.side_effect = [
[('/Library/LaunchAgents', [], ['com.apple.lla1.plist', 'com.apple.lla2.plist'])],
[('/Library/LaunchDaemons', [], ['com.apple.lld1.plist', 'com.apple.lld2.plist'])],
[('/System/Library/LaunchAgents', [], ['com.apple.slla1.plist', 'com.apple.slla2.plist'])],
[('/System/Library/LaunchDaemons', [], ['com.apple.slld1.plist', 'com.apple.slld2.plist'])],
]
attrs = {'cmd.run': MagicMock(return_value='<some xml>')}
def getitem(name):
return attrs[name]
mock_run.__getitem__.side_effect = getitem
mock_run.configure_mock(**attrs)
results = {'/Library/LaunchAgents': ['com.apple.lla1.plist']}
mock_os_walk.side_effect = _get_walk_side_effects(results)
mock_exists.return_value = True
mock_read_plist.side_effect = Exception()
mock_read_plist_from_string.side_effect = [
MagicMock(Label='com.apple.lla1'),
MagicMock(Label='com.apple.lla2'),
MagicMock(Label='com.apple.lld1'),
MagicMock(Label='com.apple.lld2'),
MagicMock(Label='com.apple.slla1'),
MagicMock(Label='com.apple.slla2'),
MagicMock(Label='com.apple.slld1'),
MagicMock(Label='com.apple.slld2'),
]
ret = mac_utils._available_services()
plists = [{'Label': 'com.apple.lla1'}]
cmd = '/usr/bin/plutil -convert xml1 -o - -- "{0}"'
calls = [
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchAgents', 'com.apple.lla1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchAgents', 'com.apple.lla2.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchDaemons', 'com.apple.lld1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchDaemons', 'com.apple.lld2.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchAgents', 'com.apple.slla1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchAgents', 'com.apple.slla2.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchDaemons', 'com.apple.slld1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchDaemons', 'com.apple.slld2.plist'))),),
]
mock_run.assert_has_calls(calls, any_order=True)
if six.PY2:
attrs = {'cmd.run': MagicMock()}
# Make sure it's a dict with 8 items
self.assertTrue(isinstance(ret, dict))
self.assertEqual(len(ret), 8)
def getitem(name):
return attrs[name]
self.assertEqual(
ret['com.apple.lla1']['file_name'],
'com.apple.lla1.plist')
mock_run.__getitem__.side_effect = getitem
mock_run.configure_mock(**attrs)
cmd = '/usr/bin/plutil -convert xml1 -o - -- "{}"'.format(
'/Library/LaunchAgents/com.apple.lla1.plist')
calls = [call.cmd.run(cmd)]
self.assertEqual(
ret['com.apple.lla1']['file_path'],
os.path.realpath(
os.path.join('/Library/LaunchAgents', 'com.apple.lla1.plist')))
mock_read_plist.side_effect = xml.parsers.expat.ExpatError
mock_read_plist_from_string.side_effect = plists
ret = mac_utils._available_services()
else:
# Py3 plistlib knows how to handle binary plists without
# any extra work, so this test doesn't really do anything
# new.
ret = _run_available_services(plists)
self.assertEqual(
ret['com.apple.slld2']['file_name'],
'com.apple.slld2.plist')
expected = {
'com.apple.lla1': {
'file_name': 'com.apple.lla1.plist',
'file_path': '/Library/LaunchAgents/com.apple.lla1.plist',
'plist': plists[0]}}
self.assertEqual(ret, expected)
self.assertEqual(
ret['com.apple.slld2']['file_path'],
os.path.realpath(
os.path.join('/System/Library/LaunchDaemons', 'com.apple.slld2.plist')))
if six.PY2:
mock_run.assert_has_calls(calls, any_order=True)
@patch('salt.utils.path.os_walk')
@patch('os.path.exists')
@patch('plistlib.readPlist')
def test_available_services_invalid_file(self, mock_exists, mock_os_walk):
'''
test available_services excludes invalid files.
The py3 plistlib raises an InvalidFileException when a plist
file cannot be parsed. This test only asserts things for py3.
'''
if six.PY3:
results = {'/Library/LaunchAgents': ['com.apple.lla1.plist']}
mock_os_walk.side_effect = _get_walk_side_effects(results)
mock_exists.return_value = True
plists = [{'Label': 'com.apple.lla1'}]
mock_load = MagicMock()
mock_load.side_effect = plistlib.InvalidFileException
with patch('salt.utils.files.fopen', mock_open()):
with patch('plistlib.load', mock_load):
ret = mac_utils._available_services()
self.assertEqual(len(ret), 0)
@patch('salt.utils.mac_utils.__salt__')
@patch('plistlib.readPlistFromString' if six.PY2 else 'plistlib.loads')
def test_available_services_non_xml_malformed_plist(self,
mock_read_plist_from_string,
mock_run,
mock_read_plist,
mock_exists,
mock_os_walk):
@patch('plistlib.readPlist')
@patch('salt.utils.path.os_walk')
@patch('os.path.exists')
def test_available_services_expat_error(self,
mock_exists,
mock_os_walk,
mock_read_plist,
mock_run):
'''
test available_services
test available_services excludes files with expat errors.
Poorly formed XML will raise an ExpatError on py2. It will
also be raised by some almost-correct XML on py3.
'''
mock_os_walk.side_effect = [
[('/Library/LaunchAgents', [], ['com.apple.lla1.plist', 'com.apple.lla2.plist'])],
[('/Library/LaunchDaemons', [], ['com.apple.lld1.plist', 'com.apple.lld2.plist'])],
[('/System/Library/LaunchAgents', [], ['com.apple.slla1.plist', 'com.apple.slla2.plist'])],
[('/System/Library/LaunchDaemons', [], ['com.apple.slld1.plist', 'com.apple.slld2.plist'])],
]
attrs = {'cmd.run': MagicMock(return_value='<some xml>')}
def getitem(name):
return attrs[name]
mock_run.__getitem__.side_effect = getitem
mock_run.configure_mock(**attrs)
results = {'/Library/LaunchAgents': ['com.apple.lla1.plist']}
mock_os_walk.side_effect = _get_walk_side_effects(results)
mock_exists.return_value = True
mock_read_plist.side_effect = Exception()
mock_read_plist_from_string.return_value = 'malformedness'
ret = mac_utils._available_services()
if six.PY3:
mock_load = MagicMock()
mock_load.side_effect = xml.parsers.expat.ExpatError
with patch('salt.utils.files.fopen', mock_open()):
with patch('plistlib.load', mock_load):
ret = mac_utils._available_services()
else:
attrs = {'cmd.run': MagicMock()}
cmd = '/usr/bin/plutil -convert xml1 -o - -- "{0}"'
calls = [
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchAgents', 'com.apple.lla1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchAgents', 'com.apple.lla2.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchDaemons', 'com.apple.lld1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/Library/LaunchDaemons', 'com.apple.lld2.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchAgents', 'com.apple.slla1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchAgents', 'com.apple.slla2.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchDaemons', 'com.apple.slld1.plist'))),),
call.cmd.run(cmd.format(os.path.realpath(os.path.join(
'/System/Library/LaunchDaemons', 'com.apple.slld2.plist'))),),
]
mock_run.assert_has_calls(calls, any_order=True)
def getitem(name):
return attrs[name]
# Make sure it's a dict with 8 items
self.assertTrue(isinstance(ret, dict))
self.assertEqual(len(ret), 8)
mock_run.__getitem__.side_effect = getitem
mock_run.configure_mock(**attrs)
cmd = '/usr/bin/plutil -convert xml1 -o - -- "{}"'.format(
'/Library/LaunchAgents/com.apple.lla1.plist')
calls = [call.cmd.run(cmd)]
self.assertEqual(
ret['com.apple.lla1.plist']['file_name'],
'com.apple.lla1.plist')
mock_raise_expat_error = MagicMock(
side_effect=xml.parsers.expat.ExpatError)
self.assertEqual(
ret['com.apple.lla1.plist']['file_path'],
os.path.realpath(
os.path.join('/Library/LaunchAgents', 'com.apple.lla1.plist')))
with patch('plistlib.readPlist', mock_raise_expat_error):
with patch('plistlib.readPlistFromString', mock_raise_expat_error):
ret = mac_utils._available_services()
self.assertEqual(
ret['com.apple.slld2.plist']['file_name'],
'com.apple.slld2.plist')
mock_run.assert_has_calls(calls, any_order=True)
self.assertEqual(
ret['com.apple.slld2.plist']['file_path'],
os.path.realpath(
os.path.join('/System/Library/LaunchDaemons', 'com.apple.slld2.plist')))
self.assertEqual(len(ret), 0)
def _get_walk_side_effects(results):
'''
Data generation helper function for service tests.
'''
def walk_side_effect(*args, **kwargs):
return [(args[0], [], results.get(args[0], []))]
return walk_side_effect
def _run_available_services(plists):
if six.PY2:
mock_read_plist = MagicMock()
mock_read_plist.side_effect = plists
with patch('plistlib.readPlist', mock_read_plist):
ret = mac_utils._available_services()
else:
mock_load = MagicMock()
mock_load.side_effect = plists
with patch('salt.utils.files.fopen', mock_open()):
with patch('plistlib.load', mock_load):
ret = mac_utils._available_services()
return ret

22
tests/unit/utils/test_timed_subprocess.py Normal file
View File

@ -0,0 +1,22 @@
# -*- coding: utf-8 -*-
# Import python libs
from __future__ import absolute_import, print_function, unicode_literals
# Import Salt Testing libs
from tests.support.unit import TestCase
# Import salt libs
import salt.utils.timed_subprocess as timed_subprocess
class TestTimedSubprocess(TestCase):
def test_timedproc_with_shell_true_and_list_args(self):
'''
This test confirms the fix for the regression introduced in 1f7d50d.
The TimedProc dunder init would result in a traceback if the args were
passed as a list and shell=True was set.
'''
p = timed_subprocess.TimedProc(['echo', 'foo'], shell=True)
del p # Don't need this anymore