valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-07 08:58:59 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #38945 from rallytime/merge-develop

Browse Source 
[develop] Merge forward from 2016.11 to develop
This commit is contained in:
Mike Place 2017-01-25 12:36:18 -07:00 committed by GitHub
commit bbadc37301
16 changed files with 1921 additions and 127 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
conf/master
View File

@ -400,7 +400,7 @@
# Pass in an alternative location for the salt-ssh roster file
#roster_file: /etc/salt/roster
# Define a location for roster files so they can be chosen when using Salt API.
# Define locations for roster files so they can be chosen when using Salt API.
# An administrator can place roster files into these locations. Then when
# calling Salt API, parameter 'roster_file' should contain a relative path to
# these locations. That is, "roster_file=/foo/roster" will be resolved as

95
doc/ref/configuration/master.rst
View File

@ -245,7 +245,7 @@ each of Salt's module types such as ``runners``, ``output``, ``wheel``,
extension_modules: /root/salt_extmods
.. conf_minion:: module_dirs
.. conf_master:: module_dirs
``module_dirs``
---------------
@ -1172,6 +1172,99 @@ root of the base environment.
state_top: top.sls
.. conf_master:: state_top_saltenv
``state_top_saltenv``
---------------------
This option has no default value. Set it to an environment name to ensure that
*only* the top file from that environment is considered during a
:ref:`highstate <running-highstate>`.
.. note::
Using this value does not change the merging strategy. For instance, if
:conf_master:`top_file_merging_strategy` is set to ``merge``, and
:conf_master:`state_top_saltenv` is set to ``foo``, then any sections for
environments other than ``foo`` in the top file for the ``foo`` environment
will be ignored. With :conf_master:`state_top_saltenv` set to ``base``, all
states from all environments in the ``base`` top file will be applied,
while all other top files are ignored. The only way to set
:conf_master:`state_top_saltenv` to something other than ``base`` and not
have the other environments in the targeted top file ignored, would be to
set :conf_master:`top_file_merging_strategy` to ``merge_all``.
.. code-block:: yaml
state_top_saltenv: dev
.. conf_master:: top_file_merging_strategy
``top_file_merging_strategy``
-----------------------------
.. versionchanged:: 2016.11.0
A ``merge_all`` strategy has been added.
Default: ``merge``
When no specific fileserver environment (a.k.a. ``saltenv``) has been specified
for a :ref:`highstate <running-highstate>`, all environments' top files are
inspected. This config option determines how the SLS targets in those top files
are handled.
When set to ``merge``, the ``base`` environment's top file is evaluated first,
followed by the other environments' top files. The first target expression
(e.g. ``'*'``) for a given environment is kept, and when the same target
expression is used in a different top file evaluated later, it is ignored.
Because ``base`` is evaluated first, it is authoritative. For example, if there
is a target for ``'*'`` for the ``foo`` environment in both the ``base`` and
``foo`` environment's top files, the one in the ``foo`` environment would be
ignored. The environments will be evaluated in no specific order (aside from
``base`` coming first). For greater control over the order in which the
environments are evaluated, use :conf_master:`env_order`. Note that, aside from
the ``base`` environment's top file, any sections in top files that do not
match that top file's environment will be ignored. So, for example, a section
for the ``qa`` environment would be ignored if it appears in the ``dev``
environment's top file. To keep use cases like this from being ignored, use the
``merge_all`` strategy.
When set to ``same``, then for each environment, only that environment's top
file is processed, with the others being ignored. For example, only the ``dev``
environment's top file will be processed for the ``dev`` environment, and any
SLS targets defined for ``dev`` in the ``base`` environment's (or any other
environment's) top file will be ignored. If an environment does not have a top
file, then the top file from the :conf_master:`default_top` config parameter
will be used as a fallback.
When set to ``merge_all``, then all states in all environments in all top files
will be applied. The order in which individual SLS files will be executed will
depend on the order in which the top files were evaluated, and the environments
will be evaluated in no specific order. For greater control over the order in
which the environments are evaluated, use :conf_master:`env_order`.
.. code-block:: yaml
top_file_merging_strategy: same
.. conf_master:: env_order
``env_order``
-------------
Default: ``[]``
When :conf_master:`top_file_merging_strategy` is set to ``merge``, and no
environment is specified for a :ref:`highstate <running-highstate>`, this
config option allows for the order in which top files are evaluated to be
explicitly defined.
.. code-block:: yaml
env_order:
- base
- dev
- qa
.. conf_master:: master_tops
``master_tops``

235
doc/ref/configuration/minion.rst
View File

@ -1184,6 +1184,136 @@ below.
service: systemd
Top File Settings
=================
These parameters only have an effect if running a masterless minion.
.. conf_minion:: state_top
``state_top``
-------------
Default: ``top.sls``
The state system uses a "top" file to tell the minions what environment to
use and what modules to use. The state_top file is defined relative to the
root of the base environment.
.. code-block:: yaml
state_top: top.sls
.. conf_minion:: state_top_saltenv
``state_top_saltenv``
---------------------
This option has no default value. Set it to an environment name to ensure that
*only* the top file from that environment is considered during a
:ref:`highstate <running-highstate>`.
.. note::
Using this value does not change the merging strategy. For instance, if
:conf_minion:`top_file_merging_strategy` is set to ``merge``, and
:conf_minion:`state_top_saltenv` is set to ``foo``, then any sections for
environments other than ``foo`` in the top file for the ``foo`` environment
will be ignored. With :conf_minion:`state_top_saltenv` set to ``base``, all
states from all environments in the ``base`` top file will be applied,
while all other top files are ignored. The only way to set
:conf_minion:`state_top_saltenv` to something other than ``base`` and not
have the other environments in the targeted top file ignored, would be to
set :conf_minion:`top_file_merging_strategy` to ``merge_all``.
.. code-block:: yaml
state_top_saltenv: dev
.. conf_minion:: top_file_merging_strategy
``top_file_merging_strategy``
-----------------------------
.. versionchanged:: 2016.11.0
A ``merge_all`` strategy has been added.
Default: ``merge``
When no specific fileserver environment (a.k.a. ``saltenv``) has been specified
for a :ref:`highstate <running-highstate>`, all environments' top files are
inspected. This config option determines how the SLS targets in those top files
are handled.
When set to ``merge``, the ``base`` environment's top file is evaluated first,
followed by the other environments' top files. The first target expression
(e.g. ``'*'``) for a given environment is kept, and when the same target
expression is used in a different top file evaluated later, it is ignored.
Because ``base`` is evaluated first, it is authoritative. For example, if there
is a target for ``'*'`` for the ``foo`` environment in both the ``base`` and
``foo`` environment's top files, the one in the ``foo`` environment would be
ignored. The environments will be evaluated in no specific order (aside from
``base`` coming first). For greater control over the order in which the
environments are evaluated, use :conf_minion:`env_order`. Note that, aside from
the ``base`` environment's top file, any sections in top files that do not
match that top file's environment will be ignored. So, for example, a section
for the ``qa`` environment would be ignored if it appears in the ``dev``
environment's top file. To keep use cases like this from being ignored, use the
``merge_all`` strategy.
When set to ``same``, then for each environment, only that environment's top
file is processed, with the others being ignored. For example, only the ``dev``
environment's top file will be processed for the ``dev`` environment, and any
SLS targets defined for ``dev`` in the ``base`` environment's (or any other
environment's) top file will be ignored. If an environment does not have a top
file, then the top file from the :conf_minion:`default_top` config parameter
will be used as a fallback.
When set to ``merge_all``, then all states in all environments in all top files
will be applied. The order in which individual SLS files will be executed will
depend on the order in which the top files were evaluated, and the environments
will be evaluated in no specific order. For greater control over the order in
which the environments are evaluated, use :conf_minion:`env_order`.
.. code-block:: yaml
top_file_merging_strategy: same
.. conf_minion:: env_order
``env_order``
-------------
Default: ``[]``
When :conf_minion:`top_file_merging_strategy` is set to ``merge``, and no
environment is specified for a :ref:`highstate <running-highstate>`, this
config option allows for the order in which top files are evaluated to be
explicitly defined.
.. code-block:: yaml
env_order:
- base
- dev
- qa
.. conf_minion:: default_top
``default_top``
---------------
Default: ``base``
When :conf_minion:`top_file_merging_strategy` is set to ``same``, and no
environment is specified for a :ref:`highstate <running-highstate>` (i.e.
:conf_minion:`environment` is not set for the minion), this config option
specifies a fallback environment in which to look for a top file if an
environment lacks one.
.. code-block:: yaml
default_top: dev
State Management Settings
=========================
@ -1286,111 +1416,6 @@ environments is to isolate via the top file.
environment: dev
.. conf_minion:: state_top_saltenv
``state_top_saltenv``
---------------------
This option has no default value. Set it to an environment name to ensure that
*only* the top file from that environment is considered during a
:ref:`highstate <running-highstate>`.
.. note::
Using this value does not change the merging strategy. For instance, if
:conf_minion:`top_file_merging_strategy` is set to ``merge``, and
:conf_minion:`state_top_saltenv` is set to ``foo``, then any sections for
environments other than ``foo`` in the top file for the ``foo`` environment
will be ignored. With :conf_minion:`state_top_saltenv` set to ``base``, all
states from all environments in the ``base`` top file will be applied,
while all other top files are ignored. The only way to set
:conf_minion:`state_top_saltenv` to something other than ``base`` and not
have the other environments in the targeted top file ignored, would be to
set :conf_minion:`top_file_merging_strategy` to ``merge_all``.
.. code-block:: yaml
state_top_saltenv: dev
.. conf_minion:: top_file_merging_strategy
``top_file_merging_strategy``
-----------------------------
.. versionchanged:: 2016.11.0
A ``merge_all`` strategy has been added.
Default: ``merge``
When no specific fileserver environment (a.k.a. ``saltenv``) has been specified
for a :ref:`highstate <running-highstate>`, all environments' top files are
inspected. This config option determines how the SLS targets in those top files
are handled.
When set to ``merge``, the ``base`` environment's top file is evaluated first,
followed by the other environments' top files. The first target expression
(e.g. ``'*'``) for a given environment is kept, and when the same target
expression is used in a different top file evaluated later, it is ignored.
Because ``base`` is evaluated first, it is authoritative. For example, if there
is a target for ``'*'`` for the ``foo`` environment in both the ``base`` and
``foo`` environment's top files, the one in the ``foo`` environment would be
ignored. The environments will be evaluated in no specific order (aside from
``base`` coming first). For greater control over the order in which the
environments are evaluated, use :conf_minion:`env_order`.
When set to ``same``, then for each environment, only that environment's top
file is processed, with the others being ignored. For example, only the ``dev``
environment's top file will be processed for the ``dev`` environment, and any
SLS targets defined for ``dev`` in the ``base`` environment's (or any other
environment's) top file will be ignored. If an environment does not have a top
file, then the top file from the :conf_minion:`default_top` config parameter
will be used as a fallback.
When set to ``merge_all``, then all states in all environments in all top files
will be applied. The order in which individual SLS files will be executed will
depend on the order in which the top files were evaluated, and the environments
will be evaluated in no specific order. For greater control over the order in
which the environments are evaluated, use :conf_minion:`env_order`.
.. code-block:: yaml
top_file_merging_strategy: same
.. conf_minion:: env_order
``env_order``
-------------
Default: ``[]``
When :conf_minion:`top_file_merging_strategy` is set to ``merge``, and no
environment is specified for a :ref:`highstate <running-highstate>`, this
config option allows for the order in which top files are evaluated to be
explicitly defined.
.. code-block:: yaml
env_order:
- base
- dev
- qa
.. conf_minion:: default_top
``default_top``
---------------
Default: ``base``
When :conf_minion:`top_file_merging_strategy` is set to ``same``, and no
environment is specified for a :ref:`highstate <running-highstate>` (i.e.
:conf_minion:`environment` is not set for the minion), this config option
specifies a fallback environment in which to look for a top file if an
environment lacks one.
.. code-block:: yaml
default_top: dev
.. conf_minion:: snapper_states
``snapper_states``

1655
doc/topics/releases/2016.11.2.rst
View File

File diff suppressed because it is too large Load Diff

6
salt/cache/localfs.py vendored
View File

@ -129,13 +129,17 @@ def list_(bank, cachedir):
if not os.path.isdir(base):
return []
try:
return os.listdir(base)
items = os.listdir(base)
except OSError as exc:
raise SaltCacheError(
'There was an error accessing directory "{0}": {1}'.format(
base, exc
)
)
ret = []
for item in items:
ret.append(item.rstrip('.p'))
return ret
getlist = list_

2
salt/modules/cassandra_cql.py
View File

@ -194,7 +194,7 @@ def _get_ssl_opts():
def _connect(contact_points=None, port=None, cql_user=None, cql_pass=None,
protocol_version=4):
protocol_version=None):
'''
Connect to a Cassandra cluster.

7
salt/utils/network.py
View File

@ -1317,8 +1317,11 @@ def _freebsd_remotes_on(port, which_end):
continue # ignore header
if len(chunks) < 2:
continue
local = chunks[5]
remote = chunks[6]
# sockstat -4 -c -p 4506 does this with high PIDs:
# USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
# salt-master python2.781106 35 tcp4 192.168.12.34:4506 192.168.12.45:60143
local = chunks[-2]
remote = chunks[-1]
lhost, lport = local.split(':')
rhost, rport = remote.split(':')
if which_end == 'local' and int(lport) != port: # ignore if local port not port

16
tests/integration/__init__.py
View File

@ -784,7 +784,7 @@ class TestDaemon(object):
)
sys.stdout.flush()
process.start()
process.wait_until_running(timeout=15)
process.wait_until_running(timeout=60)
sys.stdout.write(
'\r{0}\r'.format(
' ' * getattr(self.parser.options, 'output_columns', PNUM)
@ -1849,14 +1849,14 @@ class ShellCase(AdaptedConfigurationTestCaseMixIn, ShellTestCase, ScriptPathMixi
except OSError:
os.chdir(INTEGRATION_TEST_DIR)
def run_salt(self, arg_str, with_retcode=False, catch_stderr=False, timeout=30): # pylint: disable=W0221
def run_salt(self, arg_str, with_retcode=False, catch_stderr=False, timeout=60): # pylint: disable=W0221
'''
Execute salt
'''
arg_str = '-c {0} {1}'.format(self.get_config_dir(), arg_str)
return self.run_script('salt', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=timeout)
def run_ssh(self, arg_str, with_retcode=False, catch_stderr=False, timeout=25): # pylint: disable=W0221
def run_ssh(self, arg_str, with_retcode=False, catch_stderr=False, timeout=60): # pylint: disable=W0221
'''
Execute salt-ssh
'''
@ -1871,7 +1871,7 @@ class ShellCase(AdaptedConfigurationTestCaseMixIn, ShellTestCase, ScriptPathMixi
arg_str,
timeout=timeout,
async_flag=' --async' if async else '')
return self.run_script('salt-run', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=30)
return self.run_script('salt-run', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=60)
def run_run_plus(self, fun, *arg, **kwargs):
'''
@ -1918,7 +1918,7 @@ class ShellCase(AdaptedConfigurationTestCaseMixIn, ShellTestCase, ScriptPathMixi
arg_str,
catch_stderr=catch_stderr,
with_retcode=with_retcode,
timeout=30
timeout=60
)
def run_cp(self, arg_str, with_retcode=False, catch_stderr=False):
@ -1926,16 +1926,16 @@ class ShellCase(AdaptedConfigurationTestCaseMixIn, ShellTestCase, ScriptPathMixi
Execute salt-cp
'''
arg_str = '--config-dir {0} {1}'.format(self.get_config_dir(), arg_str)
return self.run_script('salt-cp', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=30)
return self.run_script('salt-cp', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=60)
def run_call(self, arg_str, with_retcode=False, catch_stderr=False):
'''
Execute salt-call.
'''
arg_str = '--config-dir {0} {1}'.format(self.get_config_dir(), arg_str)
return self.run_script('salt-call', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=30)
return self.run_script('salt-call', arg_str, with_retcode=with_retcode, catch_stderr=catch_stderr, timeout=60)
def run_cloud(self, arg_str, catch_stderr=False, timeout=15):
def run_cloud(self, arg_str, catch_stderr=False, timeout=30):
'''
Execute salt-cloud
'''

2
tests/integration/runners/state.py
View File

@ -109,7 +109,7 @@ class OrchEventTest(integration.ShellCase):
Tests for orchestration events
'''
def setUp(self):
self.timeout = 15
self.timeout = 60
self.master_d_dir = os.path.join(self.get_config_dir(), 'master.d')
try:
os.makedirs(self.master_d_dir)

6
tests/integration/shell/call.py
View File

@ -269,7 +269,7 @@ class CallTest(integration.ShellCase, testprogram.TestProgramCase, integration.S
'--config-dir {0} --local cmd.run "echo foo"'.format(
config_dir
),
timeout=15
timeout=60
)
try:
self.assertIn('local:', ret)
@ -292,7 +292,7 @@ class CallTest(integration.ShellCase, testprogram.TestProgramCase, integration.S
'--config-dir {0} cmd.run "echo foo"'.format(
config_dir
),
timeout=15
timeout=60
)
self.assertIn('local:', ret)
finally:
@ -322,7 +322,7 @@ class CallTest(integration.ShellCase, testprogram.TestProgramCase, integration.S
'--config-dir {0} cmd.run "echo foo"'.format(
config_dir
),
timeout=15,
timeout=60,
catch_stderr=True,
with_retcode=True
)

2
tests/integration/shell/key.py
View File

@ -265,7 +265,7 @@ class KeyTest(integration.ShellCase, integration.ShellCaseCommonTestsMixIn):
'--config-dir {0} -L'.format(
config_dir
),
timeout=15
timeout=60
)
try:
self.assertIn('minion', '\n'.join(ret))

2
tests/integration/shell/matcher.py
View File

@ -353,7 +353,7 @@ class MatchTest(integration.ShellCase, integration.ShellCaseCommonTestsMixIn):
'--config-dir {0} minion test.ping'.format(
config_dir
),
timeout=15,
timeout=60,
catch_stderr=True,
with_retcode=True
)

2
tests/integration/shell/runner.py
View File

@ -109,7 +109,7 @@ class RunTest(integration.ShellCase, testprogram.TestProgramCase, integration.Sh
'--config-dir {0} -d'.format(
config_dir
),
timeout=15,
timeout=60,
catch_stderr=True,
with_retcode=True
)

2
tests/unit/cache/localfs_test.py vendored
View File

@ -241,7 +241,7 @@ class LocalFSTest(TestCase):
# Now test the return of the list function
with patch.dict(localfs.__opts__, {'cachedir': tmp_dir}):
self.assertEqual(localfs.list_(bank='bank', cachedir=tmp_dir), ['key.p'])
self.assertEqual(localfs.list_(bank='bank', cachedir=tmp_dir), ['key'])
# 'contains' function tests: 1

1
tests/unit/doc_test.py
View File

@ -54,6 +54,7 @@ class DocTestCase(TestCase):
if 'man' in key \
or key.endswith('doc_test.py') \
or key.endswith('doc/conf.py') \
or key.endswith('doc/topics/releases/2016.11.2.rst') \
or key.endswith('/conventions/documentation.rst'):
continue

13
tests/unit/utils/network_test.py
View File

@ -92,6 +92,11 @@ USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root python2.7 1294 41 tcp4 127.0.0.1:61115 127.0.0.1:4506
'''
FREEBSD_SOCKSTAT_WITH_FAT_PID = '''\
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
salt-master python2.781106 35 tcp4 127.0.0.1:61115 127.0.0.1:4506
'''
@skipIf(NO_MOCK, NO_MOCK_REASON)
class NetworkTestCase(TestCase):
@ -237,6 +242,14 @@ class NetworkTestCase(TestCase):
remotes = network._freebsd_remotes_on('4506', 'remote')
self.assertEqual(remotes, set(['127.0.0.1']))
def test_freebsd_remotes_on_with_fat_pid(self):
with patch('salt.utils.is_sunos', lambda: False):
with patch('salt.utils.is_freebsd', lambda: True):
with patch('subprocess.check_output',
return_value=FREEBSD_SOCKSTAT_WITH_FAT_PID):
remotes = network._freebsd_remotes_on('4506', 'remote')
self.assertEqual(remotes, set(['127.0.0.1']))
@patch('platform.node', MagicMock(return_value='nodename'))
@patch('socket.gethostname', MagicMock(return_value='hostname'))
@patch('socket.getfqdn', MagicMock(return_value='hostname.domainname.blank'))