valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-07 08:58:59 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #24937 from basepi/merge-forward-develop

Browse Source 
Merge forward from 2015.5 to develop
This commit is contained in:
Colton Myers 2015-06-25 16:03:47 -06:00
commit f65e663353
15 changed files with 552 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
conf/minion
View File

@ -22,6 +22,18 @@
# If only one master is listed, this setting is ignored and a warning will be logged.
#random_master: False
# Minions can connect to multiple masters simultaneously (all masters
# are "hot"), or can be configured to failover if a master becomes
# unavailable. Multiple hot masters are configured by setting this
# value to "standard". Failover masters can be requested by setting
# to "failover". MAKE SURE TO SET master_alive_interval if you are
# using failover.
# master_type: standard
# Poll interval in seconds for checking if the master is still there. Only
# respected if master_type above is "failover".
# master_alive_interval: 30
# Set whether the minion should connect to the master via IPv6:
#ipv6: False

1
doc/contents-2.rst
View File

@ -9,6 +9,7 @@ Salt Table of Contents
topics/jobs/index
topics/event/index
topics/topology/index
topics/highavailability/index
topics/windows/index
topics/cloud/index
topics/netapi/index

1
doc/contents.rst
View File

@ -21,6 +21,7 @@ Salt Table of Contents
topics/event/index
topics/beacons/index
topics/ext_processes/index
topics/highavailability/index
topics/topology/index
topics/transports/raet/index
topics/windows/index

35
doc/ref/configuration/minion.rst
View File

@ -82,10 +82,24 @@ The option can can also be set to a list of masters, enabling
.. versionadded:: 2014.7.0
Default: ``str``
Default: ``standard``
The type of the :conf_minion:`master` variable. Can be either ``func`` or
``failover``.
The type of the :conf_minion:`master` variable. Can be ``standard``, ``failover`` or
``func``.
.. code-block:: yaml
master_type: failover
If this option is set to ``failover``, :conf_minion:`master` must be a list of
master addresses. The minion will then try each master in the order specified
in the list until it successfully connects. :conf_minion:`master_alive_interval`
must also be set, this determines how often the minion will verify the presence
of the master.
.. code-block:: yaml
master_type: func
If the master needs to be dynamically assigned by executing a function instead
of reading in the static master value, set this to ``func``. This can be used
@ -93,19 +107,16 @@ to manage the minion's master setting from an execution module. By simply
changing the algorithm in the module to return a new master ip/fqdn, restart
the minion and it will connect to the new master.
``master_alive_interval``
-------------------------
.. code-block:: yaml
master_type: func
master_alive_interval: 30
If this option is set to ``failover``, :conf_minion:`master` must be a list of
master addresses. The minion will then try each master in the order specified
in the list until it successfully connects.
.. code-block:: yaml
master_type: failover
Configures how often, in seconds, the minion will verify that the current
master is alive and responding. The minion will try to establish a connection
to the next master in the list if it finds the existing one is dead.
``master_shuffle``
------------------

65
doc/topics/highavailability/index.rst Normal file
View File

@ -0,0 +1,65 @@
.. _highavailability:
==================================
High Availability Features in Salt
==================================
Salt supports several features for high availability and fault tolerance.
Brief documentation for these features is listed alongside their configuration
parameters in :ref:`Configuration file examples <configuration/examples>`.
Multimaster
===========
Salt minions can connect to multiple masters at one time by configuring the
`master` configuration paramter as a YAML list of all the available masters. By
default, all masters are "hot", meaning that any master can direct commands to
the Salt infrastructure.
In a multimaster configuration, each master must have the same cryptographic
keys, and minion keys must be accepted on all masters separately. The contents
of file_roots and pillar_roots need to be kept in sync with processes external
to Salt as well
A tutorial on setting up multimaster with "hot" masters is here:
:doc:`Multimaster Tutorial </topics/tutorials/multimaster>`
Multimaster with Failover
=========================
Changing the ``master_type`` parameter from ``standard`` to ``failover`` will
cause minions to connect to the first responding master in the list of masters.
Every ``master_alive_check`` seconds the minions will check to make sure
the current master is still responding. If the master does not respond,
the minion will attempt to connect to the next master in the list. If the
minion runs out of masters, the list will be recycled in case dead masters
have been restored. Note that ``master_alive_check`` must be present in the
minion configuration, or else the recurring job to check master status
will not get scheduled.
Failover can be combined with PKI-style encrypted keys, but PKI is NOT
REQUIRED to use failover.
Multimaster with PKI and Failover is discussed in
:doc:`this tutorial </topics/tutorials/multimaster_pki>`
``master_type: failover`` can be combined with ``master_shuffle: True``
to spread minion connections across all masters (one master per
minion, not each minion connecting to all masters). Adding Salt Syndics
into the mix makes it possible to create a load-balanced Salt infrastructure.
If a master fails, minions will notice and select another master from the
available list.
Syndic
======
Salt's Syndic feature is a way to create differing infrastructure
topologies. It is not strictly an HA feature, but can be treated as such.
With the syndic, a Salt infrastructure can be partitioned in such a way that
certain masters control certain segments of the infrastructure, and "Master
of Masters" nodes can control multiple segments underneath them.
Syndics are covered in depth in :doc:`Salt Syndic </topics/topology/syndic>`.

25
salt/client/ssh/wrapper/state.py
View File

@ -22,6 +22,9 @@ import salt.minion
import salt.log
from salt.ext.six import string_types
__func_alias__ = {
'apply_': 'apply'
}
log = logging.getLogger(__name__)
@ -259,6 +262,28 @@ def high(data, **kwargs):
return stdout
def apply_(mods=None,
**kwargs):
'''
.. versionadded:: 2015.5.3
Apply states! This function will call highstate or state.sls based on the
arguments passed in, state.apply is intended to be the main gateway for
all state executions.
CLI Example:
.. code-block:: bash
salt '*' state.apply
salt '*' state.apply test
salt '*' state.apply test,pkgs
'''
if mods:
return sls(mods, **kwargs)
return highstate(**kwargs)
def highstate(test=None, **kwargs):
'''
Retrieve the state data from the salt master for this minion and execute it

3
salt/grains/core.py
View File

@ -739,10 +739,13 @@ def _virtual(osdata):
if maker.startswith('Bochs'):
grains['virtual'] = 'kvm'
if sysctl:
hv_vendor = __salt__['cmd.run']('{0} hw.hv_vendor'.format(sysctl))
model = __salt__['cmd.run']('{0} hw.model'.format(sysctl))
jail = __salt__['cmd.run'](
'{0} -n security.jail.jailed'.format(sysctl)
)
if 'bhyve' in hv_vendor:
grains['virtual'] = 'bhyve'
if jail == '1':
grains['virtual_subtype'] = 'jail'
if 'QEMU Virtual CPU' in model:

2
salt/modules/rh_service.py
View File

@ -58,7 +58,7 @@ def __virtual__():
))
if __grains__['os'] in enable:
if __grains__['os'] == 'SUSE':
if __grains__['osrelease'].startswith('11'):
if str(__grains__['osrelease']).startswith('11'):
return __virtualname__
else:
return False

4
salt/modules/yumpkg.py
View File

@ -1358,9 +1358,9 @@ def get_locked_packages(pattern=None, full=True):
_pat = r'\d\:({0}\-\S+)'.format(pattern)
else:
if full:
_pat = r'(\d\:\w+\-\S+)'
_pat = r'(\d\:\w+(?:[\.\-][^\-]+)*-\S+)'
else:
_pat = r'\d\:(\w+\-\S+)'
_pat = r'\d\:(\w+(?:[\.\-][^\-]+)*-\S+)'
pat = re.compile(_pat)
current_locks = []

4
salt/state.py
View File

@ -630,9 +630,11 @@ class State(object):
of a state package that has a mod_init function, then execute the
mod_init function in the state module.
'''
# ensure that the module is loaded
self.states['{0}.{1}'.format(low['state'], low['fun'])] # pylint: disable=W0106
minit = '{0}.mod_init'.format(low['state'])
if low['state'] not in self.mod_init:
if minit in self.states:
if minit in self.states._dict:
mret = self.states[minit](low)
if not mret:
return

1
salt/states/ssh_auth.py
View File

@ -189,6 +189,7 @@ def _absent_test(user, name, enc, comment, options, source, config):
comment = ('Key {0} for user {1} is set for removal').format(name, user)
else:
comment = ('Key is already absent')
result = True
return result, comment

28
salt/utils/cloud.py
View File

@ -94,6 +94,8 @@ NSTATES = {
}
SSH_PASSWORD_PROMP_RE = re.compile(r'(?:.*)[Pp]assword(?: for .*)?:\ *$', re.M)
SSH_PASSWORD_PROMP_SUDO_RE = \
re.compile(r'(?:.*sudo)(?:.*)[Pp]assword(?: for .*)?:', re.M)
# Get logging started
log = logging.getLogger(__name__)
@ -315,8 +317,14 @@ def bootstrap(vm_, opts):
if stat.S_ISSOCK(os.stat(os.environ['SSH_AUTH_SOCK']).st_mode):
has_ssh_agent = True
if key_filename is None and ('password' not in vm_
or not vm_['password']) and ('win_password' not in vm_ or not vm_['win_password']) and has_ssh_agent is False:
if (key_filename is None and
salt.config.get_cloud_config_value(
'password', vm_, opts, default=None
) is None and
salt.config.get_cloud_config_value(
'win_password', vm_, opts, default=None
) is None and
has_ssh_agent is False):
raise SaltCloudSystemExit(
'Cannot deploy Salt in a VM if the \'key_filename\' setting '
'is not set and there is no password set for the VM. '
@ -1661,12 +1669,26 @@ def _exec_ssh_cmd(cmd, error_msg=None, allow_failure=False, **kwargs):
stdout, stderr = proc.recv()
if stdout and is_not_checked:
if SSH_PASSWORD_PROMP_RE.search(stdout.split('\n')[0]):
if (
# if authenticating with an SSH key and 'sudo' is found
# in the password prompt
if ('key_filename' in kwargs and kwargs['key_filename']
and SSH_PASSWORD_PROMP_SUDO_RE.search(stdout)
):
# do nothing, as command already has adjustments to
# echo out the sudo password as part of the ssh command
# keep waiting for proc output
continue
# elif authenticating via password and haven't exhausted our
# password_retires
elif (
kwargs.get('password', None)
and (sent_password < password_retries)
):
sent_password += 1
proc.sendline(kwargs['password'])
# else raise an error as we are not authenticating properly
# * not authenticating with an SSH key
# * not authenticating with a Password
else:
raise SaltCloudPasswordError(error_msg)
is_not_checked = False

114
tests/unit/states/ssh_auth_test.py Normal file
View File

@ -0,0 +1,114 @@
# -*- coding: utf-8 -*-
'''
:codeauthor: :email:`Jayesh Kariya <jayeshk@saltstack.com>`
'''
# Import Python libs
from __future__ import absolute_import
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting.mock import (
NO_MOCK,
NO_MOCK_REASON,
MagicMock,
patch
)
from salttesting.helpers import ensure_in_syspath
ensure_in_syspath('../../')
# Import Salt Libs
from salt.states import ssh_auth
ssh_auth.__salt__ = {}
ssh_auth.__opts__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
class SshAuthTestCase(TestCase):
'''
Test cases for salt.states.ssh_auth
'''
# 'present' function tests: 1
def test_present(self):
'''
Test to verifies that the specified SSH key
is present for the specified user.
'''
name = 'sshkeys'
user = 'root'
source = 'salt://ssh_keys/id_rsa.pub'
ret = {'name': name,
'changes': {},
'result': True,
'comment': ''}
mock = MagicMock(return_value='exists')
mock_data = MagicMock(side_effect=['replace', 'new'])
with patch.dict(ssh_auth.__salt__, {'ssh.check_key': mock,
'ssh.set_auth_key': mock_data}):
with patch.dict(ssh_auth.__opts__, {'test': True}):
comt = ('The authorized host key sshkeys is already '
'present for user root')
ret.update({'comment': comt})
self.assertDictEqual(ssh_auth.present(name, user, source), ret)
with patch.dict(ssh_auth.__opts__, {'test': False}):
comt = ('The authorized host key sshkeys '
'for user root was updated')
ret.update({'comment': comt, 'changes': {name: 'Updated'}})
self.assertDictEqual(ssh_auth.present(name, user, source), ret)
comt = ('The authorized host key sshkeys '
'for user root was added')
ret.update({'comment': comt, 'changes': {name: 'New'}})
self.assertDictEqual(ssh_auth.present(name, user, source), ret)
# 'absent' function tests: 1
def test_absent(self):
'''
Test to verifies that the specified SSH key is absent.
'''
name = 'sshkeys'
user = 'root'
source = 'salt://ssh_keys/id_rsa.pub'
ret = {'name': name,
'changes': {},
'result': None,
'comment': ''}
mock = MagicMock(side_effect=['User authorized keys file not present',
'User authorized keys file not present',
'User authorized keys file not present',
'Key removed'])
mock_up = MagicMock(side_effect=['update', 'updated'])
with patch.dict(ssh_auth.__salt__, {'ssh.rm_auth_key': mock,
'ssh.check_key': mock_up}):
with patch.dict(ssh_auth.__opts__, {'test': True}):
comt = ('Key sshkeys for user root is set for removal')
ret.update({'comment': comt})
self.assertDictEqual(ssh_auth.absent(name, user, source), ret)
comt = ('Key is already absent')
ret.update({'comment': comt, 'result': True})
self.assertDictEqual(ssh_auth.absent(name, user, source), ret)
with patch.dict(ssh_auth.__opts__, {'test': False}):
comt = ('User authorized keys file not present')
ret.update({'comment': comt, 'result': False})
self.assertDictEqual(ssh_auth.absent(name, user, source), ret)
comt = ('Key removed')
ret.update({'comment': comt, 'result': True,
'changes': {name: 'Removed'}})
self.assertDictEqual(ssh_auth.absent(name, user, source), ret)
if __name__ == '__main__':
from integration import run_tests
run_tests(SshAuthTestCase, needs_daemon=False)

177
tests/unit/states/ssh_known_hosts_test.py Normal file
View File

@ -0,0 +1,177 @@
# -*- coding: utf-8 -*-
'''
:codeauthor: :email:`Jayesh Kariya <jayeshk@saltstack.com>`
'''
# Import Python libs
from __future__ import absolute_import
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting.mock import (
NO_MOCK,
NO_MOCK_REASON,
MagicMock,
patch
)
from salttesting.helpers import ensure_in_syspath
import os
ensure_in_syspath('../../')
# Import Salt Libs
from salt.states import ssh_known_hosts
ssh_known_hosts.__salt__ = {}
ssh_known_hosts.__opts__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
class SshKnownHostsTestCase(TestCase):
'''
Test cases for salt.states.ssh_known_hosts
'''
# 'present' function tests: 1
def test_present(self):
'''
Test to verifies that the specified host is known by the specified user.
'''
name = 'github.com'
user = 'root'
key = '16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48'
fingerprint = [key]
ret = {'name': name,
'changes': {},
'result': False,
'comment': ''}
with patch.dict(ssh_known_hosts.__opts__, {'test': True}):
with patch.object(os.path, 'isabs', MagicMock(return_value=False)):
comt = ('If not specifying a "user", '
'specify an absolute "config".')
ret.update({'comment': comt})
self.assertDictEqual(ssh_known_hosts.present(name), ret)
comt = ('Specify either "key" or "fingerprint", not both.')
ret.update({'comment': comt})
self.assertDictEqual(ssh_known_hosts.present(name, user, key=key,
fingerprint=[key]),
ret)
comt = ('Required argument "enc" if using "key" argument.')
ret.update({'comment': comt})
self.assertDictEqual(ssh_known_hosts.present(name, user, key=key),
ret)
mock = MagicMock(side_effect=['exists', 'add', 'update'])
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.check_known_host': mock}):
comt = ('Host github.com is already in .ssh/known_hosts')
ret.update({'comment': comt, 'result': True})
self.assertDictEqual(ssh_known_hosts.present(name, user), ret)
comt = ('Key for github.com is set to be'
' added to .ssh/known_hosts')
ret.update({'comment': comt, 'result': None})
self.assertDictEqual(ssh_known_hosts.present(name, user), ret)
comt = ('Key for github.com is set to be '
'updated in .ssh/known_hosts')
ret.update({'comment': comt})
self.assertDictEqual(ssh_known_hosts.present(name, user), ret)
with patch.dict(ssh_known_hosts.__opts__, {'test': False}):
result = {'status': 'exists', 'error': ''}
mock = MagicMock(return_value=result)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.set_known_host': mock}):
comt = ('github.com already exists in .ssh/known_hosts')
ret.update({'comment': comt, 'result': True})
self.assertDictEqual(ssh_known_hosts.present(name, user), ret)
result = {'status': 'error', 'error': ''}
mock = MagicMock(return_value=result)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.set_known_host': mock}):
ret.update({'comment': '', 'result': False})
self.assertDictEqual(ssh_known_hosts.present(name, user), ret)
result = {'status': 'updated', 'error': '',
'new': {'fingerprint': fingerprint, 'key': key},
'old': ''}
mock = MagicMock(return_value=result)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.set_known_host': mock}):
comt = ("{0}'s key saved to .ssh/known_hosts (key: {1})"
.format(name, key))
ret.update({'comment': comt, 'result': True,
'changes': {'new': {'fingerprint': fingerprint,
'key': key}, 'old': ''}})
self.assertDictEqual(ssh_known_hosts.present(name, user,
key=key), ret)
comt = ("{0}'s key saved to .ssh/known_hosts (fingerprint: {1})"
.format(name, fingerprint))
ret.update({'comment': comt})
self.assertDictEqual(ssh_known_hosts.present(name, user), ret)
# 'absent' function tests: 1
def test_absent(self):
'''
Test to verifies that the specified host is not known by the given user.
'''
name = 'github.com'
user = 'root'
ret = {'name': name,
'changes': {},
'result': False,
'comment': ''}
with patch.object(os.path, 'isabs', MagicMock(return_value=False)):
comt = ('If not specifying a "user", '
'specify an absolute "config".')
ret.update({'comment': comt})
self.assertDictEqual(ssh_known_hosts.absent(name), ret)
mock = MagicMock(return_value=False)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.get_known_host': mock}):
comt = ('Host is already absent')
ret.update({'comment': comt, 'result': True})
self.assertDictEqual(ssh_known_hosts.absent(name, user), ret)
mock = MagicMock(return_value=True)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.get_known_host': mock}):
with patch.dict(ssh_known_hosts.__opts__, {'test': True}):
comt = ('Key for github.com is set to be'
' removed from .ssh/known_hosts')
ret.update({'comment': comt, 'result': None})
self.assertDictEqual(ssh_known_hosts.absent(name, user), ret)
with patch.dict(ssh_known_hosts.__opts__, {'test': False}):
result = {'status': 'error', 'error': ''}
mock = MagicMock(return_value=result)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.rm_known_host': mock}):
ret.update({'comment': '', 'result': False})
self.assertDictEqual(ssh_known_hosts.absent(name, user),
ret)
result = {'status': 'removed', 'error': '',
'comment': 'removed'}
mock = MagicMock(return_value=result)
with patch.dict(ssh_known_hosts.__salt__,
{'ssh.rm_known_host': mock}):
ret.update({'comment': 'removed', 'result': True,
'changes': {'new': None, 'old': True}})
self.assertDictEqual(ssh_known_hosts.absent(name, user),
ret)
if __name__ == '__main__':
from integration import run_tests
run_tests(SshKnownHostsTestCase, needs_daemon=False)

99
tests/unit/states/status_test.py Normal file
View File

@ -0,0 +1,99 @@
# -*- coding: utf-8 -*-
'''
:codeauthor: :email:`Jayesh Kariya <jayeshk@saltstack.com>`
'''
# Import Python libs
from __future__ import absolute_import
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting.mock import (
NO_MOCK,
NO_MOCK_REASON,
MagicMock,
patch
)
from salttesting.helpers import ensure_in_syspath
ensure_in_syspath('../../')
# Import Salt Libs
from salt.states import status
status.__salt__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
class StatusTestCase(TestCase):
'''
Test cases for salt.states.status
'''
# 'loadavg' function tests: 1
def test_loadavg(self):
'''
Test to return the current load average for the specified minion.
'''
name = 'mymonitor'
ret = {'name': name,
'changes': {},
'result': True,
'data': {},
'comment': ''}
mock = MagicMock(return_value=[])
with patch.dict(status.__salt__, {'status.loadavg': mock}):
comt = ('Requested load average mymonitor not available ')
ret.update({'comment': comt, 'result': False})
self.assertDictEqual(status.loadavg(name), ret)
mock = MagicMock(return_value={name: 3})
with patch.dict(status.__salt__, {'status.loadavg': mock}):
comt = ('Min must be less than max')
ret.update({'comment': comt, 'result': False})
self.assertDictEqual(status.loadavg(name, 1, 5), ret)
comt = ('Load avg is below minimum of 4 at 3.0')
ret.update({'comment': comt, 'data': 3})
self.assertDictEqual(status.loadavg(name, 5, 4), ret)
comt = ('Load avg above maximum of 2 at 3.0')
ret.update({'comment': comt, 'data': 3})
self.assertDictEqual(status.loadavg(name, 2, 1), ret)
comt = ('Load avg in acceptable range')
ret.update({'comment': comt, 'result': True})
self.assertDictEqual(status.loadavg(name, 3, 1), ret)
# 'process' function tests: 1
def test_process(self):
'''
Test to return whether the specified signature
is found in the process tree.
'''
name = 'mymonitor'
ret = {'name': name,
'changes': {},
'result': True,
'data': {},
'comment': ''}
mock = MagicMock(side_effect=[{}, {name: 1}])
with patch.dict(status.__salt__, {'status.pid': mock}):
comt = ('Process signature "mymonitor" not found ')
ret.update({'comment': comt, 'result': False})
self.assertDictEqual(status.process(name), ret)
comt = ('Process signature "mymonitor" was found ')
ret.update({'comment': comt, 'result': True,
'data': {name: 1}})
self.assertDictEqual(status.process(name), ret)
if __name__ == '__main__':
from integration import run_tests
run_tests(StatusTestCase, needs_daemon=False)