valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-07 08:58:59 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch '2014.7' into develop

Browse Source 
Conflicts:
	salt/utils/__init__.py
This commit is contained in:
Thomas S Hatch 2014-11-14 14:38:55 -07:00
commit 6d38d2db62
9 changed files with 350 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

281
doc/topics/development/tests/unit.rst
View File

@ -13,9 +13,34 @@ on the singular implementation of individual functions.
Preparing to Write a Unit Test
==============================
Unit tests live in: `tests/unit/`__.
This guide assumes you've followed the `directions for setting up salt testing`__.
.. __: https://github.com/saltstack/salt/tree/develop/tests/unit
Unit tests should be written to the following specifications:
- All the tests for a specific module at salt/.../<module>.py need to be
contained in a file called tests/unit/.../<module>_test.py, e.g. the tests
for salt/modules/fib.py need to be contained in a file called
tests/unit/modules/fib_test.py.
- The tests within tests/unit/modules/fib_test.py file must be member functions
of a class which subclasses salttesting.Testcase
- Each external resource used in the code to be tested, such as function calls,
external data either globally available or passed in through the function
arguments, file data, etc. needs to be mocked.
- Each raise and return statement of the code to be tested needs to be
separately and independently tested.
- Test functions should contain only one test and contain all necessary mock
data and mock code for that test.
- Test functions should be named test_<fcn>_<test-name> where <fcn> is the name
of the function being tested and <test-name> describes which raise or return
within the function is being tested and whether that raise or return
statement is considered a success or a failure condition.
.. __: http://docs.saltstack.com/topics/installation/index.html
Most commonly, the following imports are necessary to create a unit test:
@ -63,19 +88,16 @@ we might write the skeleton for testing ``fib.py``:
class FibTestCase(TestCase):
'''
If we want to set up variables common to all unit tests, we can do so
by defining a setUp method, which will be run automatically before
tests begin.
This class contains a set of functions that test salt.modules.fib.
'''
def setUp(self):
# Declare a simple set of five Fibonacci numbers starting at zero that we know are correct.
self.fib_five = [0, 1, 1, 2, 3]
def test_fib(self):
'''
To create a unit test, we should prefix the name with `test_' so that it's recognized by the test runner.
To create a unit test, we should prefix the name with `test_' so
that it's recognized by the test runner.
'''
self.assertEqual(fib.calculate(5), self.fib_five)
fib_five = (0, 1, 1, 2, 3)
self.assertEqual(fib.calculate(5), fib_five)
At this point, the test can now be run, either individually or as a part of a
@ -162,7 +184,8 @@ additional imports for MagicMock:
# Create test case class and inherit from Salt's customized TestCase
@skipIf(NO_MOCK, NO_MOCK_REASON) # Skip this test case if we don't have access to mock!
# Skip this test case if we don't have access to mock!
@skipIf(NO_MOCK, NO_MOCK_REASON)
class DbTestCase(TestCase):
def test_create_user(self):
# First, we replace 'execute_query' with our own mock function
@ -172,31 +195,35 @@ additional imports for MagicMock:
db.create_user('testuser')
# We could now query our mock object to see which calls were made to it.
# We could now query our mock object to see which calls were made
# to it.
## print db.execute_query.mock_calls
'''
We want to test to ensure that the correct query was formed.
This is a contrived example, just designed to illustrate the concepts at hand.
We want to test to ensure that the correct query was formed. This
is a contrived example, just designed to illustrate the concepts at
hand.
We're going to first contruct a call() object that represents the way we expect
our mocked execute_query() function to have been called.
Then, we'll examine the list of calls that were actually made to to execute_function().
We're going to first construct a call() object that represents the
way we expect our mocked execute_query() function to have been
called. Then, we'll examine the list of calls that were actually
made to to execute_function().
By comparing our expected call to execute_query() with create_user()'s call to
execute_query(), we can determine the success or failure of our unit test.
By comparing our expected call to execute_query() with
create_user()'s call to execute_query(), we can determine the
success or failure of our unit test.
'''
expected_call = call('CREATE USER testuser')
# Do the comparison! Will assert False if execute_query() was not called with the given call
# Do the comparison! Will assert False if execute_query() was not
# called with the given call
db.execute_query.assert_has_calls(expected_call)
.. __: http://www.voidspace.org.uk/python/mock/index.html
Modifying ``__salt__`` In Place
===============================
@ -210,5 +237,215 @@ function into ``__salt__`` that's actually a MagicMock instance.
.. code-block:: python
def show_patch(self):
with patch.dict(my_module.__salt__, {'function.to_replace': MagicMock()}:
with patch.dict(my_module.__salt__,
{'function.to_replace': MagicMock()}:
# From this scope, carry on with testing, with a modified __salt__!
A More Complete Example
=======================
Consider the following function from salt/modules/linux_sysctl.py.
.. code-block:: python
def get(name):
'''
Return a single sysctl parameter for this minion
CLI Example:
.. code-block:: bash
salt '*' sysctl.get net.ipv4.ip_forward
'''
cmd = 'sysctl -n {0}'.format(name)
out = __salt__['cmd.run'](cmd)
return out
This function is very simple, comprising only four source lines of code and
having only one return statement, so we know only one test is needed. There
are also two inputs to the function, the ``name`` function argument and the call
to ``__salt__['cmd.run']()``, both of which need to be appropriately mocked.
Mocking a function parameter is straightforward, whereas mocking a function
call will require, in this case, the use of MagicMock. For added isolation, we
will also redefine the ``__salt__`` dictionary such that it only contains
``'cmd.run'``.
.. code-block:: python
# Import Salt Libs
from salt.modules import linux_sysctl
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting.helpers import ensure_in_syspath
from salttesting.mock import (
MagicMock,
patch,
NO_MOCK,
NO_MOCK_REASON
)
ensure_in_syspath('../../')
# Globals
linux_sysctl.__salt__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
class LinuxSysctlTestCase(TestCase):
'''
TestCase for salt.modules.linux_sysctl module
'''
def test_get(self):
'''
Tests the return of get function
'''
mock_cmd = MagicMock(return_value=1)
with patch.dict(linux_sysctl.__salt__, {'cmd.run': mock_cmd}):
self.assertEqual(linux_sysctl.get('net.ipv4.ip_forward'), 1)
if __name__ == '__main__':
from integration import run_tests
run_tests(LinuxSysctlTestCase, needs_daemon=False)
Since ``get()`` has only one raise or return statement and that statement is a
success condition, the test function is simply named ``test_get()``. As
described, the single function call parameter, ``name`` is mocked with
``net.ipv4.ip_forward`` and ``__salt__['cmd.run']`` is replaced by a MagicMock
function object. We are only interested in the return value of
``__salt__['cmd.run']``, which MagicMock allows to be specified via
``return_value=1``. Finally, the test itself tests for equality between the
return value of ``get()`` and the expected return of ``1``. This assertion is
expected to succeed because ``get()`` will determine its return value from
``__salt__['cmd.run']``, which we have mocked to return ``1``.
A Complex Example
=================
Now consider the ``assign()`` function from the same
salt/modules/linux_sysctl.py source file.
.. code-block:: python
def assign(name, value):
'''
Assign a single sysctl parameter for this minion
CLI Example:
.. code-block:: bash
salt '*' sysctl.assign net.ipv4.ip_forward 1
'''
value = str(value)
sysctl_file = '/proc/sys/{0}'.format(name.replace('.', '/'))
if not os.path.exists(sysctl_file):
raise CommandExecutionError('sysctl {0} does not exist'.format(name))
ret = {}
cmd = 'sysctl -w {0}="{1}"'.format(name, value)
data = __salt__['cmd.run_all'](cmd)
out = data['stdout']
err = data['stderr']
# Example:
# # sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
# net.ipv4.tcp_rmem = 4096 87380 16777216
regex = re.compile(r'^{0}\s+=\s+{1}$'.format(re.escape(name),
re.escape(value)))
if not regex.match(out) or 'Invalid argument' in str(err):
if data['retcode'] != 0 and err:
error = err
else:
error = out
raise CommandExecutionError('sysctl -w failed: {0}'.format(error))
new_name, new_value = out.split(' = ', 1)
ret[new_name] = new_value
return ret
This function contains two raise statements and one return statement, so we
know that we will need (at least) three tests. It has two function arguments
and many references to non-builtin functions. In the tests below you will see
that MagicMock's ``patch()`` method may be used as a context manager or as a
decorator.
There are three test functions, one for each raise and return statement in the
source function. Each function is self-contained and contains all and only the
mocks and data needed to test the raise or return statement it is concerned
with.
.. code-block:: python
# Import Salt Libs
from salt.modules import linux_sysctl
from salt.exceptions import CommandExecutionError
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting.helpers import ensure_in_syspath
from salttesting.mock import (
MagicMock,
patch,
NO_MOCK,
NO_MOCK_REASON
)
ensure_in_syspath('../../')
# Globals
linux_sysctl.__salt__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
class LinuxSysctlTestCase(TestCase):
'''
TestCase for salt.modules.linux_sysctl module
'''
@patch('os.path.exists', MagicMock(return_value=False))
def test_assign_proc_sys_failed(self):
'''
Tests if /proc/sys/<kernel-subsystem> exists or not
'''
cmd = {'pid': 1337, 'retcode': 0, 'stderr': '',
'stdout': 'net.ipv4.ip_forward = 1'}
mock_cmd = MagicMock(return_value=cmd)
with patch.dict(linux_sysctl.__salt__, {'cmd.run_all': mock_cmd}):
self.assertRaises(CommandExecutionError,
linux_sysctl.assign,
'net.ipv4.ip_forward', 1)
@patch('os.path.exists', MagicMock(return_value=True))
def test_assign_cmd_failed(self):
'''
Tests if the assignment was successful or not
'''
cmd = {'pid': 1337, 'retcode': 0, 'stderr':
'sysctl: setting key "net.ipv4.ip_forward": Invalid argument',
'stdout': 'net.ipv4.ip_forward = backward'}
mock_cmd = MagicMock(return_value=cmd)
with patch.dict(linux_sysctl.__salt__, {'cmd.run_all': mock_cmd}):
self.assertRaises(CommandExecutionError,
linux_sysctl.assign,
'net.ipv4.ip_forward', 'backward')
@patch('os.path.exists', MagicMock(return_value=True))
def test_assign_success(self):
'''
Tests the return of successful assign function
'''
cmd = {'pid': 1337, 'retcode': 0, 'stderr': '',
'stdout': 'net.ipv4.ip_forward = 1'}
ret = {'net.ipv4.ip_forward': '1'}
mock_cmd = MagicMock(return_value=cmd)
with patch.dict(linux_sysctl.__salt__, {'cmd.run_all': mock_cmd}):
self.assertEqual(linux_sysctl.assign(
'net.ipv4.ip_forward', 1), ret)
if __name__ == '__main__':
from integration import run_tests
run_tests(LinuxSysctlTestCase, needs_daemon=False)

5
doc/topics/mine/index.rst
View File

@ -48,9 +48,8 @@ calls of the same function with different arguments.
mine_function: network.ip_addrs
cidr: 192.168.0.0/16
loopback_ip_addrs:
- mine_function: network.ip_addrs
- lo
- True
mine_function: network.ip_addrs
lo: True
Mine Interval
=============

4
salt/cloud/clouds/nova.py
View File

@ -502,6 +502,10 @@ def request_instance(vm_=None, call=None):
with salt.utils.fopen(userdata_file, 'r') as fp:
kwargs['userdata'] = fp.read()
kwargs['config_drive'] = config.get_cloud_config_value(
'config_drive', vm_, __opts__, search_global=False
)
salt.utils.cloud.fire_event(
'event',
'requesting instance',

6
salt/cloud/clouds/openstack.py
View File

@ -55,6 +55,8 @@ Set up in the cloud configuration at ``/etc/salt/cloud.providers`` or
base_url: http://192.168.1.101:3000/v2/12345
provider: openstack
userdata_file: /tmp/userdata.txt
# config_drive is required for userdata at rackspace
config_drive: True
For in-house Openstack Essex installation, libcloud needs the service_type :
@ -533,6 +535,10 @@ def request_instance(vm_=None, call=None):
with salt.utils.fopen(userdata_file, 'r') as fp:
kwargs['ex_userdata'] = fp.read()
kwargs['ex_config_drive'] = config.get_cloud_config_value(
'config_drive', vm_, __opts__, search_global=False
)
salt.utils.cloud.fire_event(
'event',
'requesting instance',

2
salt/minion.py
View File

@ -1809,7 +1809,7 @@ class Minion(MinionBase):
self._running = False
if getattr(self, 'poller', None) is not None:
if isinstance(self.poller.sockets, dict):
for socket in self.poller.sockets:
for socket in self.poller.sockets.keys():
if socket.closed is False:
socket.close()
self.poller.unregister(socket)

8
salt/modules/aptpkg.py
View File

@ -1347,10 +1347,10 @@ def mod_repo(repo, saltenv='base', **kwargs):
# implementation via apt-add-repository. The code path for
# secure PPAs should be the same as urllib method
if HAS_SOFTWAREPROPERTIES and 'ppa_auth' not in kwargs:
try:
get_repo(repo)
return {repo: ''}
except Exception:
repo_info = get_repo(repo)
if repo_info:
return {repo: repo_info}
else:
if float(__grains__['osrelease']) < 12.04:
cmd = 'apt-add-repository {0}'.format(repo)
else:

53
salt/modules/file.py
View File

@ -2908,6 +2908,59 @@ def check_managed(
return True, 'The file {0} is in the correct state'.format(name)
def check_managed_changes(
name,
source,
source_hash,
user,
group,
mode,
template,
context,
defaults,
saltenv,
contents=None,
**kwargs):
'''
Return a dictionary of what changes need to be made for a file
CLI Example:
.. code-block:: bash
salt '*' file.check_managed_changes /etc/httpd/conf.d/httpd.conf salt://http/httpd.conf '{hash_type: 'md5', 'hsum': <md5sum>}' root, root, '755' jinja True None None base
'''
# If the source is a list then find which file exists
source, source_hash = source_list(source, # pylint: disable=W0633
source_hash,
saltenv)
sfn = ''
source_sum = None
if contents is None:
# Gather the source file from the server
sfn, source_sum, comments = get_managed(
name,
template,
source,
source_hash,
user,
group,
mode,
saltenv,
context,
defaults,
**kwargs)
if comments:
__clean_tmp(sfn)
return False, comments
changes = check_file_meta(name, sfn, source, source_sum, user,
group, mode, saltenv, template, contents)
__clean_tmp(sfn)
return changes
def check_file_meta(
name,
sfn,

22
salt/states/file.py
View File

@ -1335,7 +1335,8 @@ def managed(name,
try:
if __opts__['test']:
ret['result'], ret['comment'] = __salt__['file.check_managed'](
ret['result'] = None
ret['changes'] = __salt__['file.check_managed_changes'](
name,
source,
source_hash,
@ -1349,6 +1350,12 @@ def managed(name,
contents,
**kwargs
)
if ret['changes']:
ret['comment'] = 'The file {0} is set to be changed'.format(name)
else:
ret['comment'] = 'The file {0} is in the correct state'.format(name)
return ret
# If the source is a list then find which file exists
@ -2285,7 +2292,7 @@ def replace(name,
not_found_content=None,
backup='.bak',
show_changes=True):
'''
r'''
Maintain an edit in a file
.. versionadded:: 0.17.0
@ -2293,6 +2300,17 @@ def replace(name,
Params are identical to the remote execution function :mod:`file.replace
<salt.modules.file.replace>`.
For complex regex patterns it can be useful to avoid the need for complex
quoting and escape sequences by making use of YAML's multiline string
syntax.
.. code-block:: yaml
complex_search_and_replace:
file.replace:
# <...snip...>
- pattern: |
CentOS \(2.6.32[^\n]+\n\s+root[^\n]+\n\)+
'''
name = os.path.expanduser(name)

2
salt/utils/event.py
View File

@ -419,7 +419,7 @@ class SaltEvent(object):
# that poller gets garbage collected. The Poller itself, its
# registered sockets and the Context
if isinstance(self.poller.sockets, dict):
for socket in self.poller.sockets:
for socket in self.poller.sockets.keys():
if socket.closed is False:
socket.setsockopt(zmq.LINGER, linger)
socket.close()