salt/tests/unit/modules/test_status.py

270 lines
10 KiB
Python
Raw Normal View History

# -*- coding: utf-8 -*-
# Import Python libs
from __future__ import absolute_import, print_function, unicode_literals
import os
# Import Salt Libs
Use explicit unicode strings + break up salt.utils This PR is part of what will be an ongoing effort to use explicit unicode strings in Salt. Because Python 3 does not suport Python 2's raw unicode string syntax (i.e. `ur'\d+'`), we must use `salt.utils.locales.sdecode()` to ensure that the raw string is unicode. However, because of how `salt/utils/__init__.py` has evolved into the hulking monstrosity it is today, this means importing a large module in places where it is not needed, which could negatively impact performance. For this reason, this PR also breaks out some of the functions from `salt/utils/__init__.py` into new/existing modules under `salt/utils/`. The long term goal will be that the modules within this directory do not depend on importing `salt.utils`. A summary of the changes in this PR is as follows: * Moves the following functions from `salt.utils` to new locations (including a deprecation warning if invoked from `salt.utils`): `to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`, `dequote`, `is_hex`, `is_bin_str`, `rand_string`, `contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`, `which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`, `is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`, `is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`, `is_openbsd`, `is_aix` * Moves the functions already deprecated by @rallytime to the bottom of `salt/utils/__init__.py` for better organization, so we can keep the deprecated ones separate from the ones yet to be deprecated as we continue to break up `salt.utils` * Updates `salt/*.py` and all files under `salt/client/` to use explicit unicode string literals. * Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils import foo` becomes `import salt.utils.foo as foo`). * Renames the `test.rand_str` function to `test.random_hash` to more accurately reflect what it does * Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`) such that it returns a string matching the passed size. Previously this function would get `size` bytes from `os.urandom()`, base64-encode it, and return the result, which would in most cases not be equal to the passed size.
2017-07-25 01:47:15 +00:00
import salt.utils.platform
2017-02-19 16:43:29 +00:00
import salt.modules.status as status
[develop] Merge forward from 2016.3 to develop (#32494) * fix sorting by latest version when called with an attribute * remove reference to master_alive_check * Fixes saltstack/salt#28262 * Resolve memory leak in authentication * outputter virt_list does not exist anymore * Update proxmox documentation * Fix documentation on boto_asg and boto_elb modules and states * modules.win_timezone: don't list all zones in debug log * Correcty index glusterfs bricks Fixes issue #32311 * Cleaner deprecation process with decorators * Add deprecation decorator scaffold * Capture type error and unhandled exceptions while function calls * Aware of the current and future version of deprecation * Implement initially is_deprecated decorator * Add an alias for the capitalization * Fix capitalization easier way * Remove an extra line * Add successor name to the deprecation decorator. * Granulate logging and error messages. * Implement function swapper * Raise later the caught exception * Clarify exception message * Save function original name * Remove an extra line * Hide an alternative hidden function name in the error message, preserving the error itself * Rename variable as private * Add a method to detect if a function is using its previous version * Message to the log and/or raise an exception accordingly to the status of used function * Log an error along with the exception * Add internal method documentation * Add documentation and usage process for decorator "is_deprecated" * Add documentation and process usage for the decorator "with_deprecated" * Hide private method name * Fix PEP8, re-word the error message * Deprecate basic uptime function * Add initial decorator unit test * Rename old/new functions, mock versions * Move frequent data to the test setup * Add logging on EOL exception * Rename and document high to low version test on is_deprecated * Implement a test on low to high version of is_deprecated decorator * Add a correction to the test description * Remove a dead code * Implement a test for high to low version on is_deprecated, using with_successor param * Correct typso adn mistaeks * Implement high to low version with successor param on is_deprecated * Setup a virtual name for the module * Implement test for with_deprecated should raise an exception if same deprecated function not found * Implement test for with_deprecated an old function is picked up if configured * Correct test description purpose * Implement test with_deprecated when no deprecation is requested * Add logging test to the configured deprecation request * Add logging testing when deprecated version wasn't requested * Implement test EOL for with_deprecated decorator * Correct test explanation * Rename the test * Implement with_deprecated no EOL, deprecated other function name * Implement with_deprecated, deprecated other function name, EOL reached * Add test description for the with_deprecated + with_name + EOL * Fix confusing test names * Add logging test to the is_deprecated decorator when function as not found. * Add more test point to each test, remove empty lines * Bugfix: at certain conditions a wrong alias name is reported to the log * Fix a typo in a comment * Add test for the logging * Disable a pylint: None will _never_ be raised * Fix test for the deprecated "status.uptime" version * Bugfix: Do not yank raised exceptions * Remove unnecessary decorator * Add test for the new uptime * Add test for the new uptime fails when /proc/uptime does not exists * Rename old test case * Skip test for the UTC time, unless freeze time is used. * Fix pylint * Fix documentation * Bugfix: proxy-pass the docstring of the decorated function * Lint fix * Fixes saltstack/salt#28262 for 2015.5 branch * Update master config docs * Improve git_pillar documentation/logging * Add note about different behavior of top file in git_pillar * Make log entry for a missing pillar SLS file more accurate for git_pillar * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * Update glusterfs_test to be inline with #32312 * Fix salt-cloud paralell provisioning Closes #31632 * Ignore Raspbian in service.py __virtual__ (#32421) * Ignore Raspbian in service.py __virtual__ This prevents more than one execution module from trying to load as the service virtual module. Refs: #32413 * pack __salt__ before loading provider overrides We can (and should) pack here since we're just packing a reference to the object. __salt__ needs to be available when we're loading our provider overrides * Fix broken __salt__ dict in provider override Using ret.items() here sets ``__salt__`` to its items (tuple containing function name and reference), breaking usage of ``__salt__`` inside overridden functions. * Merge #32293 with test fixes (#32418) * Fix issue #11497 * Remove check for working directory presence in tests * Fix Domainname introspection Default value needs to be extracted from the container itself, because dockerd set Domainname value when network_mode=host. * Add pgjsonb_queue to queue doc index * Pylint fixes * Pass parser options into batch mode Resolves #31738 * Changed the target file in file.symlink test (#32443) * Argument name in docs should match actual arg name (#32445) Fixes #31851 * tests.integration: bypass MacOS TMPDIR, gettempdir (#32447) Updates 0edd532, 8f558a5. When logging in as root over `ssh root@host`, `$TMPDIR` and `tempfile.gettempdir()` are both set to a variation of: ``` /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/ ``` When logging in as root over `sudo -i`, `$TMPDIR` is unset and `tempfile.gettempdir()` is set to `/tmp`. My guess is that the second case is an unintended or uncorrected omision by Apple as they have introduced the longer, randomized temp path in a recent version of MacOS. * Issue #28706: Fix state user.present behavior. (#32448) - As mentionned in issue #28706, state user.present no longer remove user from groups if the keyword 'groups' with empty value '[]' is not explicitly set, salt will assume current groups are still wanted. * tests.integration: fix 4230c8a * Move the tables of virtual modules to individual documentation pages * Add new doc pages to toctree * Add external ref to windows package manager docs * Improve docstrings * Add documentation on virtual module provider overrides to the module docs * Clarify the scope of the provider param in states. * Add link to provider override docs to all package providers * Add link to provider override docs to all service providers * Add link to provider override docs to all user providers * dd link to provider override docs to all shadow providers * Add link to provider override docs to all group providers * Backport 31164 and 31364 (#32474) * Don't send REQ while another one is waiting for response. The message has to be removed from the queue the only *after* it's already processed to don't confuse send() functionality that expects empty queue means: there's no active sendings. * Fixed zeromq ReqMessageClient destroy * Add link to provider override docs to opkg.py This is a companion to https://github.com/saltstack/salt/pull/32458, but this module was not added until the 2016.3 branch, so the documentation is being updated there for this module. * Add documentation for some master/minion configs (#32454) Refs #32400 Adds docs for: - cli_summary - event_return_queue - event_return_whitelist - event_return_blacklist - file_recv_max_size - fileserver_followsymlinks - fileserver_ignoresymlinks - fileserver_limit_traversal * Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440) * Automatically detect MySQL password column * Fix changing password in MySQL 5.7 * Fix lint test * Fix unit tests (?) They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that. * Additional unit test fix * Only unsub if we have a jid Closes #32479
2016-04-11 23:07:15 +00:00
from salt.exceptions import CommandExecutionError
from salt.ext import six
# Import Salt Testing Libs
2017-02-19 16:43:29 +00:00
from tests.support.mixins import LoaderModuleMockMixin
from tests.support.unit import TestCase
from tests.support.mock import (
MagicMock,
patch,
mock_open,
)
2017-02-19 16:43:29 +00:00
class StatusTestCase(TestCase, LoaderModuleMockMixin):
'''
test modules.status functions
'''
def setup_loader_modules(self):
return {status: {}}
2017-02-19 16:43:29 +00:00
def _set_up_test_uptime(self):
'''
Define common mock data for status.uptime tests
'''
class MockData(object):
'''
Store mock data
'''
m = MockData()
m.now = 1477004312
m.ut = 1540154.00
m.idle = 3047777.32
m.ret = {
'users': 3,
'seconds': 1540154,
'since_t': 1475464158,
'days': 17,
'since_iso': '2016-10-03T03:09:18',
'time': '19:49',
}
return m
def _set_up_test_uptime_sunos(self):
'''
Define common mock data for cmd.run_all for status.uptime on SunOS
[develop] Merge forward from 2016.3 to develop (#32494) * fix sorting by latest version when called with an attribute * remove reference to master_alive_check * Fixes saltstack/salt#28262 * Resolve memory leak in authentication * outputter virt_list does not exist anymore * Update proxmox documentation * Fix documentation on boto_asg and boto_elb modules and states * modules.win_timezone: don't list all zones in debug log * Correcty index glusterfs bricks Fixes issue #32311 * Cleaner deprecation process with decorators * Add deprecation decorator scaffold * Capture type error and unhandled exceptions while function calls * Aware of the current and future version of deprecation * Implement initially is_deprecated decorator * Add an alias for the capitalization * Fix capitalization easier way * Remove an extra line * Add successor name to the deprecation decorator. * Granulate logging and error messages. * Implement function swapper * Raise later the caught exception * Clarify exception message * Save function original name * Remove an extra line * Hide an alternative hidden function name in the error message, preserving the error itself * Rename variable as private * Add a method to detect if a function is using its previous version * Message to the log and/or raise an exception accordingly to the status of used function * Log an error along with the exception * Add internal method documentation * Add documentation and usage process for decorator "is_deprecated" * Add documentation and process usage for the decorator "with_deprecated" * Hide private method name * Fix PEP8, re-word the error message * Deprecate basic uptime function * Add initial decorator unit test * Rename old/new functions, mock versions * Move frequent data to the test setup * Add logging on EOL exception * Rename and document high to low version test on is_deprecated * Implement a test on low to high version of is_deprecated decorator * Add a correction to the test description * Remove a dead code * Implement a test for high to low version on is_deprecated, using with_successor param * Correct typso adn mistaeks * Implement high to low version with successor param on is_deprecated * Setup a virtual name for the module * Implement test for with_deprecated should raise an exception if same deprecated function not found * Implement test for with_deprecated an old function is picked up if configured * Correct test description purpose * Implement test with_deprecated when no deprecation is requested * Add logging test to the configured deprecation request * Add logging testing when deprecated version wasn't requested * Implement test EOL for with_deprecated decorator * Correct test explanation * Rename the test * Implement with_deprecated no EOL, deprecated other function name * Implement with_deprecated, deprecated other function name, EOL reached * Add test description for the with_deprecated + with_name + EOL * Fix confusing test names * Add logging test to the is_deprecated decorator when function as not found. * Add more test point to each test, remove empty lines * Bugfix: at certain conditions a wrong alias name is reported to the log * Fix a typo in a comment * Add test for the logging * Disable a pylint: None will _never_ be raised * Fix test for the deprecated "status.uptime" version * Bugfix: Do not yank raised exceptions * Remove unnecessary decorator * Add test for the new uptime * Add test for the new uptime fails when /proc/uptime does not exists * Rename old test case * Skip test for the UTC time, unless freeze time is used. * Fix pylint * Fix documentation * Bugfix: proxy-pass the docstring of the decorated function * Lint fix * Fixes saltstack/salt#28262 for 2015.5 branch * Update master config docs * Improve git_pillar documentation/logging * Add note about different behavior of top file in git_pillar * Make log entry for a missing pillar SLS file more accurate for git_pillar * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * Update glusterfs_test to be inline with #32312 * Fix salt-cloud paralell provisioning Closes #31632 * Ignore Raspbian in service.py __virtual__ (#32421) * Ignore Raspbian in service.py __virtual__ This prevents more than one execution module from trying to load as the service virtual module. Refs: #32413 * pack __salt__ before loading provider overrides We can (and should) pack here since we're just packing a reference to the object. __salt__ needs to be available when we're loading our provider overrides * Fix broken __salt__ dict in provider override Using ret.items() here sets ``__salt__`` to its items (tuple containing function name and reference), breaking usage of ``__salt__`` inside overridden functions. * Merge #32293 with test fixes (#32418) * Fix issue #11497 * Remove check for working directory presence in tests * Fix Domainname introspection Default value needs to be extracted from the container itself, because dockerd set Domainname value when network_mode=host. * Add pgjsonb_queue to queue doc index * Pylint fixes * Pass parser options into batch mode Resolves #31738 * Changed the target file in file.symlink test (#32443) * Argument name in docs should match actual arg name (#32445) Fixes #31851 * tests.integration: bypass MacOS TMPDIR, gettempdir (#32447) Updates 0edd532, 8f558a5. When logging in as root over `ssh root@host`, `$TMPDIR` and `tempfile.gettempdir()` are both set to a variation of: ``` /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/ ``` When logging in as root over `sudo -i`, `$TMPDIR` is unset and `tempfile.gettempdir()` is set to `/tmp`. My guess is that the second case is an unintended or uncorrected omision by Apple as they have introduced the longer, randomized temp path in a recent version of MacOS. * Issue #28706: Fix state user.present behavior. (#32448) - As mentionned in issue #28706, state user.present no longer remove user from groups if the keyword 'groups' with empty value '[]' is not explicitly set, salt will assume current groups are still wanted. * tests.integration: fix 4230c8a * Move the tables of virtual modules to individual documentation pages * Add new doc pages to toctree * Add external ref to windows package manager docs * Improve docstrings * Add documentation on virtual module provider overrides to the module docs * Clarify the scope of the provider param in states. * Add link to provider override docs to all package providers * Add link to provider override docs to all service providers * Add link to provider override docs to all user providers * dd link to provider override docs to all shadow providers * Add link to provider override docs to all group providers * Backport 31164 and 31364 (#32474) * Don't send REQ while another one is waiting for response. The message has to be removed from the queue the only *after* it's already processed to don't confuse send() functionality that expects empty queue means: there's no active sendings. * Fixed zeromq ReqMessageClient destroy * Add link to provider override docs to opkg.py This is a companion to https://github.com/saltstack/salt/pull/32458, but this module was not added until the 2016.3 branch, so the documentation is being updated there for this module. * Add documentation for some master/minion configs (#32454) Refs #32400 Adds docs for: - cli_summary - event_return_queue - event_return_whitelist - event_return_blacklist - file_recv_max_size - fileserver_followsymlinks - fileserver_ignoresymlinks - fileserver_limit_traversal * Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440) * Automatically detect MySQL password column * Fix changing password in MySQL 5.7 * Fix lint test * Fix unit tests (?) They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that. * Additional unit test fix * Only unsub if we have a jid Closes #32479
2016-04-11 23:07:15 +00:00
'''
class MockData(object):
'''
Store mock data
'''
[develop] Merge forward from 2016.3 to develop (#32494) * fix sorting by latest version when called with an attribute * remove reference to master_alive_check * Fixes saltstack/salt#28262 * Resolve memory leak in authentication * outputter virt_list does not exist anymore * Update proxmox documentation * Fix documentation on boto_asg and boto_elb modules and states * modules.win_timezone: don't list all zones in debug log * Correcty index glusterfs bricks Fixes issue #32311 * Cleaner deprecation process with decorators * Add deprecation decorator scaffold * Capture type error and unhandled exceptions while function calls * Aware of the current and future version of deprecation * Implement initially is_deprecated decorator * Add an alias for the capitalization * Fix capitalization easier way * Remove an extra line * Add successor name to the deprecation decorator. * Granulate logging and error messages. * Implement function swapper * Raise later the caught exception * Clarify exception message * Save function original name * Remove an extra line * Hide an alternative hidden function name in the error message, preserving the error itself * Rename variable as private * Add a method to detect if a function is using its previous version * Message to the log and/or raise an exception accordingly to the status of used function * Log an error along with the exception * Add internal method documentation * Add documentation and usage process for decorator "is_deprecated" * Add documentation and process usage for the decorator "with_deprecated" * Hide private method name * Fix PEP8, re-word the error message * Deprecate basic uptime function * Add initial decorator unit test * Rename old/new functions, mock versions * Move frequent data to the test setup * Add logging on EOL exception * Rename and document high to low version test on is_deprecated * Implement a test on low to high version of is_deprecated decorator * Add a correction to the test description * Remove a dead code * Implement a test for high to low version on is_deprecated, using with_successor param * Correct typso adn mistaeks * Implement high to low version with successor param on is_deprecated * Setup a virtual name for the module * Implement test for with_deprecated should raise an exception if same deprecated function not found * Implement test for with_deprecated an old function is picked up if configured * Correct test description purpose * Implement test with_deprecated when no deprecation is requested * Add logging test to the configured deprecation request * Add logging testing when deprecated version wasn't requested * Implement test EOL for with_deprecated decorator * Correct test explanation * Rename the test * Implement with_deprecated no EOL, deprecated other function name * Implement with_deprecated, deprecated other function name, EOL reached * Add test description for the with_deprecated + with_name + EOL * Fix confusing test names * Add logging test to the is_deprecated decorator when function as not found. * Add more test point to each test, remove empty lines * Bugfix: at certain conditions a wrong alias name is reported to the log * Fix a typo in a comment * Add test for the logging * Disable a pylint: None will _never_ be raised * Fix test for the deprecated "status.uptime" version * Bugfix: Do not yank raised exceptions * Remove unnecessary decorator * Add test for the new uptime * Add test for the new uptime fails when /proc/uptime does not exists * Rename old test case * Skip test for the UTC time, unless freeze time is used. * Fix pylint * Fix documentation * Bugfix: proxy-pass the docstring of the decorated function * Lint fix * Fixes saltstack/salt#28262 for 2015.5 branch * Update master config docs * Improve git_pillar documentation/logging * Add note about different behavior of top file in git_pillar * Make log entry for a missing pillar SLS file more accurate for git_pillar * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * Update glusterfs_test to be inline with #32312 * Fix salt-cloud paralell provisioning Closes #31632 * Ignore Raspbian in service.py __virtual__ (#32421) * Ignore Raspbian in service.py __virtual__ This prevents more than one execution module from trying to load as the service virtual module. Refs: #32413 * pack __salt__ before loading provider overrides We can (and should) pack here since we're just packing a reference to the object. __salt__ needs to be available when we're loading our provider overrides * Fix broken __salt__ dict in provider override Using ret.items() here sets ``__salt__`` to its items (tuple containing function name and reference), breaking usage of ``__salt__`` inside overridden functions. * Merge #32293 with test fixes (#32418) * Fix issue #11497 * Remove check for working directory presence in tests * Fix Domainname introspection Default value needs to be extracted from the container itself, because dockerd set Domainname value when network_mode=host. * Add pgjsonb_queue to queue doc index * Pylint fixes * Pass parser options into batch mode Resolves #31738 * Changed the target file in file.symlink test (#32443) * Argument name in docs should match actual arg name (#32445) Fixes #31851 * tests.integration: bypass MacOS TMPDIR, gettempdir (#32447) Updates 0edd532, 8f558a5. When logging in as root over `ssh root@host`, `$TMPDIR` and `tempfile.gettempdir()` are both set to a variation of: ``` /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/ ``` When logging in as root over `sudo -i`, `$TMPDIR` is unset and `tempfile.gettempdir()` is set to `/tmp`. My guess is that the second case is an unintended or uncorrected omision by Apple as they have introduced the longer, randomized temp path in a recent version of MacOS. * Issue #28706: Fix state user.present behavior. (#32448) - As mentionned in issue #28706, state user.present no longer remove user from groups if the keyword 'groups' with empty value '[]' is not explicitly set, salt will assume current groups are still wanted. * tests.integration: fix 4230c8a * Move the tables of virtual modules to individual documentation pages * Add new doc pages to toctree * Add external ref to windows package manager docs * Improve docstrings * Add documentation on virtual module provider overrides to the module docs * Clarify the scope of the provider param in states. * Add link to provider override docs to all package providers * Add link to provider override docs to all service providers * Add link to provider override docs to all user providers * dd link to provider override docs to all shadow providers * Add link to provider override docs to all group providers * Backport 31164 and 31364 (#32474) * Don't send REQ while another one is waiting for response. The message has to be removed from the queue the only *after* it's already processed to don't confuse send() functionality that expects empty queue means: there's no active sendings. * Fixed zeromq ReqMessageClient destroy * Add link to provider override docs to opkg.py This is a companion to https://github.com/saltstack/salt/pull/32458, but this module was not added until the 2016.3 branch, so the documentation is being updated there for this module. * Add documentation for some master/minion configs (#32454) Refs #32400 Adds docs for: - cli_summary - event_return_queue - event_return_whitelist - event_return_blacklist - file_recv_max_size - fileserver_followsymlinks - fileserver_ignoresymlinks - fileserver_limit_traversal * Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440) * Automatically detect MySQL password column * Fix changing password in MySQL 5.7 * Fix lint test * Fix unit tests (?) They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that. * Additional unit test fix * Only unsub if we have a jid Closes #32479
2016-04-11 23:07:15 +00:00
m = MockData()
m.ret = {
'retcode': 0,
'stdout': 'unix:0:system_misc:boot_time 1475464158',
}
return m
def test_uptime_linux(self):
'''
Test modules.status.uptime function for Linux
'''
m = self._set_up_test_uptime()
[develop] Merge forward from 2016.3 to develop (#32494) * fix sorting by latest version when called with an attribute * remove reference to master_alive_check * Fixes saltstack/salt#28262 * Resolve memory leak in authentication * outputter virt_list does not exist anymore * Update proxmox documentation * Fix documentation on boto_asg and boto_elb modules and states * modules.win_timezone: don't list all zones in debug log * Correcty index glusterfs bricks Fixes issue #32311 * Cleaner deprecation process with decorators * Add deprecation decorator scaffold * Capture type error and unhandled exceptions while function calls * Aware of the current and future version of deprecation * Implement initially is_deprecated decorator * Add an alias for the capitalization * Fix capitalization easier way * Remove an extra line * Add successor name to the deprecation decorator. * Granulate logging and error messages. * Implement function swapper * Raise later the caught exception * Clarify exception message * Save function original name * Remove an extra line * Hide an alternative hidden function name in the error message, preserving the error itself * Rename variable as private * Add a method to detect if a function is using its previous version * Message to the log and/or raise an exception accordingly to the status of used function * Log an error along with the exception * Add internal method documentation * Add documentation and usage process for decorator "is_deprecated" * Add documentation and process usage for the decorator "with_deprecated" * Hide private method name * Fix PEP8, re-word the error message * Deprecate basic uptime function * Add initial decorator unit test * Rename old/new functions, mock versions * Move frequent data to the test setup * Add logging on EOL exception * Rename and document high to low version test on is_deprecated * Implement a test on low to high version of is_deprecated decorator * Add a correction to the test description * Remove a dead code * Implement a test for high to low version on is_deprecated, using with_successor param * Correct typso adn mistaeks * Implement high to low version with successor param on is_deprecated * Setup a virtual name for the module * Implement test for with_deprecated should raise an exception if same deprecated function not found * Implement test for with_deprecated an old function is picked up if configured * Correct test description purpose * Implement test with_deprecated when no deprecation is requested * Add logging test to the configured deprecation request * Add logging testing when deprecated version wasn't requested * Implement test EOL for with_deprecated decorator * Correct test explanation * Rename the test * Implement with_deprecated no EOL, deprecated other function name * Implement with_deprecated, deprecated other function name, EOL reached * Add test description for the with_deprecated + with_name + EOL * Fix confusing test names * Add logging test to the is_deprecated decorator when function as not found. * Add more test point to each test, remove empty lines * Bugfix: at certain conditions a wrong alias name is reported to the log * Fix a typo in a comment * Add test for the logging * Disable a pylint: None will _never_ be raised * Fix test for the deprecated "status.uptime" version * Bugfix: Do not yank raised exceptions * Remove unnecessary decorator * Add test for the new uptime * Add test for the new uptime fails when /proc/uptime does not exists * Rename old test case * Skip test for the UTC time, unless freeze time is used. * Fix pylint * Fix documentation * Bugfix: proxy-pass the docstring of the decorated function * Lint fix * Fixes saltstack/salt#28262 for 2015.5 branch * Update master config docs * Improve git_pillar documentation/logging * Add note about different behavior of top file in git_pillar * Make log entry for a missing pillar SLS file more accurate for git_pillar * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * Update glusterfs_test to be inline with #32312 * Fix salt-cloud paralell provisioning Closes #31632 * Ignore Raspbian in service.py __virtual__ (#32421) * Ignore Raspbian in service.py __virtual__ This prevents more than one execution module from trying to load as the service virtual module. Refs: #32413 * pack __salt__ before loading provider overrides We can (and should) pack here since we're just packing a reference to the object. __salt__ needs to be available when we're loading our provider overrides * Fix broken __salt__ dict in provider override Using ret.items() here sets ``__salt__`` to its items (tuple containing function name and reference), breaking usage of ``__salt__`` inside overridden functions. * Merge #32293 with test fixes (#32418) * Fix issue #11497 * Remove check for working directory presence in tests * Fix Domainname introspection Default value needs to be extracted from the container itself, because dockerd set Domainname value when network_mode=host. * Add pgjsonb_queue to queue doc index * Pylint fixes * Pass parser options into batch mode Resolves #31738 * Changed the target file in file.symlink test (#32443) * Argument name in docs should match actual arg name (#32445) Fixes #31851 * tests.integration: bypass MacOS TMPDIR, gettempdir (#32447) Updates 0edd532, 8f558a5. When logging in as root over `ssh root@host`, `$TMPDIR` and `tempfile.gettempdir()` are both set to a variation of: ``` /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/ ``` When logging in as root over `sudo -i`, `$TMPDIR` is unset and `tempfile.gettempdir()` is set to `/tmp`. My guess is that the second case is an unintended or uncorrected omision by Apple as they have introduced the longer, randomized temp path in a recent version of MacOS. * Issue #28706: Fix state user.present behavior. (#32448) - As mentionned in issue #28706, state user.present no longer remove user from groups if the keyword 'groups' with empty value '[]' is not explicitly set, salt will assume current groups are still wanted. * tests.integration: fix 4230c8a * Move the tables of virtual modules to individual documentation pages * Add new doc pages to toctree * Add external ref to windows package manager docs * Improve docstrings * Add documentation on virtual module provider overrides to the module docs * Clarify the scope of the provider param in states. * Add link to provider override docs to all package providers * Add link to provider override docs to all service providers * Add link to provider override docs to all user providers * dd link to provider override docs to all shadow providers * Add link to provider override docs to all group providers * Backport 31164 and 31364 (#32474) * Don't send REQ while another one is waiting for response. The message has to be removed from the queue the only *after* it's already processed to don't confuse send() functionality that expects empty queue means: there's no active sendings. * Fixed zeromq ReqMessageClient destroy * Add link to provider override docs to opkg.py This is a companion to https://github.com/saltstack/salt/pull/32458, but this module was not added until the 2016.3 branch, so the documentation is being updated there for this module. * Add documentation for some master/minion configs (#32454) Refs #32400 Adds docs for: - cli_summary - event_return_queue - event_return_whitelist - event_return_blacklist - file_recv_max_size - fileserver_followsymlinks - fileserver_ignoresymlinks - fileserver_limit_traversal * Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440) * Automatically detect MySQL password column * Fix changing password in MySQL 5.7 * Fix lint test * Fix unit tests (?) They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that. * Additional unit test fix * Only unsub if we have a jid Closes #32479
2016-04-11 23:07:15 +00:00
Use explicit unicode strings + break up salt.utils This PR is part of what will be an ongoing effort to use explicit unicode strings in Salt. Because Python 3 does not suport Python 2's raw unicode string syntax (i.e. `ur'\d+'`), we must use `salt.utils.locales.sdecode()` to ensure that the raw string is unicode. However, because of how `salt/utils/__init__.py` has evolved into the hulking monstrosity it is today, this means importing a large module in places where it is not needed, which could negatively impact performance. For this reason, this PR also breaks out some of the functions from `salt/utils/__init__.py` into new/existing modules under `salt/utils/`. The long term goal will be that the modules within this directory do not depend on importing `salt.utils`. A summary of the changes in this PR is as follows: * Moves the following functions from `salt.utils` to new locations (including a deprecation warning if invoked from `salt.utils`): `to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`, `dequote`, `is_hex`, `is_bin_str`, `rand_string`, `contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`, `which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`, `is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`, `is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`, `is_openbsd`, `is_aix` * Moves the functions already deprecated by @rallytime to the bottom of `salt/utils/__init__.py` for better organization, so we can keep the deprecated ones separate from the ones yet to be deprecated as we continue to break up `salt.utils` * Updates `salt/*.py` and all files under `salt/client/` to use explicit unicode string literals. * Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils import foo` becomes `import salt.utils.foo as foo`). * Renames the `test.rand_str` function to `test.random_hash` to more accurately reflect what it does * Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`) such that it returns a string matching the passed size. Previously this function would get `size` bytes from `os.urandom()`, base64-encode it, and return the result, which would in most cases not be equal to the passed size.
2017-07-25 01:47:15 +00:00
with patch.multiple(salt.utils.platform,
2017-04-10 13:00:57 +00:00
is_linux=MagicMock(return_value=True),
is_sunos=MagicMock(return_value=False),
is_darwin=MagicMock(return_value=False),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=os.linesep.join(['1', '2', '3']))}), \
patch('time.time', MagicMock(return_value=m.now)), \
patch('os.path.exists', MagicMock(return_value=True)):
proc_uptime = salt.utils.stringutils.to_str('{0} {1}'.format(m.ut, m.idle))
with patch('salt.utils.files.fopen', mock_open(read_data=proc_uptime)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
with patch('os.path.exists', MagicMock(return_value=False)):
with self.assertRaises(CommandExecutionError):
status.uptime()
2017-04-10 13:00:57 +00:00
def test_uptime_sunos(self):
'''
Test modules.status.uptime function for SunOS
'''
m = self._set_up_test_uptime()
m2 = self._set_up_test_uptime_sunos()
Use explicit unicode strings + break up salt.utils This PR is part of what will be an ongoing effort to use explicit unicode strings in Salt. Because Python 3 does not suport Python 2's raw unicode string syntax (i.e. `ur'\d+'`), we must use `salt.utils.locales.sdecode()` to ensure that the raw string is unicode. However, because of how `salt/utils/__init__.py` has evolved into the hulking monstrosity it is today, this means importing a large module in places where it is not needed, which could negatively impact performance. For this reason, this PR also breaks out some of the functions from `salt/utils/__init__.py` into new/existing modules under `salt/utils/`. The long term goal will be that the modules within this directory do not depend on importing `salt.utils`. A summary of the changes in this PR is as follows: * Moves the following functions from `salt.utils` to new locations (including a deprecation warning if invoked from `salt.utils`): `to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`, `dequote`, `is_hex`, `is_bin_str`, `rand_string`, `contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`, `which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`, `is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`, `is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`, `is_openbsd`, `is_aix` * Moves the functions already deprecated by @rallytime to the bottom of `salt/utils/__init__.py` for better organization, so we can keep the deprecated ones separate from the ones yet to be deprecated as we continue to break up `salt.utils` * Updates `salt/*.py` and all files under `salt/client/` to use explicit unicode string literals. * Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils import foo` becomes `import salt.utils.foo as foo`). * Renames the `test.rand_str` function to `test.random_hash` to more accurately reflect what it does * Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`) such that it returns a string matching the passed size. Previously this function would get `size` bytes from `os.urandom()`, base64-encode it, and return the result, which would in most cases not be equal to the passed size.
2017-07-25 01:47:15 +00:00
with patch.multiple(salt.utils.platform,
2017-04-10 13:00:57 +00:00
is_linux=MagicMock(return_value=False),
is_sunos=MagicMock(return_value=True),
is_darwin=MagicMock(return_value=False),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=os.linesep.join(['1', '2', '3'])),
'cmd.run_all': MagicMock(return_value=m2.ret)}), \
patch('time.time', MagicMock(return_value=m.now)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
def test_uptime_macos(self):
'''
Test modules.status.uptime function for macOS
'''
m = self._set_up_test_uptime()
kern_boottime = ('{{ sec = {0}, usec = {1:0<6} }} Mon Oct 03 03:09:18.23 2016'
''.format(*six.text_type(m.now - m.ut).split('.')))
Use explicit unicode strings + break up salt.utils This PR is part of what will be an ongoing effort to use explicit unicode strings in Salt. Because Python 3 does not suport Python 2's raw unicode string syntax (i.e. `ur'\d+'`), we must use `salt.utils.locales.sdecode()` to ensure that the raw string is unicode. However, because of how `salt/utils/__init__.py` has evolved into the hulking monstrosity it is today, this means importing a large module in places where it is not needed, which could negatively impact performance. For this reason, this PR also breaks out some of the functions from `salt/utils/__init__.py` into new/existing modules under `salt/utils/`. The long term goal will be that the modules within this directory do not depend on importing `salt.utils`. A summary of the changes in this PR is as follows: * Moves the following functions from `salt.utils` to new locations (including a deprecation warning if invoked from `salt.utils`): `to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`, `dequote`, `is_hex`, `is_bin_str`, `rand_string`, `contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`, `which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`, `is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`, `is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`, `is_openbsd`, `is_aix` * Moves the functions already deprecated by @rallytime to the bottom of `salt/utils/__init__.py` for better organization, so we can keep the deprecated ones separate from the ones yet to be deprecated as we continue to break up `salt.utils` * Updates `salt/*.py` and all files under `salt/client/` to use explicit unicode string literals. * Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils import foo` becomes `import salt.utils.foo as foo`). * Renames the `test.rand_str` function to `test.random_hash` to more accurately reflect what it does * Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`) such that it returns a string matching the passed size. Previously this function would get `size` bytes from `os.urandom()`, base64-encode it, and return the result, which would in most cases not be equal to the passed size.
2017-07-25 01:47:15 +00:00
with patch.multiple(salt.utils.platform,
2017-04-10 13:00:57 +00:00
is_linux=MagicMock(return_value=False),
is_sunos=MagicMock(return_value=False),
is_darwin=MagicMock(return_value=True),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=os.linesep.join(['1', '2', '3'])),
'sysctl.get': MagicMock(return_value=kern_boottime)}), \
patch('time.time', MagicMock(return_value=m.now)):
ret = status.uptime()
self.assertDictEqual(ret, m.ret)
with patch.dict(status.__salt__, {'sysctl.get': MagicMock(return_value='')}):
with self.assertRaises(CommandExecutionError):
status.uptime()
def test_uptime_return_success_not_supported(self):
[develop] Merge forward from 2016.3 to develop (#32494) * fix sorting by latest version when called with an attribute * remove reference to master_alive_check * Fixes saltstack/salt#28262 * Resolve memory leak in authentication * outputter virt_list does not exist anymore * Update proxmox documentation * Fix documentation on boto_asg and boto_elb modules and states * modules.win_timezone: don't list all zones in debug log * Correcty index glusterfs bricks Fixes issue #32311 * Cleaner deprecation process with decorators * Add deprecation decorator scaffold * Capture type error and unhandled exceptions while function calls * Aware of the current and future version of deprecation * Implement initially is_deprecated decorator * Add an alias for the capitalization * Fix capitalization easier way * Remove an extra line * Add successor name to the deprecation decorator. * Granulate logging and error messages. * Implement function swapper * Raise later the caught exception * Clarify exception message * Save function original name * Remove an extra line * Hide an alternative hidden function name in the error message, preserving the error itself * Rename variable as private * Add a method to detect if a function is using its previous version * Message to the log and/or raise an exception accordingly to the status of used function * Log an error along with the exception * Add internal method documentation * Add documentation and usage process for decorator "is_deprecated" * Add documentation and process usage for the decorator "with_deprecated" * Hide private method name * Fix PEP8, re-word the error message * Deprecate basic uptime function * Add initial decorator unit test * Rename old/new functions, mock versions * Move frequent data to the test setup * Add logging on EOL exception * Rename and document high to low version test on is_deprecated * Implement a test on low to high version of is_deprecated decorator * Add a correction to the test description * Remove a dead code * Implement a test for high to low version on is_deprecated, using with_successor param * Correct typso adn mistaeks * Implement high to low version with successor param on is_deprecated * Setup a virtual name for the module * Implement test for with_deprecated should raise an exception if same deprecated function not found * Implement test for with_deprecated an old function is picked up if configured * Correct test description purpose * Implement test with_deprecated when no deprecation is requested * Add logging test to the configured deprecation request * Add logging testing when deprecated version wasn't requested * Implement test EOL for with_deprecated decorator * Correct test explanation * Rename the test * Implement with_deprecated no EOL, deprecated other function name * Implement with_deprecated, deprecated other function name, EOL reached * Add test description for the with_deprecated + with_name + EOL * Fix confusing test names * Add logging test to the is_deprecated decorator when function as not found. * Add more test point to each test, remove empty lines * Bugfix: at certain conditions a wrong alias name is reported to the log * Fix a typo in a comment * Add test for the logging * Disable a pylint: None will _never_ be raised * Fix test for the deprecated "status.uptime" version * Bugfix: Do not yank raised exceptions * Remove unnecessary decorator * Add test for the new uptime * Add test for the new uptime fails when /proc/uptime does not exists * Rename old test case * Skip test for the UTC time, unless freeze time is used. * Fix pylint * Fix documentation * Bugfix: proxy-pass the docstring of the decorated function * Lint fix * Fixes saltstack/salt#28262 for 2015.5 branch * Update master config docs * Improve git_pillar documentation/logging * Add note about different behavior of top file in git_pillar * Make log entry for a missing pillar SLS file more accurate for git_pillar * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * Update glusterfs_test to be inline with #32312 * Fix salt-cloud paralell provisioning Closes #31632 * Ignore Raspbian in service.py __virtual__ (#32421) * Ignore Raspbian in service.py __virtual__ This prevents more than one execution module from trying to load as the service virtual module. Refs: #32413 * pack __salt__ before loading provider overrides We can (and should) pack here since we're just packing a reference to the object. __salt__ needs to be available when we're loading our provider overrides * Fix broken __salt__ dict in provider override Using ret.items() here sets ``__salt__`` to its items (tuple containing function name and reference), breaking usage of ``__salt__`` inside overridden functions. * Merge #32293 with test fixes (#32418) * Fix issue #11497 * Remove check for working directory presence in tests * Fix Domainname introspection Default value needs to be extracted from the container itself, because dockerd set Domainname value when network_mode=host. * Add pgjsonb_queue to queue doc index * Pylint fixes * Pass parser options into batch mode Resolves #31738 * Changed the target file in file.symlink test (#32443) * Argument name in docs should match actual arg name (#32445) Fixes #31851 * tests.integration: bypass MacOS TMPDIR, gettempdir (#32447) Updates 0edd532, 8f558a5. When logging in as root over `ssh root@host`, `$TMPDIR` and `tempfile.gettempdir()` are both set to a variation of: ``` /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/ ``` When logging in as root over `sudo -i`, `$TMPDIR` is unset and `tempfile.gettempdir()` is set to `/tmp`. My guess is that the second case is an unintended or uncorrected omision by Apple as they have introduced the longer, randomized temp path in a recent version of MacOS. * Issue #28706: Fix state user.present behavior. (#32448) - As mentionned in issue #28706, state user.present no longer remove user from groups if the keyword 'groups' with empty value '[]' is not explicitly set, salt will assume current groups are still wanted. * tests.integration: fix 4230c8a * Move the tables of virtual modules to individual documentation pages * Add new doc pages to toctree * Add external ref to windows package manager docs * Improve docstrings * Add documentation on virtual module provider overrides to the module docs * Clarify the scope of the provider param in states. * Add link to provider override docs to all package providers * Add link to provider override docs to all service providers * Add link to provider override docs to all user providers * dd link to provider override docs to all shadow providers * Add link to provider override docs to all group providers * Backport 31164 and 31364 (#32474) * Don't send REQ while another one is waiting for response. The message has to be removed from the queue the only *after* it's already processed to don't confuse send() functionality that expects empty queue means: there's no active sendings. * Fixed zeromq ReqMessageClient destroy * Add link to provider override docs to opkg.py This is a companion to https://github.com/saltstack/salt/pull/32458, but this module was not added until the 2016.3 branch, so the documentation is being updated there for this module. * Add documentation for some master/minion configs (#32454) Refs #32400 Adds docs for: - cli_summary - event_return_queue - event_return_whitelist - event_return_blacklist - file_recv_max_size - fileserver_followsymlinks - fileserver_ignoresymlinks - fileserver_limit_traversal * Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440) * Automatically detect MySQL password column * Fix changing password in MySQL 5.7 * Fix lint test * Fix unit tests (?) They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that. * Additional unit test fix * Only unsub if we have a jid Closes #32479
2016-04-11 23:07:15 +00:00
'''
Test modules.status.uptime function for other platforms
[develop] Merge forward from 2016.3 to develop (#32494) * fix sorting by latest version when called with an attribute * remove reference to master_alive_check * Fixes saltstack/salt#28262 * Resolve memory leak in authentication * outputter virt_list does not exist anymore * Update proxmox documentation * Fix documentation on boto_asg and boto_elb modules and states * modules.win_timezone: don't list all zones in debug log * Correcty index glusterfs bricks Fixes issue #32311 * Cleaner deprecation process with decorators * Add deprecation decorator scaffold * Capture type error and unhandled exceptions while function calls * Aware of the current and future version of deprecation * Implement initially is_deprecated decorator * Add an alias for the capitalization * Fix capitalization easier way * Remove an extra line * Add successor name to the deprecation decorator. * Granulate logging and error messages. * Implement function swapper * Raise later the caught exception * Clarify exception message * Save function original name * Remove an extra line * Hide an alternative hidden function name in the error message, preserving the error itself * Rename variable as private * Add a method to detect if a function is using its previous version * Message to the log and/or raise an exception accordingly to the status of used function * Log an error along with the exception * Add internal method documentation * Add documentation and usage process for decorator "is_deprecated" * Add documentation and process usage for the decorator "with_deprecated" * Hide private method name * Fix PEP8, re-word the error message * Deprecate basic uptime function * Add initial decorator unit test * Rename old/new functions, mock versions * Move frequent data to the test setup * Add logging on EOL exception * Rename and document high to low version test on is_deprecated * Implement a test on low to high version of is_deprecated decorator * Add a correction to the test description * Remove a dead code * Implement a test for high to low version on is_deprecated, using with_successor param * Correct typso adn mistaeks * Implement high to low version with successor param on is_deprecated * Setup a virtual name for the module * Implement test for with_deprecated should raise an exception if same deprecated function not found * Implement test for with_deprecated an old function is picked up if configured * Correct test description purpose * Implement test with_deprecated when no deprecation is requested * Add logging test to the configured deprecation request * Add logging testing when deprecated version wasn't requested * Implement test EOL for with_deprecated decorator * Correct test explanation * Rename the test * Implement with_deprecated no EOL, deprecated other function name * Implement with_deprecated, deprecated other function name, EOL reached * Add test description for the with_deprecated + with_name + EOL * Fix confusing test names * Add logging test to the is_deprecated decorator when function as not found. * Add more test point to each test, remove empty lines * Bugfix: at certain conditions a wrong alias name is reported to the log * Fix a typo in a comment * Add test for the logging * Disable a pylint: None will _never_ be raised * Fix test for the deprecated "status.uptime" version * Bugfix: Do not yank raised exceptions * Remove unnecessary decorator * Add test for the new uptime * Add test for the new uptime fails when /proc/uptime does not exists * Rename old test case * Skip test for the UTC time, unless freeze time is used. * Fix pylint * Fix documentation * Bugfix: proxy-pass the docstring of the decorated function * Lint fix * Fixes saltstack/salt#28262 for 2015.5 branch * Update master config docs * Improve git_pillar documentation/logging * Add note about different behavior of top file in git_pillar * Make log entry for a missing pillar SLS file more accurate for git_pillar * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info. * Update glusterfs_test to be inline with #32312 * Fix salt-cloud paralell provisioning Closes #31632 * Ignore Raspbian in service.py __virtual__ (#32421) * Ignore Raspbian in service.py __virtual__ This prevents more than one execution module from trying to load as the service virtual module. Refs: #32413 * pack __salt__ before loading provider overrides We can (and should) pack here since we're just packing a reference to the object. __salt__ needs to be available when we're loading our provider overrides * Fix broken __salt__ dict in provider override Using ret.items() here sets ``__salt__`` to its items (tuple containing function name and reference), breaking usage of ``__salt__`` inside overridden functions. * Merge #32293 with test fixes (#32418) * Fix issue #11497 * Remove check for working directory presence in tests * Fix Domainname introspection Default value needs to be extracted from the container itself, because dockerd set Domainname value when network_mode=host. * Add pgjsonb_queue to queue doc index * Pylint fixes * Pass parser options into batch mode Resolves #31738 * Changed the target file in file.symlink test (#32443) * Argument name in docs should match actual arg name (#32445) Fixes #31851 * tests.integration: bypass MacOS TMPDIR, gettempdir (#32447) Updates 0edd532, 8f558a5. When logging in as root over `ssh root@host`, `$TMPDIR` and `tempfile.gettempdir()` are both set to a variation of: ``` /private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/ ``` When logging in as root over `sudo -i`, `$TMPDIR` is unset and `tempfile.gettempdir()` is set to `/tmp`. My guess is that the second case is an unintended or uncorrected omision by Apple as they have introduced the longer, randomized temp path in a recent version of MacOS. * Issue #28706: Fix state user.present behavior. (#32448) - As mentionned in issue #28706, state user.present no longer remove user from groups if the keyword 'groups' with empty value '[]' is not explicitly set, salt will assume current groups are still wanted. * tests.integration: fix 4230c8a * Move the tables of virtual modules to individual documentation pages * Add new doc pages to toctree * Add external ref to windows package manager docs * Improve docstrings * Add documentation on virtual module provider overrides to the module docs * Clarify the scope of the provider param in states. * Add link to provider override docs to all package providers * Add link to provider override docs to all service providers * Add link to provider override docs to all user providers * dd link to provider override docs to all shadow providers * Add link to provider override docs to all group providers * Backport 31164 and 31364 (#32474) * Don't send REQ while another one is waiting for response. The message has to be removed from the queue the only *after* it's already processed to don't confuse send() functionality that expects empty queue means: there's no active sendings. * Fixed zeromq ReqMessageClient destroy * Add link to provider override docs to opkg.py This is a companion to https://github.com/saltstack/salt/pull/32458, but this module was not added until the 2016.3 branch, so the documentation is being updated there for this module. * Add documentation for some master/minion configs (#32454) Refs #32400 Adds docs for: - cli_summary - event_return_queue - event_return_whitelist - event_return_blacklist - file_recv_max_size - fileserver_followsymlinks - fileserver_ignoresymlinks - fileserver_limit_traversal * Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440) * Automatically detect MySQL password column * Fix changing password in MySQL 5.7 * Fix lint test * Fix unit tests (?) They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that. * Additional unit test fix * Only unsub if we have a jid Closes #32479
2016-04-11 23:07:15 +00:00
'''
Use explicit unicode strings + break up salt.utils This PR is part of what will be an ongoing effort to use explicit unicode strings in Salt. Because Python 3 does not suport Python 2's raw unicode string syntax (i.e. `ur'\d+'`), we must use `salt.utils.locales.sdecode()` to ensure that the raw string is unicode. However, because of how `salt/utils/__init__.py` has evolved into the hulking monstrosity it is today, this means importing a large module in places where it is not needed, which could negatively impact performance. For this reason, this PR also breaks out some of the functions from `salt/utils/__init__.py` into new/existing modules under `salt/utils/`. The long term goal will be that the modules within this directory do not depend on importing `salt.utils`. A summary of the changes in this PR is as follows: * Moves the following functions from `salt.utils` to new locations (including a deprecation warning if invoked from `salt.utils`): `to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`, `dequote`, `is_hex`, `is_bin_str`, `rand_string`, `contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`, `which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`, `is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`, `is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`, `is_openbsd`, `is_aix` * Moves the functions already deprecated by @rallytime to the bottom of `salt/utils/__init__.py` for better organization, so we can keep the deprecated ones separate from the ones yet to be deprecated as we continue to break up `salt.utils` * Updates `salt/*.py` and all files under `salt/client/` to use explicit unicode string literals. * Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils import foo` becomes `import salt.utils.foo as foo`). * Renames the `test.rand_str` function to `test.random_hash` to more accurately reflect what it does * Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`) such that it returns a string matching the passed size. Previously this function would get `size` bytes from `os.urandom()`, base64-encode it, and return the result, which would in most cases not be equal to the passed size.
2017-07-25 01:47:15 +00:00
with patch.multiple(salt.utils.platform,
2017-04-10 13:00:57 +00:00
is_linux=MagicMock(return_value=False),
is_sunos=MagicMock(return_value=False),
is_darwin=MagicMock(return_value=False),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=False),
is_netbsd=MagicMock(return_value=False)):
exc_mock = MagicMock(side_effect=CommandExecutionError)
with self.assertRaises(CommandExecutionError):
with patch.dict(status.__salt__, {'cmd.run': exc_mock}):
status.uptime()
2017-12-11 10:55:53 +00:00
def _set_up_test_cpustats_openbsd(self):
'''
Define mock data for status.cpustats on OpenBSD
'''
class MockData(object):
'''
Store mock data
'''
m = MockData()
m.ret = {
'0': {
'User': '0.0%',
'Nice': '0.0%',
'System': '4.5%',
'Interrupt': '0.5%',
'Idle': '95.0%',
}
}
return m
def test_cpustats_openbsd(self):
'''
Test modules.status.cpustats function for OpenBSD
'''
m = self._set_up_test_cpustats_openbsd()
systat = '\n' \
'\n' \
' 1 users Load 0.20 0.07 0.05 salt.localdomain 09:42:42\n' \
'CPU User Nice System Interrupt Idle\n' \
'0 0.0% 0.0% 4.5% 0.5% 95.0%\n'
with patch.multiple(salt.utils.platform,
is_linux=MagicMock(return_value=False),
is_sunos=MagicMock(return_value=False),
is_darwin=MagicMock(return_value=False),
is_freebsd=MagicMock(return_value=False),
is_openbsd=MagicMock(return_value=True),
is_netbsd=MagicMock(return_value=False)), \
patch('salt.utils.path.which', MagicMock(return_value=True)), \
patch.dict(status.__grains__, {'kernel': 'OpenBSD'}), \
patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=systat)}):
ret = status.cpustats()
self.assertDictEqual(ret, m.ret)
def _set_up_test_cpuinfo_bsd(self):
class MockData(object):
'''
Store mock data
'''
m = MockData()
m.ret = {
'hw.model': 'Intel(R) Core(TM) i5-7287U CPU @ 3.30GHz',
'hw.ncpu': '4',
}
return m
def test_cpuinfo_freebsd(self):
m = self._set_up_test_cpuinfo_bsd()
sysctl = 'hw.model:Intel(R) Core(TM) i5-7287U CPU @ 3.30GHz\nhw.ncpu:4'
with patch.dict(status.__grains__, {'kernel': 'FreeBSD'}):
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=sysctl)}):
ret = status.cpuinfo()
self.assertDictEqual(ret, m.ret)
def test_cpuinfo_openbsd(self):
m = self._set_up_test_cpuinfo_bsd()
sysctl = 'hw.model=Intel(R) Core(TM) i5-7287U CPU @ 3.30GHz\nhw.ncpu=4'
for bsd in ['NetBSD', 'OpenBSD']:
with patch.dict(status.__grains__, {'kernel': bsd}):
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=sysctl)}):
ret = status.cpuinfo()
self.assertDictEqual(ret, m.ret)
2017-12-11 11:03:46 +00:00
def _set_up_test_meminfo_openbsd(self):
class MockData(object):
'''
Store mock data
'''
m = MockData()
m.ret = {
'active virtual pages': '355M',
'free list size': '305M',
'page faults': '845',
'pages reclaimed': '1',
'pages paged in': '2',
'pages paged out': '3',
'pages freed': '4',
'pages scanned': '5'
}
return m
def test_meminfo_openbsd(self):
m = self._set_up_test_meminfo_openbsd()
vmstat = ' procs memory page disks traps cpu\n' \
' r s avm fre flt re pi po fr sr cd0 sd0 int sys cs us sy id\n' \
' 2 103 355M 305M 845 1 2 3 4 5 0 1 21 682 86 1 1 98'
with patch.dict(status.__grains__, {'kernel': 'OpenBSD'}):
with patch.dict(status.__salt__, {'cmd.run': MagicMock(return_value=vmstat)}):
ret = status.meminfo()
self.assertDictEqual(ret, m.ret)