Merge remote-tracking branch 'upstream/2015.5' into merge-forward-2015.8

Conflicts:
    salt/renderers/gpg.py
    tests/integration/loader/loader.py
This commit is contained in:
Colton Myers 2015-08-21 16:44:59 -06:00
commit 667dfcee14
11 changed files with 388 additions and 28 deletions

View File

@ -448,12 +448,13 @@ Ink servers in the master configuration.
highstate_run:
local.state.highstate:
- tgt: {{ data['id'] }}
- ret: smtp_return
- ret: smtp
The above will also return the highstate result data using the `smtp_return`
returner. The returner needs to be configured on the minion for this to
work. See :mod:`salt.returners.smtp_return <salt.returners.smtp_return>` documentation for
that.
returner (use virtualname like when using from the command line with `--return`).
The returner needs to be configured on the minion for this to work.
See :mod:`salt.returners.smtp_return <salt.returners.smtp_return>` documentation
for that.
.. _minion-start-reactor:

View File

@ -0,0 +1,287 @@
Changes for v2015.5.4..v2015.5.5
--------------------------------
Extended changelog courtesy of Todd Stansell (https://github.com/tjstansell/salt-changelogs):
*Generated at: 2015-08-20T17:02:37Z*
Statistics:
- Total Merges: **33**
- Total Issue references: **28**
- Total PR references: **41**
Changes:
- **PR** `#26486`_: (*thusoy*) Git: Don't leak https user/pw to log
@ *2015-08-20T16:04:52Z*
- **ISSUE** `#26484`_: (*thusoy*) Git state leaks HTTPS user/pw to log
| refs: `#26486`_
- **ISSUE** `#26482`_: (*thusoy*) Git states doesn't allow user-only auth
| refs: `#26483`_
- **PR** `#26483`_: (*thusoy*) Handle user-only http auth in git module
| refs: `#26486`_
- **PR** `#26476`_: (*jacobhammons*) Minor doc bug fixes
@ *2015-08-19T22:52:35Z*
- **ISSUE** `#26432`_: (*centromere*) Documentation incorrectly references salt-key on the minion
| refs: `#26476`_
- **ISSUE** `#26403`_: (*adelcast*) Grains documentation incorrectly states they are static
| refs: `#26476`_
- **ISSUE** `#26329`_: (*cro*) Add note to eauth docs indicating default PAM service.
| refs: `#26476`_
- **ISSUE** `#26264`_: (*grep4linux*) state trees cannot have 'dots' in the name
| refs: `#26476`_
- **ISSUE** `#26233`_: (*dove-young*) pip install salt, then start master failed on Fedora 22
| refs: `#26476`_
- **PR** `#26443`_: (*cachedout*) Fix connect issue in event init
@ *2015-08-19T22:50:22Z*
- **ISSUE** `#26366`_: (*GreatSnoopy*) The development tree produces hanging, 100%cpu salt-master processes
| refs: `#26443`_
- **ISSUE** `#26301`_: (*waynew*) CPU pegged out running salt-master (after running command)
| refs: `#26443`_
- **ISSUE** `#25998`_: (*driskell*) Event subsystem discarding required events during --batch breaking it for slow running commands
| refs: `#26000`_
- **PR** `#26000`_: (*driskell*) Implement full event caching for subscribed tags
| refs: `#26443`_
- **PR** `#26445`_: (*cachedout*) Raise clean error when no minions targeted in batch mode
@ *2015-08-19T22:50:07Z*
- **ISSUE** `#26343`_: (*jfindlay*) batch error when no minions match target
| refs: `#26445`_
- **PR** `#26483`_: (*thusoy*) Handle user-only http auth in git module
| refs: `#26486`_
@ *2015-08-19T22:47:41Z*
- **ISSUE** `#26482`_: (*thusoy*) Git states doesn't allow user-only auth
| refs: `#26483`_
- **PR** `#26496`_: (*jfindlay*) add dateutil dependency reporting
@ *2015-08-19T22:46:31Z*
- **PR** `#26494`_: (*cachedout*) Remove unecessary debug statements
@ *2015-08-19T20:46:00Z*
- **PR** `#26465`_: (*rallytime*) Back-port `#26457`_ to 2015.5
@ *2015-08-19T16:08:16Z*
- **PR** `#26457`_: (*arthurlogilab*) docstring improvement for network.ping module execution
| refs: `#26465`_
- **PR** `#26434`_: (*s0undt3ch*) Fix missed typo
@ *2015-08-18T18:14:29Z*
- **PR** `#26430`_: (*rallytime*) List public and private ips under the correct label
@ *2015-08-18T16:20:32Z*
- **ISSUE** `#26426`_: (*alxbse*) Private/public IPs are interchanged when listing nova driver cloud nodes
| refs: `#26430`_
- **PR** `#26431`_: (*rallytime*) Back-port `#26417`_ to 2015.5
@ *2015-08-18T15:41:58Z*
- **PR** `#26417`_: (*scottjpack*) Changed t1 -> t2 micro
| refs: `#26431`_
- **PR** `#26378`_: (*stanislavb*) Fix EC2 credentials from IAM roles for s3fs and s3 ext_pillar in 2015.5
@ *2015-08-18T14:01:53Z*
- **PR** `#26420`_: (*terminalmage*) Only use pygit2.errors if it exists (2015.5 branch)
@ *2015-08-18T14:00:01Z*
- **ISSUE** `#26245`_: (*bradthurber*) salt v2015.5.3 gitfs.py using newer pygit2 feature than required minimum
| refs: `#26420`_
- **PR** `#26409`_: (*basepi*) [2015.5] Merge forward from 2014.7 to 2015.5
@ *2015-08-17T23:19:56Z*
- **PR** `#26242`_: (*cro*) Remove dead code
- **PR** `#26216`_: (*cro*) Fix LDAP configuration issue.
- **PR** `#26406`_: (*jfindlay*) fix syntax error in lvm exec module
@ *2015-08-17T21:18:25Z*
- **ISSUE** `#26404`_: (*ssgward*) Syntax error in lvm.vg_absent state causing failure
| refs: `#26406`_
- **PR** `#26405`_: (*TheBigBear*) dependency zip files moved to new site
@ *2015-08-17T21:17:24Z*
- **PR** `#26298`_: (*vr-jack*) Keep $HOME from being interpretted by Master shell
@ *2015-08-17T21:15:11Z*
- **PR** `#26324`_: (*s0undt3ch*) Salt is now pip install'able in windows
@ *2015-08-17T20:41:34Z*
- **PR** `#26371`_: (*bastiaanb*) fix issue `#26161`_: on RedHat family systems touch /var/lock/subsys/$SE…
@ *2015-08-17T20:39:28Z*
- **ISSUE** `#26161`_: (*bastiaanb*) salt initscripts do not set lock file in /var/lock/subsys as required on RedHat family OSes
- **PR** `#26402`_: (*twangboy*) Removed documentation no longer required
@ *2015-08-17T20:35:37Z*
- **ISSUE** `#25801`_: (*themalkolm*) Update docs that salt.states.winrepo requires `roles:salt-master` in grains.
| refs: `#26328`_
- **ISSUE** `#25562`_: (*jefftucker*) winrepo state does not run on masterless minion
| refs: `#26328`_
- **PR** `#26328`_: (*twangboy*) Removed salt-master role requirement
| refs: `#26402`_
- **PR** `#26392`_: (*rallytime*) Back-port `#26376`_ to 2015.5
@ *2015-08-17T19:39:51Z*
- **PR** `#26376`_: (*TheBigBear*) minor edit spelling
| refs: `#26392`_
- **PR** `#26342`_: (*rallytime*) Don't call boto_elb._attributes_present if no attributes were provided
@ *2015-08-17T19:19:08Z*
- **ISSUE** `#16049`_: (*ryan-lane*) boto_elb.present state requires attributes argument
| refs: `#26342`_
- **PR** `#26389`_: (*rallytime*) Back-port `#26160`_ to 2015.5
@ *2015-08-17T19:09:16Z*
- **ISSUE** `#26155`_: (*silenius*) pip availability in states/pip_state
| refs: `#26160`_
- **PR** `#26160`_: (*silenius*) proposed fix for `#26155`_
| refs: `#26389`_
- **PR** `#26300`_: (*jfindlay*) mock pwd function calls in pw_user exec module
@ *2015-08-17T18:56:41Z*
- **ISSUE** `#26266`_: (*o-sleep*) limit pw_user.getent() from returning entire corporate list
| refs: `#26300`_
- **PR** `#26386`_: (*jahamn*) Fixes autosign_timeout usage in check_autosign_dir
@ *2015-08-17T18:34:40Z*
- **ISSUE** `#24334`_: (*afletch*) autosign_timeout not honoured
| refs: `#26386`_
- **PR** `#26328`_: (*twangboy*) Removed salt-master role requirement
| refs: `#26402`_
@ *2015-08-17T18:30:17Z*
- **ISSUE** `#25801`_: (*themalkolm*) Update docs that salt.states.winrepo requires `roles:salt-master` in grains.
| refs: `#26328`_
- **ISSUE** `#25562`_: (*jefftucker*) winrepo state does not run on masterless minion
| refs: `#26328`_
- **PR** `#26362`_: (*garethgreenaway*) Fixes to mount state.
@ *2015-08-17T17:44:55Z*
- **ISSUE** `#26327`_: (*bradthurber*) mount.mounted opts incorrect "forced unmount and mount because options (tcp) changed"
| refs: `#26362`_
- **PR** `#26379`_: (*s0undt3ch*) [2015.5] Backport `#26353`_
@ *2015-08-17T17:19:29Z*
- **PR** `#26353`_: (*sixninetynine*) fixed a typo in setup.py
| refs: `#26379`_
- **PR** `#26277`_: (*rallytime*) Handle exception when user is not found in keystone.user_get
@ *2015-08-14T19:41:59Z*
- **ISSUE** `#26240`_: (*0xf10e*) keystone.user_get raises exception when user is not found
| refs: `#26277`_
- **PR** `#26326`_: (*rallytime*) Make ec2.create_snapshot return less unweildly and more relevant
@ *2015-08-14T19:40:47Z*
- **ISSUE** `#24484`_: (*codehotter*) clouds/ec2.py: create_snapshot throws exception
| refs: `#26326`_
- **PR** `#26306`_: (*rallytime*) Move VM creation details dict to log.trace
@ *2015-08-14T17:39:52Z*
- **ISSUE** `#16179`_: (*UtahDave*) Salt Cloud -l debug includes the entire bootstrap script twice in its output
| refs: `#26306`_
.. _`#16049`: https://github.com/saltstack/salt/issues/16049
.. _`#16179`: https://github.com/saltstack/salt/issues/16179
.. _`#24334`: https://github.com/saltstack/salt/issues/24334
.. _`#24484`: https://github.com/saltstack/salt/issues/24484
.. _`#25562`: https://github.com/saltstack/salt/issues/25562
.. _`#25801`: https://github.com/saltstack/salt/issues/25801
.. _`#25998`: https://github.com/saltstack/salt/issues/25998
.. _`#26000`: https://github.com/saltstack/salt/pull/26000
.. _`#26155`: https://github.com/saltstack/salt/issues/26155
.. _`#26160`: https://github.com/saltstack/salt/pull/26160
.. _`#26161`: https://github.com/saltstack/salt/issues/26161
.. _`#26216`: https://github.com/saltstack/salt/pull/26216
.. _`#26233`: https://github.com/saltstack/salt/issues/26233
.. _`#26240`: https://github.com/saltstack/salt/issues/26240
.. _`#26242`: https://github.com/saltstack/salt/pull/26242
.. _`#26245`: https://github.com/saltstack/salt/issues/26245
.. _`#26264`: https://github.com/saltstack/salt/issues/26264
.. _`#26266`: https://github.com/saltstack/salt/issues/26266
.. _`#26277`: https://github.com/saltstack/salt/pull/26277
.. _`#26298`: https://github.com/saltstack/salt/pull/26298
.. _`#26300`: https://github.com/saltstack/salt/pull/26300
.. _`#26301`: https://github.com/saltstack/salt/issues/26301
.. _`#26306`: https://github.com/saltstack/salt/pull/26306
.. _`#26324`: https://github.com/saltstack/salt/pull/26324
.. _`#26326`: https://github.com/saltstack/salt/pull/26326
.. _`#26327`: https://github.com/saltstack/salt/issues/26327
.. _`#26328`: https://github.com/saltstack/salt/pull/26328
.. _`#26329`: https://github.com/saltstack/salt/issues/26329
.. _`#26342`: https://github.com/saltstack/salt/pull/26342
.. _`#26343`: https://github.com/saltstack/salt/issues/26343
.. _`#26353`: https://github.com/saltstack/salt/pull/26353
.. _`#26362`: https://github.com/saltstack/salt/pull/26362
.. _`#26366`: https://github.com/saltstack/salt/issues/26366
.. _`#26371`: https://github.com/saltstack/salt/pull/26371
.. _`#26376`: https://github.com/saltstack/salt/pull/26376
.. _`#26378`: https://github.com/saltstack/salt/pull/26378
.. _`#26379`: https://github.com/saltstack/salt/pull/26379
.. _`#26386`: https://github.com/saltstack/salt/pull/26386
.. _`#26389`: https://github.com/saltstack/salt/pull/26389
.. _`#26392`: https://github.com/saltstack/salt/pull/26392
.. _`#26402`: https://github.com/saltstack/salt/pull/26402
.. _`#26403`: https://github.com/saltstack/salt/issues/26403
.. _`#26404`: https://github.com/saltstack/salt/issues/26404
.. _`#26405`: https://github.com/saltstack/salt/pull/26405
.. _`#26406`: https://github.com/saltstack/salt/pull/26406
.. _`#26409`: https://github.com/saltstack/salt/pull/26409
.. _`#26417`: https://github.com/saltstack/salt/pull/26417
.. _`#26420`: https://github.com/saltstack/salt/pull/26420
.. _`#26426`: https://github.com/saltstack/salt/issues/26426
.. _`#26430`: https://github.com/saltstack/salt/pull/26430
.. _`#26431`: https://github.com/saltstack/salt/pull/26431
.. _`#26432`: https://github.com/saltstack/salt/issues/26432
.. _`#26434`: https://github.com/saltstack/salt/pull/26434
.. _`#26443`: https://github.com/saltstack/salt/pull/26443
.. _`#26445`: https://github.com/saltstack/salt/pull/26445
.. _`#26457`: https://github.com/saltstack/salt/pull/26457
.. _`#26465`: https://github.com/saltstack/salt/pull/26465
.. _`#26476`: https://github.com/saltstack/salt/pull/26476
.. _`#26482`: https://github.com/saltstack/salt/issues/26482
.. _`#26483`: https://github.com/saltstack/salt/pull/26483
.. _`#26484`: https://github.com/saltstack/salt/issues/26484
.. _`#26486`: https://github.com/saltstack/salt/pull/26486
.. _`#26494`: https://github.com/saltstack/salt/pull/26494
.. _`#26496`: https://github.com/saltstack/salt/pull/26496
.. _`bp-26160`: https://github.com/saltstack/salt/pull/26160
.. _`bp-26376`: https://github.com/saltstack/salt/pull/26376
.. _`bp-26417`: https://github.com/saltstack/salt/pull/26417
.. _`bp-26457`: https://github.com/saltstack/salt/pull/26457
.. _`fix-16049`: https://github.com/saltstack/salt/issues/16049
.. _`fix-16179`: https://github.com/saltstack/salt/issues/16179
.. _`fix-26240`: https://github.com/saltstack/salt/issues/26240
.. _`fix-26426`: https://github.com/saltstack/salt/issues/26426

View File

@ -52,7 +52,7 @@ Follow :issue:`11801` for any changes to this behavior.
Dealing with various username forms
===================================
Salt does not understand the various forms that Windows usernames can come in,
e.g. username, mydomain\username, username@mydomain.tld can all refer to the
e.g. username, mydomain\\username, username@mydomain.tld can all refer to the
same user. In fact, Salt generally only considers the raw username value, i.e.
the username without the domain or host information.
@ -86,4 +86,4 @@ platforms. In general, avoid symlink loops on either platform.
Modifying security properties (ACLs) on files
=============================================
There is no support in Salt for modifying ACLs, and therefore no support for
changing file permissions, besides modifying the owner/user.
changing file permissions, besides modifying the owner/user.

View File

@ -230,7 +230,10 @@ def file_hash(load, fnd):
cache_dir = os.path.dirname(cache_path)
# make cache directory if it doesn't exist
if not os.path.exists(cache_dir):
os.makedirs(cache_dir)
try:
os.makedirs(cache_dir)
except OSError:
pass
# save the cache object "hash:mtime"
cache_object = '{0}:{1}'.format(ret['hsum'], os.path.getmtime(path))
with salt.utils.flopen(cache_path, 'w') as fp_:

View File

@ -15,6 +15,7 @@ import distutils.version # pylint: disable=import-error,no-name-in-module
# Import salt libs
import salt.utils
import salt.modules.cmdmod
from salt.exceptions import CommandExecutionError
@ -32,7 +33,7 @@ def __virtual__():
'''
try:
if salt.utils.which('npm') is not None:
_check_valid_version(__salt__)
_check_valid_version()
return True
else:
return (False, 'npm execution module could not be loaded '
@ -41,14 +42,14 @@ def __virtual__():
return (False, str(exc))
def _check_valid_version(salt):
def _check_valid_version():
'''
Check the version of npm to ensure this module will work. Currently
npm must be at least version 1.2.
'''
# pylint: disable=no-member
npm_version = distutils.version.LooseVersion(
salt['cmd.run']('npm --version', python_shell=True))
salt.modules.cmdmod.run('npm --version', python_shell=True))
valid_version = distutils.version.LooseVersion('1.2')
# pylint: enable=no-member
if npm_version < valid_version:

View File

@ -16,6 +16,7 @@ import sys
# Import Salt libs
import salt.utils
import salt.modules.cmdmod
import salt.utils.decorators as decorators
log = logging.getLogger(__name__)
@ -90,7 +91,7 @@ def __virtual__():
else:
cmd = 'ls /sys/module/zfs'
if cmd and __salt__['cmd.retcode'](
if cmd and salt.modules.cmdmod.retcode(
cmd, output_loglevel='quiet', ignore_retcode=True
) == 0:
# Build dynamic functions and allow loading module

View File

@ -454,22 +454,19 @@ def _elb_present(
raise SaltInvocationError('Listeners must have at minimum port,'
' instance_port and protocol values in'
' the provided list.')
if 'elb_port' not in listener:
raise SaltInvocationError('elb_port is a required value for'
' listeners.')
if 'instance_port' not in listener:
raise SaltInvocationError('instance_port is a required value for'
' listeners.')
if 'elb_protocol' not in listener:
raise SaltInvocationError('elb_protocol is a required value for'
' listeners.')
for config in ('elb_port', 'instance_port', 'elb_protocol'):
if not listener.get(config):
raise SaltInvocationError(
'{0} is a required value for listeners.'.format(config)
)
listener['elb_protocol'] = listener['elb_protocol'].upper()
if listener['elb_protocol'] == 'HTTPS' and 'certificate' not in listener:
raise SaltInvocationError('certificate is a required value for'
' listeners if HTTPS is set for'
' elb_protocol.')
# We define all listeners as complex listeners.
if 'instance_protocol' not in listener:
if not listener.get('instance_protocol'):
listener['instance_protocol'] = listener['elb_protocol'].upper()
else:
listener['instance_protocol'] = listener['instance_protocol'].upper()

View File

@ -2521,6 +2521,12 @@ def replace(name,
# <...snip...>
- pattern: |
CentOS \(2.6.32[^\n]+\n\s+root[^\n]+\n\)+
.. note::
When using YAML multiline string syntax in ``pattern:``, make sure to
also use that syntax in the ``repl:`` part, or you might loose line
feeds.
'''
name = os.path.expanduser(name)

View File

@ -16,6 +16,7 @@ import hashlib
import imp
import json
import logging
import numbers
import os
import pprint
import random
@ -2294,9 +2295,21 @@ def compare_versions(ver1='', oper='==', ver2='', cmp_func=None):
if cmp_result is None:
return False
# Check if integer/long
if not isinstance(cmp_result, numbers.Integral):
log.error('The version comparison function did not return an '
'integer/long.')
return False
if oper == '!=':
return cmp_result not in cmp_map['==']
else:
# Gracefully handle cmp_result not in (-1, 0, 1).
if cmp_result < -1:
cmp_result = -1
elif cmp_result > 1:
cmp_result = 1
return cmp_result in cmp_map[oper]

View File

@ -51,6 +51,7 @@ __AccessKeyId__ = ''
__SecretAccessKey__ = ''
__Token__ = ''
__Expiration__ = ''
__Location__ = ''
def creds(provider):
@ -140,7 +141,7 @@ def sig2(method, endpoint, params, provider, aws_api_version):
def sig4(method, endpoint, params, prov_dict,
aws_api_version=DEFAULT_AWS_API_VERSION, location=DEFAULT_LOCATION,
aws_api_version=DEFAULT_AWS_API_VERSION, location=None,
product='ec2', uri='/', requesturl=None, data=''):
'''
Sign a query against AWS services using Signature Version 4 Signing
@ -155,6 +156,11 @@ def sig4(method, endpoint, params, prov_dict,
# Retrieve access credentials from meta-data, or use provided
access_key_id, secret_access_key, token = creds(prov_dict)
if location is None:
location = get_region_from_metadata()
if location is None:
location = DEFAULT_LOCATION
params_with_headers = params.copy()
if product != 's3':
params_with_headers['Version'] = aws_api_version
@ -442,15 +448,56 @@ def query(params=None, setname=None, requesturl=None, location=None,
return ret
def get_region_from_metadata():
'''
Try to get region from instance identity document and cache it
.. versionadded:: 2015.5.6
'''
global __Location__
if __Location__ == 'do-not-get-from-metadata':
LOG.debug('Previously failed to get AWS region from metadata. Not trying again.')
return None
# Cached region
if __Location__ != '':
return __Location__
try:
# Connections to instance meta-data must never be proxied
result = requests.get(
"http://169.254.169.254/latest/dynamic/instance-identity/document",
proxies={'http': ''},
)
except requests.exceptions.RequestException:
LOG.warning('Failed to get AWS region from instance metadata.', exc_info=True)
# Do not try again
__Location__ = 'do-not-get-from-metadata'
return None
try:
region = result.json()['region']
__Location__ = region
return __Location__
except (ValueError, KeyError):
LOG.warning('Failed to decode JSON from instance metadata.')
return None
return None
def get_location(opts, provider=None):
'''
Return the region to use, in this order:
opts['location']
provider['location']
get_region_from_metadata()
DEFAULT_LOCATION
'''
return opts.get(
'location', provider.get(
'location', DEFAULT_LOCATION
)
)
ret = opts.get('location', provider.get('location'))
if ret is None:
ret = get_region_from_metadata()
if ret is None:
ret = DEFAULT_LOCATION
return ret

View File

@ -29,7 +29,7 @@ from salt.ext.six.moves import range
from salt.config import minion_config
# pylint: enable=no-name-in-module,redefined-builtin
from salt.loader import LazyLoader, _module_dirs
from salt.loader import LazyLoader, _module_dirs, grains
class LazyLoaderVirtualEnabledTest(TestCase):
@ -39,6 +39,7 @@ class LazyLoaderVirtualEnabledTest(TestCase):
def setUp(self):
self.opts = minion_config(None)
self.opts['disable_modules'] = ['pillar']
self.opts['grains'] = grains(self.opts)
self.loader = LazyLoader(_module_dirs(self.opts, 'modules', 'module'),
self.opts,
@ -126,6 +127,7 @@ class LazyLoaderVirtualDisabledTest(TestCase):
'''
def setUp(self):
self.opts = _config = minion_config(None)
self.opts['grains'] = grains(self.opts)
self.loader = LazyLoader(_module_dirs(self.opts, 'modules', 'module'),
self.opts,
tag='module',
@ -186,6 +188,7 @@ class LazyLoaderReloadingTest(TestCase):
def setUp(self):
self.opts = _config = minion_config(None)
self.opts['grains'] = grains(self.opts)
self.tmp_dir = tempfile.mkdtemp(dir=tests.integration.TMP)
self.count = 0
@ -310,6 +313,7 @@ class LazyLoaderSubmodReloadingTest(TestCase):
def setUp(self):
self.opts = _config = minion_config(None)
self.opts['grains'] = grains(self.opts)
self.tmp_dir = tempfile.mkdtemp(dir=tests.integration.TMP)
os.makedirs(self.module_dir)