Merge pull request #39968 from rallytime/merge-develop

[develop] Merge forward from 2016.11 to develop

doc/faq.rst

@@ -168,6 +168,7 @@ information.
 
 Module ``X`` isn't available, even though the shell command it uses is installed. Why?
 --------------------------------------------------------------------------------------
+
 This is most likely a PATH issue. Did you custom-compile the software which the
 module requires? RHEL/CentOS/etc. in particular override the root user's path
 in ``/etc/init.d/functions``, setting it to ``/sbin:/usr/sbin:/bin:/usr/bin``,
@@ -246,86 +247,116 @@ specifying the pillar variable is the same one used for :py:func:`pillar.get
     <salt.states.file.managed>` state is only supported in Salt 2015.8.4 and
     newer.
 
-What is the best way to restart a Salt daemon using Salt?
+What is the best way to restart a Salt Minion daemon using Salt after upgrade?
----------------------------------------------------------
+------------------------------------------------------------------------------
 
-Updating the salt-minion package requires a restart of the salt-minion service.
+Updating the ``salt-minion`` package requires a restart of the ``salt-minion``
-But restarting the service while in the middle of a state run interrupts the
+service. But restarting the service while in the middle of a state run
-process of the minion running states and sending results back to the master.
+interrupts the process of the Minion running states and sending results back to
-It's a tricky problem to solve, and we're working on it, but in the meantime
+the Master. A common way to workaround that is to schedule restarting of the
-one way of handling this (on Linux and UNIX-based operating systems) is to use
+Minion service using :ref:`masterless mode <masterless-quickstart>` after all
-**at** (a job scheduler which predates cron) to schedule a restart of the
+other states have been applied. This allows to keep Minion to Master connection
-service. **at** is not installed by default on most distros, and requires a
+alive for the Minion to report the final results to the Master, while the
-service to be running (usually called **atd**) in order to schedule jobs.
+service is restarting in the background.
-Here's an example of how to upgrade the salt-minion package at the end of a
-Salt run, and schedule a service restart for one minute after the package
-update completes.
 
-Linux/Unix
+Upgrade without automatic restart
-**********
+*********************************
+
+Doing the Minion upgrade seems to be a simplest state in your SLS file at
+first. But the operating systems such as Debian GNU/Linux, Ununtu and their
+derivatives start the service after the package installation by default.
+To prevent this, we need to create policy layer which will prevent the Minion
+service to restart right after the upgrade:
 
 .. code-block:: yaml
 
-    salt-minion:
+    {%- if grains['os_family'] == 'Debian' %}
+
+    Disable starting services:
+      file.managed:
+        - name: /usr/sbin/policy-rc.d
+        - user: root
+        - group: root
+        - mode: 0755
+        - contents:
+          - '#!/bin/sh'
+          - exit 101
+        # do not touch if already exists
+        - replace: False
+        - prereq:
+          - pkg: Upgrade Salt Minion
+
+    {%- endif %}
+
+    Upgrade Salt Minion:
       pkg.installed:
         - name: salt-minion
-        - version: 2014.1.7-3.el6
+        - version: 2016.11.3{% if grains['os_family'] == 'Debian' %}+ds-1{% endif %}
         - order: last
-      service.running:
+
+    Enable Salt Minion:
+      service.enabled:
         - name: salt-minion
         - require:
-          - pkg: salt-minion
+          - pkg: Upgrade Salt Minion
-      cmd.run:
+
-        - name: echo service salt-minion restart | at now + 1 minute
+    {%- if grains['os_family'] == 'Debian' %}
+
+    Enable starting services:
+      file.absent:
+        - name: /usr/sbin/policy-rc.d
         - onchanges:
-          - pkg: salt-minion
+          - pkg: Upgrade Salt Minion
 
-To ensure that **at** is installed and **atd** is running, the following states
+    {%- endif %}
-can be used (be sure to double-check the package name and service name for the
+
-distro the minion is running, in case they differ from the example below.
+Restart using states
+********************
+
+Now we can apply the workaround to restart the Minion in reliable way.
+The following example works on both UNIX-like and Windows operating systems:
 
 .. code-block:: yaml
 
-    at:
+    Restart Salt Minion:
-      pkg.installed:
-        - name: at
-      service.running:
-        - name: atd
-        - enable: True
-
-An alternative to using the :program:`atd` daemon is to fork and disown the
-process.
-
-.. code-block:: yaml
-
-    restart_minion:
       cmd.run:
-        - name: |
+    {%- if grains['kernel'] == 'Windows' %}
+        - name: 'C:\salt\salt-call.bat --local service.restart salt-minion'
+    {%- else %}
+        - name: 'salt-call --local service.restart salt-minion'
+    {%- endif %}
+        - bg: True
+        - onchanges:
+          - pkg: Upgrade Salt Minion
+
+However, it requires more advanced tricks to upgrade from legacy version of
+Salt (before ``2016.3.0``), where executing commands in the background is not
+supported:
+
+.. code-block:: yaml
+
+    Restart Salt Minion:
+      cmd.run:
+    {%- if grains['kernel'] == 'Windows' %}
+        - name: 'start powershell "Restart-Service -Name salt-minion"'
+    {%- else %}
+        # fork and disown the process
+        - name: |-
             exec 0>&- # close stdin
             exec 1>&- # close stdout
             exec 2>&- # close stderr
-            nohup /bin/sh -c 'sleep 10 && salt-call --local service.restart salt-minion' &
+            nohup salt-call --local service.restart salt-minion &
-        - python_shell: True
+    {%- endif %}
-        - order: last
 
-Windows
+Restart using remote executions
-*******
+*******************************
 
-For Windows machines, restarting the minion can be accomplished using the
+Restart the Minion from the command line:
-following state:
-
-.. code-block:: yaml
-
-    schedule-start:
-      cmd.run:
-        - name: 'start powershell "Restart-Service -Name salt-minion"'
-        - order: last
-
-or running immediately from the command line:
 
 .. code-block:: bash
 
-    salt -G kernel:Windows cmd.run 'start powershell "Restart-Service -Name salt-minion"'
+    salt -G kernel:Windows cmd.run_bg 'C:\salt\salt-call.bat --local service.restart salt-minion'
+    salt -C 'not G@kernel:Windows' cmd.run_bg 'salt-call --local service.restart salt-minion'
 
 Salting the Salt Master
 -----------------------

doc/ref/configuration/minion.rst

@@ -40,6 +40,33 @@ Default: ``salt``
 
     master: salt
 
+master:port Syntax
+~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 2015.8.0
+
+The ``master`` config option can also be set to use the master's IP in
+conjunction with a port number by default.
+
+.. code-block:: yaml
+
+    master: localhost:1234
+
+For IPv6 formatting with a port, remember to add brackets around the IP address
+before adding the port and enclose the line in single quotes to make it a string:
+
+.. code-block:: yaml
+
+    master: '[2001:db8:85a3:8d3:1319:8a2e:370:7348]:1234'
+
+.. note::
+
+    If a port is specified in the ``master`` as well as :conf_minion:`master_port`,
+    the ``master_port`` setting will be overridden by the ``master`` configuration.
+
+List of Masters Syntax
+~~~~~~~~~~~~~~~~~~~~~~
+
 The option can also be set to a list of masters, enabling
 :ref:`multi-master <tutorial-multi-master>` mode.
 
@@ -90,6 +117,22 @@ will try to automatically detect IPv6 connectivity to master.
 
     ipv6: True
 
+.. conf_minion:: master_uri_format
+
+``master_uri_format``
+---------------------
+
+.. versionadded:: 2015.8.0
+
+Specify the format in which the master address will be evaluated. Valid options
+are ``default`` or ``ip_only``. If ``ip_only`` is specified, then the master
+address will not be split into IP and PORT, so be sure that only an IP (or domain
+name) is set in the :conf_minion:`master` configuration setting.
+
+.. code-block:: yaml
+
+    master_uri_format: ip_only
+
 .. conf_minion:: master_type
 
 ``master_type``
@@ -2244,6 +2287,17 @@ file.
     files are prefixed with an underscore. A common example of this is the
     ``_schedule.conf`` file.
 
+.. note::
+
+    The configuration system supports adding the special token ``{id}`` to this
+    option.  At startup ``{id}`` will be replaced by the minion's ID, and the
+    default_include directory will be set here.  For example, if the minion's
+    ID is 'webserver' and ``default_include`` is set to ``minion.d/{id}/*.conf``
+    then the default_include directive will be set to ``minion.d/webserver/*.conf``.
+    This is for situations when there are multiple minions or proxy minions
+    running on a single machine that need different configurations, specifically for
+    their schedulers.
+
 
 ``include``
 -----------

doc/ref/states/index.rst

@@ -147,16 +147,17 @@ watched file updated.
 The Top File
 ````````````
 
-The top file controls the mapping between minions and the states which should be
+The top file controls the mapping between minions and the states which should
-applied to them.
+be applied to them.
 
-The top file specifies which minions should have which SLS files applied and which
+The top file specifies which minions should have which SLS files applied and
-environments they should draw those SLS files from.
+which environments they should draw those SLS files from.
 
 The top file works by specifying environments on the top-level.
 
-Each environment contains globs to match minions. Finally, each glob contains a list of
+Each environment contains :ref:`target expressions <targeting>` to match
-lists of Salt states to apply to matching minions:
+minions. Finally, each target expression contains a list of Salt states to
+apply to matching minions:
 
 .. code-block:: yaml
 
@@ -172,12 +173,33 @@ lists of Salt states to apply to matching minions:
 This above example uses the base environment which is built into the default
 Salt setup.
 
-The base environment has two globs. First, the '*' glob contains a list of
+The base environment has target expressions. The first one matches all minions,
-SLS files to apply to all minions.
+and the SLS files below it apply to all minions.
 
-The second glob contains a regular expression that will match all minions with
+The second expression is a regular expression that will match all minions
-an ID matching saltmaster.* and specifies that for those minions, the salt.master
+with an ID matching ``saltmaster.*`` and specifies that for those minions, the
-state should be applied.
+salt.master state should be applied. To
+
+.. important::
+    Since version 2014.7.0, the default matcher (when one is not explicitly
+    defined as in the second expression in the above example) is the
+    :ref:`compound <targeting-compound>` matcher. Since this matcher parses
+    individual words in the expression, minion IDs containing spaces will not
+    match properly using this matcher. Therefore, if your target expression is
+    designed to match a minion ID containing spaces, it will be necessary to
+    specify a different match type (such as ``glob``). For example:
+
+    .. code-block:: yaml
+
+        base:
+          'test minion':
+            - match: glob
+            - foo
+            - bar
+            - baz
+
+A full table of match types available in the top file can be found :ref:`here
+<top-file-match-types>`.
 
 .. _reloading-modules:
 

doc/ref/states/top.rst

@@ -158,8 +158,8 @@ Our top file references the environments:
         - db
 
 As seen above, the top file now declares the three environments and for each,
-targets are defined to map globs of minion IDs to state files. For example,
+target expressions are defined to map minions to state files. For example, all
-all minions which have an ID beginning with the string ``webserver`` will have the
+minions which have an ID beginning with the string ``webserver`` will have the
 webserver state from the requested environment assigned to it.
 
 In this manner, a proposed change to a state could first be made in a state
@@ -219,12 +219,51 @@ also available:
 Advanced Minion Targeting
 =========================
 
-In addition to globs, minions can be specified in top files a few other
+In the examples above, notice that all of the target expressions are globs. The
-ways. Some common ones are :ref:`compound matches <targeting-compound>`
+default match type in top files (since version 2014.7.0) is actually the
-and :ref:`node groups <targeting-nodegroups>`.
+:ref:`compound matcher <targeting-compound>`, not the glob matcher as in the
+CLI.
 
-Below is a slightly more complex top file example, showing the different types
+A single glob, when passed through the compound matcher, acts the same way as
-of matches you can perform:
+matching by glob, so in most cases the two are indistinguishable. However,
+there is an edge case in which a minion ID contains whitespace. While it is not
+recommended to include spaces in a minion ID, Salt will not stop you from doing
+so. However, since compound expressions are parsed word-by-word, if a minion ID
+contains spaces it will fail to match. In this edge case, it will be necessary
+to explicitly use the ``glob`` matcher:
+
+.. code-block:: yaml
+
+    base:
+      'minion 1':
+        - match: glob
+        - foo
+
+.. _top-file-match-types:
+
+The available match types which can be set for a target expression in the top
+file are:
+
+============ ================================================================================================================
+Match Type   Description
+============ ================================================================================================================
+glob         Full minion ID or glob expression to match multiple minions (e.g. ``minion123`` or ``minion*``)
+pcre         Perl-compatible regular expression (PCRE) matching a minion ID (e.g.  ``web[0-3].domain.com``)
+grain        Match a :ref:`grain <grain>`, optionally using globbing (e.g. ``kernel:Linux`` or ``kernel:*BSD``)
+grain_pcre   Match a :ref:`grain <grain>` using PCRE (e.g. ``kernel:(Free|Open)BSD``)
+list         Comma-separated list of minions (e.g. ``minion1,minion2,minion3``)
+pillar       :ref:`Pillar <pillar>` match, optionally using globbing (e.g. ``role:webserver`` or ``role:web*``)
+pillar_pcre  :ref:`Pillar <pillar>` match using PCRE (e.g. ``role:web(server|proxy)``
+pillar_exact :ref:`Pillar <pillar>` match with no globbing or PCRE (e.g. ``role:webserver``)
+ipcidr       Subnet or IP address (e.g. ``172.17.0.0/16`` or ``10.2.9.80``)
+data         Match values kept in the minion's datastore (created using the :mod:`data <salt.modules.data>` execution module)
+range        :ref:`Range <targeting-range>` cluster
+compound     Complex expression combining multiple match types (see :ref:`here <targeting-compound>`)
+nodegroup    Pre-defined compound expressions in the master config file (see :ref:`here <targeting-nodegroups>`)
+============ ================================================================================================================
+
+Below is a slightly more complex top file example, showing some of the above
+match types:
 
 .. code-block:: yaml
 
@@ -232,6 +271,13 @@ of matches you can perform:
     # environment in the ``file_roots`` configuration value.
 
     base:
+        # All minions which begin with the strings 'nag1' or any minion with
+        # a grain set called 'role' with the value of 'monitoring' will have
+        # the 'server.sls' state file applied from the 'nagios/' directory.
+
+        'nag1* or G@role:monitoring':
+            - nagios.server
+
         # All minions get the following three state files applied
 
         '*':
@@ -296,14 +342,6 @@ of matches you can perform:
             - match: pillar
             - xyz
 
-        # All minions which begin with the strings 'nag1' or any minion with
-        # a grain set called 'role' with the value of 'monitoring' will have
-        # the 'server.sls' state file applied from the 'nagios/' directory.
-
-        'nag1* or G@role:monitoring':
-            - match: compound
-            - nagios.server
-
 How Top Files Are Compiled
 ==========================
 

doc/topics/beacons/index.rst

@@ -221,7 +221,7 @@ Add the following to ``/srv/reactor/revert.sls``:
 
 .. note::
 
-    The expression ``{{ data['data']['id] }}`` :ref:`is correct
+    The expression ``{{ data['data']['id'] }}`` :ref:`is correct
     <beacons-and-reactors>` as it matches the event structure :ref:`shown above
     <beacon-event-bus>`.
 

doc/topics/targeting/ipcidr.rst

@@ -10,7 +10,7 @@ notation).
 .. code-block:: bash
 
     salt -S 192.168.40.20 test.ping
-    salt -S 10.0.0.0/24 test.ping
+    salt -S 2001:db8::/64 test.ping
 
 Ipcidr matching can also be used in compound matches
 
@@ -27,7 +27,3 @@ It is also possible to use in both pillar and state-matching
      - internal
 
 .. _CIDR: http://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing
-
-.. note::
-
-    Only IPv4 matching is supported at this time.

doc/topics/targeting/range.rst

@@ -1,4 +1,4 @@
-.. _targeting_range:
+.. _targeting-range:
 
 ==========
 SECO Range

salt/auth/yubico.py

@@ -78,15 +78,16 @@ def auth(username, password):
     client = Yubico(_cred['id'], _cred['key'])
 
     try:
-        if client.verify(password):
+        return client.verify(password)
-            return True
-        else:
-            return False
     except yubico_exceptions.StatusCodeError as e:
         log.info('Unable to verify YubiKey `{0}`'.format(e))
         return False
 
 
+def groups(username, *args, **kwargs):
+    return False
+
+
 if __name__ == '__main__':
     __opts__ = {'yubico_users': {'damian': {'id': '12345', 'key': 'ABC123'}}}
 

salt/beacons/status.py

@@ -138,6 +138,7 @@ def beacon(config):
         # To support the old dictionary config format
         config = [config]
 
+    ret = {}
     for entry in config:
         for func in entry:
             ret[func] = {}

salt/cloud/clouds/ec2.py

@@ -2816,11 +2816,7 @@ def queue_instances(instances):
     '''
     for instance_id in instances:
         node = _get_node(instance_id=instance_id)
-        for name in node:
+        __utils__['cloud.cache_node'](node, __active_provider_name__, __opts__)
-            if instance_id == node[name]['instanceId']:
-                __utils__['cloud.cache_node'](node[name],
-                                            __active_provider_name__,
-                                            __opts__)
 
 
 def create_attach_volumes(name, kwargs, call=None, wait_to_finish=True):
@@ -3371,10 +3367,7 @@ def show_instance(name=None, instance_id=None, call=None, kwargs=None):
         )
 
     node = _get_node(name=name, instance_id=instance_id)
-    for name in node:
+    __utils__['cloud.cache_node'](node, __active_provider_name__, __opts__)
-        __utils__['cloud.cache_node'](node[name],
-                                    __active_provider_name__,
-                                    __opts__)
     return node
 
 

salt/cloud/clouds/msazure.py

@@ -1595,7 +1595,7 @@ def cleanup_unattached_disks(kwargs=None, conn=None, call=None):
     for disk in disks:
         if disks[disk]['attached_to'] is None:
             del_kwargs = {
-                'name': disks[disk]['name'][0],
+                'name': disks[disk]['name'],
                 'delete_vhd': kwargs.get('delete_vhd', False)
             }
             log.info('Deleting disk {name}, deleting VHD: {delete_vhd}'.format(**del_kwargs))

salt/config/__init__.py

@@ -2051,6 +2051,7 @@ def minion_config(path,
     overrides = load_config(path, env_var, DEFAULT_MINION_OPTS['conf_file'])
     default_include = overrides.get('default_include',
                                     defaults['default_include'])
+
     include = overrides.get('include', [])
 
     overrides.update(include_config(default_include, path, verbose=False,
@@ -3233,6 +3234,9 @@ def apply_minion_config(overrides=None,
             newdirectory = os.path.join(opts[directory], opts['id'])
             opts[directory] = newdirectory
 
+    if 'default_include' in overrides and '{id}' in overrides['default_include']:
+        opts['default_include'] = overrides['default_include'].replace('{id}', opts['id'])
+
     # pidfile can be in the list of append_minionid_config_dirs, but pidfile
     # is the actual path with the filename, not a directory.
     if 'pidfile' in opts.get('append_minionid_config_dirs', []):

salt/minion.py

@@ -88,6 +88,7 @@ import salt.utils.jid
 import salt.pillar
 import salt.utils.args
 import salt.utils.event
+import salt.utils.network
 import salt.utils.minion
 import salt.utils.minions
 import salt.utils.schedule
@@ -179,9 +180,11 @@ def resolve_dns(opts, fallback=True, connect=True):
             if master == '':
                 master = unknown_str
             if opts.get('__role') == 'syndic':
-                err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. Set \'syndic_master\' value in minion config.'.format(master)
+                err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. ' \
+                      'Set \'syndic_master\' value in minion config.'.format(master)
             else:
-                err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. Set \'master\' value in minion config.'.format(master)
+                err = 'Master address: \'{0}\' could not be resolved. Invalid or unresolveable address. ' \
+                      'Set \'master\' value in minion config.'.format(master)
             log.error(err)
             raise SaltSystemExit(code=42, msg=err)
     else:
@@ -199,7 +202,10 @@ def resolve_dns(opts, fallback=True, connect=True):
 
 def prep_ip_port(opts):
     ret = {}
-    if opts['master_uri_format'] == 'ip_only':
+    # Use given master IP if "ip_only" is set or if master_ip is an ipv6 address without
+    # a port specified. The is_ipv6 check returns False if brackets are used in the IP
+    # definition such as master: '[::1]:1234'.
+    if opts['master_uri_format'] == 'ip_only' or salt.utils.network.is_ipv6(opts['master']):
         ret['master'] = opts['master']
     else:
         ip_port = opts['master'].rsplit(":", 1)
@@ -209,9 +215,13 @@ def prep_ip_port(opts):
         else:
             # e.g. master: localhost:1234
             # e.g. master: 127.0.0.1:1234
-            # e.g. master: ::1:1234
+            # e.g. master: [::1]:1234
-            ret['master'] = ip_port[0]
+            # Strip off brackets for ipv6 support
-            ret['master_port'] = ip_port[1]
+            ret['master'] = ip_port[0].strip('[]')
+
+            # Cast port back to an int! Otherwise a TypeError is thrown
+            # on some of the socket calls elsewhere in the minion and utils code.
+            ret['master_port'] = int(ip_port[1])
     return ret
 
 

salt/modules/boto_vpc.py

@@ -2599,7 +2599,13 @@ def _maybe_set_name_tag(name, obj):
 
 def _maybe_set_tags(tags, obj):
     if tags:
-        obj.add_tags(tags)
+        # Not all objects in Boto have an 'add_tags()' method.
+        try:
+            obj.add_tags(tags)
+
+        except AttributeError:
+            for tag, value in tags.items():
+                obj.add_tag(tag, value)
 
         log.debug('The following tags: {0} were added to {1}'.format(', '.join(tags), obj))
 

salt/modules/grains.py

@@ -201,7 +201,7 @@ def item(*args, **kwargs):
     return ret
 
 
-def setvals(grains, destructive=False, refresh=True):
+def setvals(grains, destructive=False):
     '''
     Set new grains values in the grains config file
 
@@ -209,10 +209,6 @@ def setvals(grains, destructive=False, refresh=True):
         If an operation results in a key being removed, delete the key, too.
         Defaults to False.
 
-    refresh
-        Refresh modules and pillar after adding the new grains.
-        Defaults to True.
-
     CLI Example:
 
     .. code-block:: bash
@@ -287,13 +283,13 @@ def setvals(grains, destructive=False, refresh=True):
         msg = 'Unable to write to cache file {0}. Check permissions.'
         log.error(msg.format(fn_))
     if not __opts__.get('local', False):
-        # Sync the grains
+        # Refresh the grains
-        __salt__['saltutil.sync_grains'](refresh=refresh)
+        __salt__['saltutil.refresh_grains']()
     # Return the grains we just set to confirm everything was OK
     return new_grains
 
 
-def setval(key, val, destructive=False, refresh=True):
+def setval(key, val, destructive=False):
     '''
     Set a grains value in the grains config file
 
@@ -307,10 +303,6 @@ def setval(key, val, destructive=False, refresh=True):
         If an operation results in a key being removed, delete the key, too.
         Defaults to False.
 
-    refresh
-        Refresh modules and pillar after adding the new grain.
-        Defaults to True.
-
     CLI Example:
 
     .. code-block:: bash
@@ -318,7 +310,7 @@ def setval(key, val, destructive=False, refresh=True):
         salt '*' grains.setval key val
         salt '*' grains.setval key "{'sub-key': 'val', 'sub-key2': 'val2'}"
     '''
-    return setvals({key: val}, destructive, refresh)
+    return setvals({key: val}, destructive)
 
 
 def append(key, val, convert=False, delimiter=DEFAULT_TARGET_DELIM):

salt/modules/hosts.py

@@ -53,7 +53,7 @@ def _list_hosts():
             if not line:
                 continue
             if line.startswith('#'):
-                ret.setdefault('comment-{0}'.format(count), []).extend(line)
+                ret.setdefault('comment-{0}'.format(count), []).append(line)
                 count += 1
                 continue
             if '#' in line:

salt/modules/saltutil.py

@@ -335,6 +335,39 @@ def sync_states(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blackl
     return ret
 
 
+def refresh_grains(**kwargs):
+    '''
+    .. versionadded:: 2016.3.6,2016.11.4,Nitrogen
+
+    Refresh the minion's grains without syncing custom grains modules from
+    ``salt://_grains``.
+
+    .. note::
+        The available execution modules will be reloaded as part of this
+        proceess, as grains can affect which modules are available.
+
+    refresh_pillar : True
+        Set to ``False`` to keep pillar data from being refreshed.
+
+    CLI Examples:
+
+    .. code-block:: bash
+
+        salt '*' saltutil.refresh_grains
+    '''
+    kwargs = salt.utils.clean_kwargs(**kwargs)
+    _refresh_pillar = kwargs.pop('refresh_pillar', True)
+    if kwargs:
+        salt.utils.invalid_kwargs(kwargs)
+    # Modules and pillar need to be refreshed in case grains changes affected
+    # them, and the module refresh process reloads the grains and assigns the
+    # newly-reloaded grains to each execution module's __grains__ dunder.
+    refresh_modules()
+    if _refresh_pillar:
+        refresh_pillar()
+    return True
+
+
 def sync_grains(saltenv=None, refresh=True, extmod_whitelist=None, extmod_blacklist=None):
     '''
     .. versionadded:: 0.10.0

salt/modules/win_file.py

@@ -54,7 +54,8 @@ from salt.modules.file import (check_hash,  # pylint: disable=W0611
         search, _get_flags, extract_hash, _error, _sed_esc, _psed,
         RE_FLAG_TABLE, blockreplace, prepend, seek_read, seek_write, rename,
         lstat, path_exists_glob, write, pardir, join, HASHES, HASHES_REVMAP,
-        comment, uncomment, _add_flags, comment_line, apply_template_on_contents)
+        comment, uncomment, _add_flags, comment_line, _regex_to_static,
+        _get_line_indent, apply_template_on_contents)
 
 from salt.utils import namespaced_function as _namespaced_function
 
@@ -96,12 +97,13 @@ def __virtual__():
             global append, _error, directory_exists, touch, contains
             global contains_regex, contains_glob, get_source_sum
             global find, psed, get_sum, check_hash, get_hash, delete_backup
-            global get_diff, _get_flags, extract_hash, comment_line
+            global get_diff, line, _get_flags, extract_hash, comment_line
             global access, copy, readdir, rmdir, truncate, replace, search
             global _binary_replace, _get_bkroot, list_backups, restore_backup
             global blockreplace, prepend, seek_read, seek_write, rename, lstat
             global write, pardir, join, _add_flags, apply_template_on_contents
             global path_exists_glob, comment, uncomment, _mkstemp_copy
+            global _regex_to_static, _get_line_indent
 
             replace = _namespaced_function(replace, globals())
             search = _namespaced_function(search, globals())
@@ -134,6 +136,7 @@ def __virtual__():
             check_hash = _namespaced_function(check_hash, globals())
             get_hash = _namespaced_function(get_hash, globals())
             get_diff = _namespaced_function(get_diff, globals())
+            line = _namespaced_function(line, globals())
             access = _namespaced_function(access, globals())
             copy = _namespaced_function(copy, globals())
             readdir = _namespaced_function(readdir, globals())
@@ -152,6 +155,8 @@ def __virtual__():
             comment = _namespaced_function(comment, globals())
             uncomment = _namespaced_function(uncomment, globals())
             comment_line = _namespaced_function(comment_line, globals())
+            _regex_to_static = _namespaced_function(_regex_to_static, globals())
+            _get_line_indent = _namespaced_function(_get_line_indent, globals())
             _mkstemp_copy = _namespaced_function(_mkstemp_copy, globals())
             _add_flags = _namespaced_function(_add_flags, globals())
             apply_template_on_contents = _namespaced_function(apply_template_on_contents, globals())

salt/returners/rawfile_json.py

@@ -58,7 +58,7 @@ def returner(ret):
     opts = _get_options({})  # Pass in empty ret, since this is a list of events
     try:
         with salt.utils.flopen(opts['filename'], 'a') as logfile:
-            logfile.write(str(ret)+'\n')
+            logfile.write(json.dumps(ret)+'\n')
     except:
         log.error('Could not write to rawdata_json file {0}'.format(opts['filename']))
         raise

salt/states/jboss7.py

@@ -448,9 +448,9 @@ def __get_artifact(salt_source):
                 log.debug(traceback.format_exc())
                 comment = 'Unable to manage file: {0}'.format(e)
 
-            else:
+        else:
-                resolved_source = salt_source['target_file']
+            resolved_source = salt_source['target_file']
-                comment = ''
+            comment = ''
 
     return resolved_source, comment
 

salt/transport/zeromq.py

@@ -28,6 +28,7 @@ import salt.transport.mixins.auth
 from salt.exceptions import SaltReqTimeoutError
 
 import zmq
+import zmq.error
 import zmq.eventloop.ioloop
 # support pyzmq 13.0.x, TODO: remove once we force people to 14.0.x
 if not hasattr(zmq.eventloop.ioloop, 'ZMQIOLoop'):
@@ -1071,9 +1072,14 @@ class ZeroMQSocketMonitor(object):
 
     def start_poll(self):
         log.trace("Event monitor start!")
-        while self._monitor_socket is not None and self._monitor_socket.poll():
+        try:
-            msg = self._monitor_socket.recv_multipart()
+            while self._monitor_socket is not None and self._monitor_socket.poll():
-            self.monitor_callback(msg)
+                msg = self._monitor_socket.recv_multipart()
+                self.monitor_callback(msg)
+        except (AttributeError, zmq.error.ContextTerminated):
+            # We cannot log here because we'll get an interrupted system call in trying
+            # to flush the logging buffer as we terminate
+            pass
 
     @property
     def event_map(self):

salt/utils/gitfs.py

@@ -80,7 +80,11 @@ except ImportError:
     HAS_GITPYTHON = False
 
 try:
-    import pygit2
+    # Squelch warning on cent7 due to them upgrading cffi
+    import warnings
+    with warnings.catch_warnings():
+        warnings.simplefilter('ignore')
+        import pygit2
     HAS_PYGIT2 = True
     try:
         GitError = pygit2.errors.GitError

salt/utils/process.py

@@ -399,6 +399,8 @@ class ProcessManager(object):
                     yield gen.sleep(10)
                 else:
                     time.sleep(10)
+                if len(self._process_map) == 0:
+                    break
             # OSError is raised if a signal handler is called (SIGTERM) during os.wait
             except OSError:
                 break
@@ -416,6 +418,7 @@ class ProcessManager(object):
         if self._restart_processes is True:
             for pid, mapping in six.iteritems(self._process_map):
                 if not mapping['Process'].is_alive():
+                    log.trace('Process restart of {0}'.format(pid))
                     self.restart_process(pid)
 
     def kill_children(self, *args, **kwargs):

salt/utils/schedule.py

@@ -444,9 +444,6 @@ class Schedule(object):
             config_dir,
             os.path.dirname(self.opts.get('default_include',
                                           salt.config.DEFAULT_MINION_OPTS['default_include'])))
-        if salt.utils.is_proxy():
-            # each proxy will have a separate _schedule.conf file
-            minion_d_dir = os.path.join(minion_d_dir, self.opts['proxyid'])
 
         if not os.path.isdir(minion_d_dir):
             os.makedirs(minion_d_dir)
@@ -746,7 +743,10 @@ class Schedule(object):
             # This also needed for ZeroMQ transport to reset all functions
             # context data that could keep paretns connections. ZeroMQ will
             # hang on polling parents connections from the child process.
-            self.functions = salt.loader.minion_mods(self.opts, proxy=self.proxy)
+            if self.opts['__role'] == 'master':
+                self.functions = salt.loader.runner(self.opts)
+            else:
+                self.functions = salt.loader.minion_mods(self.opts, proxy=self.proxy)
             self.returners = salt.loader.returners(self.opts, self.functions, proxy=self.proxy)
         ret = {'id': self.opts.get('id', 'master'),
                'fun': func,

tests/unit/modules/test_grains.py

@@ -29,7 +29,7 @@ grainsmod.__opts__ = {
 grainsmod.__salt__ = {}
 
 
-@patch.dict(grainsmod.__salt__, {'saltutil.sync_grains': MagicMock()})
+@patch.dict(grainsmod.__salt__, {'saltutil.refresh_grains': MagicMock()})
 @skipIf(NO_MOCK, NO_MOCK_REASON)
 class GrainsModuleTestCase(TestCase):