diff --git a/doc/ref/configuration/master.rst b/doc/ref/configuration/master.rst index 94a1b996c3..fdf3798f17 100644 --- a/doc/ref/configuration/master.rst +++ b/doc/ref/configuration/master.rst @@ -2467,24 +2467,6 @@ Default: ``[]`` There are additional details at :ref:`salt-pillars` -.. conf_master:: pillar_roots_override_ext_pillar - -``pillar_roots_override_ext_pillar`` ------------------------------------- - -.. versionadded:: 2016.11.0 - -Default: ``False`` - -This option allows for external pillar sources to be evaluated before -:conf_master:`pillar_roots`, which means that values obtained from -:conf_master:`pillar_roots` take precedence over those found from -:conf_master:`ext_pillar` sources. - -.. code-block:: yaml - - pillar_roots_override_ext_pillar: False - .. conf_master:: ext_pillar_first ``ext_pillar_first`` @@ -2496,8 +2478,7 @@ Default: ``False`` This option allows for external pillar sources to be evaluated before :conf_master:`pillar_roots`. This allows for targeting file system pillar from -ext_pillar. Note that ext_pillar_first option is deprecated by -pillar_roots_override_ext_pillar option and will be removed in future releases. +ext_pillar. .. code-block:: yaml diff --git a/doc/topics/releases/2016.11.0.rst b/doc/topics/releases/2016.11.0.rst index 6d68fad9e6..b6f7bbbc78 100644 --- a/doc/topics/releases/2016.11.0.rst +++ b/doc/topics/releases/2016.11.0.rst @@ -104,6 +104,9 @@ Additional Features - The ``junos`` state module is now available. It has all the functions that are present in the ``junos`` execution module. +- The minion data cache is a pluggable data store now. It's configurable with + :conf_master:`cache` option. Default is ``localfs``. +- User names in :conf_master:`client_acl` support glob matching now. New Top File Merging Strategy for States ======================================== @@ -245,7 +248,7 @@ Beacons Changes =============== - The ``loadavg`` beacon now outputs averages as integers instead of strings. - (Via :issuse:`31124`.) + (Via :issue:`31124`.) Runner Changes ============== @@ -271,7 +274,7 @@ Pillar Changes Network Automation: NAPALM ========================== -Beginning with Carbon, network automation is inclued by default in the core +Beginning with 2016.11.0, network automation is inclued by default in the core of Salt. It is based on a the `NAPALM `_ library and provides facilities to manage the configuration and retrieve data from network devices running widely used operating systems such: JunOS, IOS-XR, @@ -295,7 +298,7 @@ In the current release, the following modules were included: - :mod:`Users management state ` Junos Module Changes -=================== +==================== - The following new functionalities were added to the junos module @@ -313,6 +316,36 @@ Returner Changes accept a `minions` keyword argument. All returners which ship with Salt have been modified to do so. +Renderer Changes +================ + +Added the ability to restrict allowed renderers. Two new config parameters, +:conf_master:`renderer_whitelist` and :conf_master:`renderer_blacklist` are +introduced for this purpose. + +eAuth Changes +============= + +- External auth modules' ``auth`` method can return an ACL list for the given + username instead of ``True``. This list should be in the same format as + described in the :doc:`eAuth documentation `. It will be + used for the user instead of one set in master config. + + Example of the ``auth`` method return that allows a user to execute functions + in the ``test`` and ``network`` modules on the minions that match the ``web*`` + target and allow access to ``wheel`` and ``runner`` modules: + + .. code-block:: python + + [{'web*': ['test.*', + 'network.*']}, + '@wheel', + '@runner'] + +- External auth is supported by :doc:`salt-run ` and + :doc:`salt-key ` now. Note that master must be started to + use them with eAuth. + External Module Packaging ========================= @@ -516,7 +549,7 @@ General Deprecations - Deprecations in ``minion.py``: - The ``salt.minion.parse_args_and_kwargs`` function has been removed. Please - use the ``salt.minion.load_args_and_kwargs`` function instead. + use the ``salt.minion.load_args_and_kwargs`` function instead. Cloud Deprecations ------------------ diff --git a/doc/topics/tutorials/salt_bootstrap.rst b/doc/topics/tutorials/salt_bootstrap.rst index 827f721f0d..d5d93a3aa4 100644 --- a/doc/topics/tutorials/salt_bootstrap.rst +++ b/doc/topics/tutorials/salt_bootstrap.rst @@ -115,39 +115,39 @@ Install using curl Using ``curl`` to install latest development version from GitHub: -.. code:: console +.. code-block:: bash - curl -o bootstrap_salt.sh -L https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh git develop + curl -o bootstrap-salt.sh -L https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh git develop If you want to install a specific release version (based on the Git tags): -.. code:: console +.. code-block:: bash - curl -o bootstrap_salt.sh -L https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh git v2015.8.8 + curl -o bootstrap-salt.sh -L https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh git v2015.8.8 To install a specific branch from a Git fork: -.. code:: console +.. code-block:: bash - curl -o bootstrap_salt.sh -L https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh -g https://github.com/myuser/salt.git git mybranch + curl -o bootstrap-salt.sh -L https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh -g https://github.com/myuser/salt.git git mybranch If all you want is to install a ``salt-master`` using latest Git: -.. code:: console +.. code-block:: bash - curl -o bootstrap_salt.sh -L https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh -M -N git develop + curl -o bootstrap-salt.sh -L https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh -M -N git develop If your host has Internet access only via HTTP proxy: -.. code:: console +.. code-block:: bash PROXY='http://user:password@myproxy.example.com:3128' - curl -o bootstrap_salt.sh -L -x "$PROXY" https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh -G -H "$PROXY" git + curl -o bootstrap-salt.sh -L -x "$PROXY" https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh -G -H "$PROXY" git Install using wget @@ -155,24 +155,24 @@ Install using wget Using ``wget`` to install your distribution's stable packages: -.. code:: console +.. code-block:: bash - wget -O bootstrap_salt.sh https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh + wget -O bootstrap-salt.sh https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh Downloading the script from develop branch: .. code-block:: bash - wget -O bootstrap_salt.sh https://bootstrap.saltstack.com/develop - sudo sh bootstrap_salt.sh + wget -O bootstrap-salt.sh https://bootstrap.saltstack.com/develop + sudo sh bootstrap-salt.sh Installing a specific version from git using ``wget``: -.. code:: console +.. code-block:: bash - wget -O bootstrap_salt.sh https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh -P git v2015.8.8 + wget -O bootstrap-salt.sh https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh -P git v2015.8.8 .. note:: @@ -185,17 +185,17 @@ Install using Python If you already have Python installed, ``python 2.6``, then it's as easy as: -.. code:: console +.. code-block:: bash - python -m urllib "https://bootstrap.saltstack.com" > bootstrap_salt.sh - sudo sh bootstrap_salt.sh git develop + python -m urllib "https://bootstrap.saltstack.com" > bootstrap-salt.sh + sudo sh bootstrap-salt.sh git develop All Python versions should support the following in-line code: -.. code:: console +.. code-block:: bash - python -c 'import urllib; print urllib.urlopen("https://bootstrap.saltstack.com").read()' > bootstrap_salt.sh - sudo sh bootstrap_salt.sh git develop + python -c 'import urllib; print urllib.urlopen("https://bootstrap.saltstack.com").read()' > bootstrap-salt.sh + sudo sh bootstrap-salt.sh git develop Install using fetch @@ -204,26 +204,26 @@ Install using fetch On a FreeBSD base system you usually don't have either of the above binaries available. You **do** have ``fetch`` available though: -.. code:: console +.. code-block:: bash - fetch -o bootstrap_salt.sh https://bootstrap.saltstack.com - sudo sh bootstrap_salt.sh + fetch -o bootstrap-salt.sh https://bootstrap.saltstack.com + sudo sh bootstrap-salt.sh If you have any SSL issues install ``ca_root_nssp``: -.. code:: console +.. code-block:: bash pkg install ca_root_nssp And either copy the certificates to the place where fetch can find them: -.. code:: console +.. code-block:: bash cp /usr/local/share/certs/ca-root-nss.crt /etc/ssl/cert.pem Or link them to the right place: -.. code:: console +.. code-block:: bash ln -s /usr/local/share/certs/ca-root-nss.crt /etc/ssl/cert.pem diff --git a/salt/modules/dockerng.py b/salt/modules/dockerng.py index 8500d6fc01..957a5d3a9b 100644 --- a/salt/modules/dockerng.py +++ b/salt/modules/dockerng.py @@ -5889,7 +5889,7 @@ def sls_build(name, base='opensuse/python', mods=None, saltenv='base', # start a new container ret = __salt__['dockerng.create'](image=base, name=name, - cmd='/usr/bin/sleep infinity', + cmd='sleep infinity', interactive=True, tty=True) id_ = ret['Id'] try: diff --git a/salt/modules/postgres.py b/salt/modules/postgres.py index 44286cf746..5c60825ed2 100644 --- a/salt/modules/postgres.py +++ b/salt/modules/postgres.py @@ -26,6 +26,9 @@ Module to provide Postgres compatibility to salt. of the postgres bin's path to the relevant minion for this module:: postgres.pg_bin: '/usr/pgsql-9.5/bin/' + +:note: Older versions of Salt had a bug where postgres.bins_dir was used + instead of postgres.pg_bin. You should upgrade this as soon as possible. ''' # This pylint error is popping up where there are no colons? @@ -51,7 +54,7 @@ except ImportError: import salt.utils import salt.utils.files import salt.utils.itertools -from salt.exceptions import SaltInvocationError +from salt.exceptions import CommandExecutionError, SaltInvocationError # Import 3rd-party libs import salt.ext.six as six @@ -113,9 +116,10 @@ _PRIVILEGE_TYPE_MAP = { def __virtual__(): ''' - Only load this module if the psql and initdb bin exist + Only load this module if the psql bin exist. + initdb bin might also be used, but its presence will be detected on runtime. ''' - utils = ['psql', 'initdb'] + utils = ['psql'] if not HAS_CSV: return False for util in utils: @@ -131,11 +135,21 @@ def _find_pg_binary(util): Helper function to locate various psql related binaries ''' - pg_bin_dir = __salt__['config.option']('postgres.bins_dir') + pg_bin_dir = __salt__['config.option']('postgres.pg_bin') + + if not pg_bin_dir: # Fallback to incorrectly-documented setting + pg_bin_dir = __salt__['config.option']('postgres.bins_dir') + if pg_bin_dir: + salt.utils.warn_until( + 'Oxygen', + 'Using \'postgres.bins_dir\' is not officially supported and ' + 'only exists as a workaround. Please replace this in your ' + 'configuration with \'postgres.pg_bin\'.') + util_bin = salt.utils.which(util) if not util_bin: if pg_bin_dir: - return os.path.join(pg_bin_dir, util) + return salt.utils.which(os.path.join(pg_bin_dir, util)) else: return util_bin @@ -211,6 +225,8 @@ def _run_initdb(name, if user is None: user = runas _INITDB_BIN = _find_pg_binary('initdb') + if not _INITDB_BIN: + raise CommandExecutionError('initdb executable not found.') cmd = [ _INITDB_BIN, '--pgdata={0}'.format(name), diff --git a/salt/pillar/__init__.py b/salt/pillar/__init__.py index e9dcb020bf..35ac252191 100644 --- a/salt/pillar/__init__.py +++ b/salt/pillar/__init__.py @@ -784,27 +784,16 @@ class Pillar(object): ''' top, top_errors = self.get_top() if ext: - if self.opts.get('pillar_roots_override_ext_pillar', False) or self.opts.get('ext_pillar_first', False): - salt.utils.warn_until('Nitrogen', - 'The \'ext_pillar_first\' option has been deprecated and ' - 'replaced by \'pillar_roots_override_ext_pillar\'.' - ) + if self.opts.get('ext_pillar_first', False): self.opts['pillar'], errors = self.ext_pillar({}, pillar_dirs) self.rend = salt.loader.render(self.opts, self.functions) matches = self.top_matches(top) pillar, errors = self.render_pillar(matches, errors=errors) - if self.opts.get('pillar_roots_override_ext_pillar', False): - pillar = merge(self.opts['pillar'], - pillar, - self.merge_strategy, - self.opts.get('renderer', 'yaml'), - self.opts.get('pillar_merge_lists', False)) - else: - pillar = merge(pillar, - self.opts['pillar'], - self.merge_strategy, - self.opts.get('renderer', 'yaml'), - self.opts.get('pillar_merge_lists', False)) + pillar = merge(self.opts['pillar'], + pillar, + self.merge_strategy, + self.opts.get('renderer', 'yaml'), + self.opts.get('pillar_merge_lists', False)) else: matches = self.top_matches(top) pillar, errors = self.render_pillar(matches) diff --git a/salt/states/archive.py b/salt/states/archive.py index a7fc9b5577..88431fa05e 100644 --- a/salt/states/archive.py +++ b/salt/states/archive.py @@ -12,6 +12,7 @@ import logging import os import re import shlex +import shutil import stat import tarfile from contextlib import closing @@ -140,6 +141,7 @@ def extracted(name, enforce_toplevel=True, enforce_ownership_on=None, archive_format=None, + overwrite=False, **kwargs): ''' .. versionadded:: 2014.1.0 @@ -497,6 +499,11 @@ def extracted(name, .. _zipfile: https://docs.python.org/2/library/zipfile.html .. _xz-utils: http://tukaani.org/xz/ + overwrite + If archive was already extracted, then setting this to True will + extract it all over again. + **WARNING: This operation will flush clean all the previous content, if exists!** + **Examples** 1. tar with lmza (i.e. xz) compression: @@ -545,10 +552,9 @@ def extracted(name, ret['comment'] = '{0} is not an absolute path'.format(name) return ret else: - if name is None: - # Only way this happens is if some doofus specifies "- name: None" - # in their SLS file. Prevent tracebacks by failing gracefully. - ret['comment'] = 'None is not a valid directory path' + if not name: + # Empty name, like None, '' etc. + ret['comment'] = 'Name of the directory path needs to be specified' return ret # os.path.isfile() returns False when there is a trailing slash, hence # our need for first stripping the slash and then adding it back later. @@ -898,7 +904,20 @@ def extracted(name, # already need to catch an OSError to cover edge cases where the minion is # running as a non-privileged user and is trying to check for the existence # of a path to which it does not have permission. - extraction_needed = False + + extraction_needed = overwrite + + if extraction_needed: + destination = os.path.join(name, contents['top_level_dirs'][0]) + if os.path.exists(destination): + try: + shutil.rmtree(destination) + except OSError as err: + ret['comment'] = 'Error removing destination directory ' \ + '"{0}": {1}'.format(destination, err) + ret['result'] = False + return ret + try: if_missing_path_exists = os.path.exists(if_missing) except TypeError: diff --git a/tests/unit/modules/dockerng_test.py b/tests/unit/modules/dockerng_test.py index 87cfe128dd..00a9db70a2 100644 --- a/tests/unit/modules/dockerng_test.py +++ b/tests/unit/modules/dockerng_test.py @@ -670,7 +670,7 @@ class DockerngTestCase(TestCase): mods='foo', ) docker_create_mock.assert_called_once_with( - cmd='/usr/bin/sleep infinity', + cmd='sleep infinity', image='opensuse/python', interactive=True, name='foo', tty=True) docker_start_mock.assert_called_once_with('ID') docker_sls_mock.assert_called_once_with('ID', 'foo', 'base')