valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-07 00:55:19 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #38759 from rallytime/merge-2016.11

Browse Source 
[2016.11] Merge forward from 2016.3 to 2016.11
This commit is contained in:
Mike Place 2017-01-17 08:22:00 -07:00 committed by GitHub
commit 751e14c523
15 changed files with 369 additions and 364 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
conf/master
View File

@ -404,6 +404,17 @@
# Pass in an alternative location for the salt-ssh roster file # Pass in an alternative location for the salt-ssh roster file
#roster_file: /etc/salt/roster #roster_file: /etc/salt/roster
# Define a location for roster files so they can be chosen when using Salt API.
# An administrator can place roster files into these locations. Then when
# calling Salt API, parameter 'roster_file' should contain a relative path to
# these locations. That is, "roster_file=/foo/roster" will be resolved as
# "/etc/salt/roster.d/foo/roster" etc. This feature prevents passing insecure
# custom rosters through the Salt API.
#
#rosters:
# - /etc/salt/roster.d
# - /opt/salt/some/more/rosters
# The log file of the salt-ssh command: # The log file of the salt-ssh command:
#ssh_log_file: /var/log/salt/ssh #ssh_log_file: /var/log/salt/ssh

4
doc/ref/configuration/minion.rst
View File

@ -608,7 +608,9 @@ The directory where Unix sockets will be kept.
Default: ``''`` Default: ``''``
Backup files replaced by file.managed and file.recurse under cachedir. Make backups of files replaced by ``file.managed`` and ``file.recurse`` state modules under
:conf_minion:`cachedir` in ``file_backup`` subdirectory preserving original paths.
Refer to :ref:`File State Backups documentation <file-state-backups>` for more details.
.. code-block:: yaml .. code-block:: yaml

27
doc/topics/development/tests/index.rst
View File

@ -139,13 +139,13 @@ Running Test Subsections
Instead of running the entire test suite all at once, which can take a long time, Instead of running the entire test suite all at once, which can take a long time,
there are several ways to run only specific groups of tests or individual tests: there are several ways to run only specific groups of tests or individual tests:
* Run unit tests only: ``./tests/runtests.py --unit-tests`` * Run :ref:`unit tests only<running-unit-tests-no-daemons>`: ``python tests/runtests.py --unit-tests``
* Run unit and integration tests for states: ``./tests/runtests.py --state`` * Run unit and integration tests for states: ``python tests/runtests.py --state``
* Run integration tests for an individual module: ``./tests/runtests.py -n integration.modules.virt`` * Run integration tests for an individual module: ``python tests/runtests.py -n integration.modules.virt``
* Run unit tests for an individual module: ``./tests/runtests.py -n unit.modules.virt_test`` * Run unit tests for an individual module: ``python tests/runtests.py -n unit.modules.virt_test``
* Run an individual test by using the class and test name (this example is for the * Run an individual test by using the class and test name (this example is for the
``test_default_kvm_profile`` test in the ``integration.module.virt``): ``test_default_kvm_profile`` test in the ``integration.module.virt``):
``./tests/runtests.py -n integration.module.virt.VirtTest.test_default_kvm_profile`` ``python tests/runtests.py -n integration.module.virt.VirtTest.test_default_kvm_profile``
For more specific examples of how to run various test subsections or individual For more specific examples of how to run various test subsections or individual
tests, please see the :ref:`Test Selection Options <test-selection-options>` tests, please see the :ref:`Test Selection Options <test-selection-options>`
@ -163,14 +163,14 @@ Since the unit tests do not require a master or minion to execute, it is often u
run unit tests individually, or as a whole group, without having to start up the integration testing run unit tests individually, or as a whole group, without having to start up the integration testing
daemons. Starting up the master, minion, and syndic daemons takes a lot of time before the tests can daemons. Starting up the master, minion, and syndic daemons takes a lot of time before the tests can
even start running and is unnecessary to run unit tests. To run unit tests without invoking the even start running and is unnecessary to run unit tests. To run unit tests without invoking the
integration test daemons, simple remove the ``/tests`` portion of the ``runtests.py`` command: integration test daemons, simply run the ``runtests.py`` script with ``--unit`` argument:
.. code-block:: bash .. code-block:: bash
./runtests.py --unit python tests/runtests.py --unit
All of the other options to run individual tests, entire classes of tests, or entire test modules still All of the other options to run individual tests, entire classes of tests, or
apply. entire test modules still apply.
Running Destructive Integration Tests Running Destructive Integration Tests
@ -191,13 +191,14 @@ successfully. Therefore, running destructive tests should be done with caution.
.. note:: .. note::
Running destructive tests will change the underlying system. Use caution when running destructive tests. Running destructive tests will change the underlying system.
Use caution when running destructive tests.
To run tests marked as destructive, set the ``--run-destructive`` flag: To run tests marked as destructive, set the ``--run-destructive`` flag:
.. code-block:: bash .. code-block:: bash
./tests/runtests.py --run-destructive python tests/runtests.py --run-destructive
Running Cloud Provider Tests Running Cloud Provider Tests
@ -259,13 +260,13 @@ Here's a simple usage example:
.. code-block:: bash .. code-block:: bash
tests/runtests.py --docked=ubuntu-12.04 -v python tests/runtests.py --docked=ubuntu-12.04 -v
The full `docker`_ container repository can also be provided: The full `docker`_ container repository can also be provided:
.. code-block:: bash .. code-block:: bash
tests/runtests.py --docked=salttest/ubuntu-12.04 -v python tests/runtests.py --docked=salttest/ubuntu-12.04 -v
The SaltStack team is creating some containers which will have the necessary The SaltStack team is creating some containers which will have the necessary

2
doc/topics/netapi/index.rst
View File

@ -38,7 +38,7 @@ simply by creating a data structure. (And this is exactly how much of Salt's
own internals work!) own internals work!)
.. autoclass:: salt.netapi.NetapiClient .. autoclass:: salt.netapi.NetapiClient
:members: local, local_async, local_batch, local_subset, ssh, ssh_async, :members: local, local_async, local_subset, ssh, ssh_async,
runner, runner_async, wheel, wheel_async runner, runner_async, wheel, wheel_async
.. toctree:: .. toctree::

31
doc/topics/releases/2015.8.13.rst Normal file
View File

@ -0,0 +1,31 @@
============================
Salt 2015.8.13 Release Notes
============================
Version 2015.8.13 is a bugfix release for :ref:`2015.8.0 <release-2015-8-0>`.
Changes for v2015.8.12..v2015.8.13
----------------------------------
Extended changelog courtesy of Todd Stansell (https://github.com/tjstansell/salt-changelogs):
*Generated at: 2017-01-09T21:17:06Z*
Statistics:
- Total Merges: **3**
- Total Issue references: **3**
- Total PR references: **5**
Changes:
* 3428232 Clean up tests and docs for batch execution
* 3d8f3d1 Remove batch execution from NetapiClient and Saltnado
* 97b0f64 Lintfix
* d151666 Add explanation comment
* 62f2c87 Add docstring
* 9b0a786 Explain what it is about and how to configure that
* 5ea3579 Pick up a specified roster file from the configured locations
* 3a8614c Disable custom rosters in API
* c0e5a11 Add roster disable flag

4
doc/topics/tutorials/jinja_to_execution_module.rst
View File

@ -58,7 +58,7 @@ Unfortunately, it can lead to code that looks like the following.
{% endfor %} {% endfor %}
This is an example from the author's salt formulae demonstrating misuse of jinja. This is an example from the author's salt formulae demonstrating misuse of jinja.
Aside from being difficult to read and maintian, Aside from being difficult to read and maintain,
accessing the logic it contains from a non-jinja renderer accessing the logic it contains from a non-jinja renderer
while probably possible is a significant barrier! while probably possible is a significant barrier!
@ -158,6 +158,6 @@ Conclusion
---------- ----------
That was... surprisingly straight-forward. That was... surprisingly straight-forward.
Now the logic is now available in every renderer, instead of just Jinja. Now the logic is available in every renderer, instead of just Jinja.
Best of all, it can be maintained in Python, Best of all, it can be maintained in Python,
which is a whole lot easier than Jinja. which is a whole lot easier than Jinja.

2
pkg/windows/req.txt
View File

@ -11,7 +11,7 @@ idna==2.1
ioflo==1.5.5 ioflo==1.5.5
ioloop==0.1a0 ioloop==0.1a0
ipaddress==1.0.16 ipaddress==1.0.16
Jinja2==2.8 Jinja2==2.9.4
libnacl==1.4.5 libnacl==1.4.5
lxml==3.6.0 lxml==3.6.0
Mako==1.0.4 Mako==1.0.4

6
salt/client/ssh/client.py
View File

@ -22,7 +22,8 @@ class SSHClient(object):
''' '''
def __init__(self, def __init__(self,
c_path=os.path.join(syspaths.CONFIG_DIR, 'master'), c_path=os.path.join(syspaths.CONFIG_DIR, 'master'),
mopts=None): mopts=None,
disable_custom_roster=False):
if mopts: if mopts:
self.opts = mopts self.opts = mopts
else: else:
@ -35,6 +36,9 @@ class SSHClient(object):
) )
self.opts = salt.config.client_config(c_path) self.opts = salt.config.client_config(c_path)
# Salt API should never offer a custom roster!
self.opts['__disable_custom_roster'] = disable_custom_roster
def _prep_ssh( def _prep_ssh(
self, self,
tgt, tgt,

408
salt/cloud/deploy/bootstrap-salt.sh
View File

@ -9,7 +9,7 @@
# #
# BUGS: https://github.com/saltstack/salt-bootstrap/issues # BUGS: https://github.com/saltstack/salt-bootstrap/issues
# #
# COPYRIGHT: (c) 2012-2016 by the SaltStack Team, see AUTHORS.rst for more # COPYRIGHT: (c) 2012-2017 by the SaltStack Team, see AUTHORS.rst for more
# details. # details.
# #
# LICENSE: Apache 2.0 # LICENSE: Apache 2.0
@ -18,7 +18,7 @@
#====================================================================================================================== #======================================================================================================================
set -o nounset # Treat unset variables as an error set -o nounset # Treat unset variables as an error
__ScriptVersion="2016.10.25" __ScriptVersion="2017.01.10"
__ScriptName="bootstrap-salt.sh" __ScriptName="bootstrap-salt.sh"
__ScriptFullName="$0" __ScriptFullName="$0"
@ -309,9 +309,10 @@ __usage() {
-F Allow copied files to overwrite existing (config, init.d, etc) -F Allow copied files to overwrite existing (config, init.d, etc)
-K If set, keep the temporary files in the temporary directories specified -K If set, keep the temporary files in the temporary directories specified
with -c and -k with -c and -k
-C Only run the configuration function. This option automatically bypasses -C Only run the configuration function. Implies -F (forced overwrite).
any installation. Implies -F (forced overwrite). To overwrite master or To overwrite Master or Syndic configs, -M or -S, respectively, must
syndic configs, -M or -S, respectively, must also be specified. also be specified. Salt installation will be ommitted, but some of the
dependencies could be installed to write configuration with -j or -J.
-A Pass the salt-master DNS name or IP. This will be stored under -A Pass the salt-master DNS name or IP. This will be stored under
\${BS_SALT_ETC_DIR}/minion.d/99-master-address.conf \${BS_SALT_ETC_DIR}/minion.d/99-master-address.conf
-i Pass the salt-minion id. This will be stored under -i Pass the salt-minion id. This will be stored under
@ -342,12 +343,12 @@ __usage() {
repo.saltstack.com. The option passed with -R replaces the repo.saltstack.com. The option passed with -R replaces the
"repo.saltstack.com". If -R is passed, -r is also set. Currently only "repo.saltstack.com". If -R is passed, -r is also set. Currently only
works on CentOS/RHEL based distributions. works on CentOS/RHEL based distributions.
-J Replace the Master config file with data passed in as a json string. If -J Replace the Master config file with data passed in as a JSON string. If
a Master config file is found, a reasonable effort will be made to save a Master config file is found, a reasonable effort will be made to save
the file with a ".bak" extension. If used in conjunction with -C or -F, the file with a ".bak" extension. If used in conjunction with -C or -F,
no ".bak" file will be created as either of those options will force no ".bak" file will be created as either of those options will force
a complete overwrite of the file. a complete overwrite of the file.
-j Replace the Minion config file with data passed in as a json string. If -j Replace the Minion config file with data passed in as a JSON string. If
a Minion config file is found, a reasonable effort will be made to save a Minion config file is found, a reasonable effort will be made to save
the file with a ".bak" extension. If used in conjunction with -C or -F, the file with a ".bak" extension. If used in conjunction with -C or -F,
no ".bak" file will be created as either of those options will force no ".bak" file will be created as either of those options will force
@ -475,7 +476,7 @@ fi
# Check that we're installing or configuring a master if we're being passed a master config json dict # Check that we're installing or configuring a master if we're being passed a master config json dict
if [ "$_CUSTOM_MASTER_CONFIG" != "null" ]; then if [ "$_CUSTOM_MASTER_CONFIG" != "null" ]; then
if [ "$_INSTALL_MASTER" -eq $BS_FALSE ] && [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then if [ "$_INSTALL_MASTER" -eq $BS_FALSE ] && [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then
echoerror "Don't pass a master config json dict (-J) if no master is going to be bootstrapped or configured." echoerror "Don't pass a master config JSON dict (-J) if no master is going to be bootstrapped or configured."
exit 1 exit 1
fi fi
fi fi
@ -483,7 +484,7 @@ fi
# Check that we're installing or configuring a minion if we're being passed a minion config json dict # Check that we're installing or configuring a minion if we're being passed a minion config json dict
if [ "$_CUSTOM_MINION_CONFIG" != "null" ]; then if [ "$_CUSTOM_MINION_CONFIG" != "null" ]; then
if [ "$_INSTALL_MINION" -eq $BS_FALSE ] && [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then if [ "$_INSTALL_MINION" -eq $BS_FALSE ] && [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then
echoerror "Don't pass a minion config json dict (-j) if no minion is going to be bootstrapped or configured." echoerror "Don't pass a minion config JSON dict (-j) if no minion is going to be bootstrapped or configured."
exit 1 exit 1
fi fi
fi fi
@ -850,7 +851,7 @@ __derive_debian_numeric_version() {
# DESCRIPTION: Strip single or double quotes from the provided string. # DESCRIPTION: Strip single or double quotes from the provided string.
#---------------------------------------------------------------------------------------------------------------------- #----------------------------------------------------------------------------------------------------------------------
__unquote_string() { __unquote_string() {
echo "$*" | sed -e "s/^\([\"']\)\(.*\)\1\$/\2/g" echo "$*" | sed -e "s/^\([\"\']\)\(.*\)\1\$/\2/g"
} }
#--- FUNCTION ------------------------------------------------------------------------------------------------------- #--- FUNCTION -------------------------------------------------------------------------------------------------------
@ -924,6 +925,8 @@ __gather_linux_system_info() {
DISTRO_NAME=$(lsb_release -si) DISTRO_NAME=$(lsb_release -si)
if [ "${DISTRO_NAME}" = "Scientific" ]; then if [ "${DISTRO_NAME}" = "Scientific" ]; then
DISTRO_NAME="Scientific Linux" DISTRO_NAME="Scientific Linux"
elif [ "$(echo "$DISTRO_NAME" | grep ^CloudLinux)" != "" ]; then
DISTRO_NAME="Cloud Linux"
elif [ "$(echo "$DISTRO_NAME" | grep ^RedHat)" != "" ]; then elif [ "$(echo "$DISTRO_NAME" | grep ^RedHat)" != "" ]; then
# Let's convert 'CamelCased' to 'Camel Cased' # Let's convert 'CamelCased' to 'Camel Cased'
n=$(__camelcase_split "$DISTRO_NAME") n=$(__camelcase_split "$DISTRO_NAME")
@ -1037,6 +1040,9 @@ __gather_linux_system_info() {
n="Arch Linux" n="Arch Linux"
v="" # Arch Linux does not provide a version. v="" # Arch Linux does not provide a version.
;; ;;
cloudlinux )
n="Cloud Linux"
;;
debian ) debian )
n="Debian" n="Debian"
v=$(__derive_debian_numeric_version "$v") v=$(__derive_debian_numeric_version "$v")
@ -1195,12 +1201,6 @@ __ubuntu_derivatives_translation() {
# Mappings # Mappings
trisquel_6_ubuntu_base="12.04" trisquel_6_ubuntu_base="12.04"
linuxmint_13_ubuntu_base="12.04" linuxmint_13_ubuntu_base="12.04"
linuxmint_14_ubuntu_base="12.10"
#linuxmint_15_ubuntu_base="13.04"
# Bug preventing add-apt-repository from working on Mint 15:
# https://bugs.launchpad.net/linuxmint/+bug/1198751
linuxmint_16_ubuntu_base="13.10"
linuxmint_17_ubuntu_base="14.04" linuxmint_17_ubuntu_base="14.04"
linuxmint_18_ubuntu_base="16.04" linuxmint_18_ubuntu_base="16.04"
linaro_12_ubuntu_base="12.04" linaro_12_ubuntu_base="12.04"
@ -1258,15 +1258,12 @@ __ubuntu_codename_translation() {
"14") "14")
DISTRO_CODENAME="trusty" DISTRO_CODENAME="trusty"
;; ;;
"15")
if [ -n "$_april" ]; then
DISTRO_CODENAME="vivid"
else
DISTRO_CODENAME="wily"
fi
;;
"16") "16")
DISTRO_CODENAME="xenial" if [ "$_april" ]; then
DISTRO_CODENAME="xenial"
else
DISTRO_CODENAME="yakkety"
fi
;; ;;
*) *)
DISTRO_CODENAME="trusty" DISTRO_CODENAME="trusty"
@ -1453,6 +1450,14 @@ if ([ "${DISTRO_NAME_L}" != "ubuntu" ] && [ $_PIP_ALL -eq $BS_TRUE ]);then
exit 1 exit 1
fi fi
# Starting from Ubuntu 16.10, gnupg-curl has been renamed to gnupg1-curl.
GNUPG_CURL="gnupg-curl"
if ([ "${DISTRO_NAME_L}" = "ubuntu" ] && [ "${DISTRO_VERSION}" = "16.10" ]); then
GNUPG_CURL="gnupg1-curl"
fi
#--- FUNCTION ------------------------------------------------------------------------------------------------------- #--- FUNCTION -------------------------------------------------------------------------------------------------------
# NAME: __function_defined # NAME: __function_defined
# DESCRIPTION: Checks if a function is defined within this scripts scope # DESCRIPTION: Checks if a function is defined within this scripts scope
@ -1497,7 +1502,7 @@ __apt_get_upgrade_noinput() {
__apt_key_fetch() { __apt_key_fetch() {
url=$1 url=$1
__apt_get_install_noinput gnupg-curl || return 1 __apt_get_install_noinput ${GNUPG_CURL} || return 1
# shellcheck disable=SC2086 # shellcheck disable=SC2086
apt-key adv ${_GPG_ARGS} --fetch-keys "$url"; return $? apt-key adv ${_GPG_ARGS} --fetch-keys "$url"; return $?
@ -1561,6 +1566,10 @@ __yum_install_noinput() {
__git_clone_and_checkout() { __git_clone_and_checkout() {
echodebug "Installed git version: $(git --version | awk '{ print $3 }')" echodebug "Installed git version: $(git --version | awk '{ print $3 }')"
# Turn off SSL verification if -I flag was set for insecure downloads
if [ "$_INSECURE_DL" -eq $BS_TRUE ]; then
export GIT_SSL_NO_VERIFY=1
fi
__SALT_GIT_CHECKOUT_PARENT_DIR=$(dirname "${_SALT_GIT_CHECKOUT_DIR}" 2>/dev/null) __SALT_GIT_CHECKOUT_PARENT_DIR=$(dirname "${_SALT_GIT_CHECKOUT_DIR}" 2>/dev/null)
__SALT_GIT_CHECKOUT_PARENT_DIR="${__SALT_GIT_CHECKOUT_PARENT_DIR:-/tmp/git}" __SALT_GIT_CHECKOUT_PARENT_DIR="${__SALT_GIT_CHECKOUT_PARENT_DIR:-/tmp/git}"
@ -1689,7 +1698,12 @@ __check_end_of_life_versions() {
# Ubuntu versions not supported # Ubuntu versions not supported
# #
# < 12.04 # < 12.04
if [ "$DISTRO_MAJOR_VERSION" -lt 12 ]; then # 13.x, 15.x
# 12.10, 14.10
if [ "$DISTRO_MAJOR_VERSION" -lt 12 ] || \
[ "$DISTRO_MAJOR_VERSION" -eq 13 ] || \
[ "$DISTRO_MAJOR_VERSION" -eq 15 ] || \
([ "$DISTRO_MAJOR_VERSION" -lt 16 ] && [ "$DISTRO_MINOR_VERSION" -eq 10 ]); then
echoerror "End of life distributions are not supported." echoerror "End of life distributions are not supported."
echoerror "Please consider upgrading to the next stable. See:" echoerror "Please consider upgrading to the next stable. See:"
echoerror " https://wiki.ubuntu.com/Releases" echoerror " https://wiki.ubuntu.com/Releases"
@ -1726,7 +1740,7 @@ __check_end_of_life_versions() {
fedora) fedora)
# Fedora lower than 18 are no longer supported # Fedora lower than 18 are no longer supported
if [ "$DISTRO_MAJOR_VERSION" -lt 18 ]; then if [ "$DISTRO_MAJOR_VERSION" -lt 23 ]; then
echoerror "End of life distributions are not supported." echoerror "End of life distributions are not supported."
echoerror "Please consider upgrading to the next stable. See:" echoerror "Please consider upgrading to the next stable. See:"
echoerror " https://fedoraproject.org/wiki/Releases" echoerror " https://fedoraproject.org/wiki/Releases"
@ -2284,49 +2298,30 @@ __enable_universe_repository() {
echodebug "Enabling the universe repository" echodebug "Enabling the universe repository"
# Ubuntu versions higher than 12.04 do not live in the old repositories add-apt-repository -y "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) universe" || return 1
if [ "$DISTRO_MAJOR_VERSION" -gt 12 ] || ([ "$DISTRO_MAJOR_VERSION" -eq 12 ] && [ "$DISTRO_MINOR_VERSION" -gt 04 ]); then
add-apt-repository -y "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) universe" || return 1
elif [ "$DISTRO_MAJOR_VERSION" -lt 11 ] && [ "$DISTRO_MINOR_VERSION" -lt 10 ]; then
# Below Ubuntu 11.10, the -y flag to add-apt-repository is not supported
add-apt-repository "deb http://old-releases.ubuntu.com/ubuntu $(lsb_release -sc) universe" || return 1
else
add-apt-repository -y "deb http://old-releases.ubuntu.com/ubuntu $(lsb_release -sc) universe" || return 1
fi
add-apt-repository -y "deb http://old-releases.ubuntu.com/ubuntu $(lsb_release -sc) universe" || return 1
return 0 return 0
} }
install_ubuntu_deps() { install_ubuntu_deps() {
if [ "$DISTRO_MAJOR_VERSION" -gt 12 ] || ([ "$DISTRO_MAJOR_VERSION" -eq 12 ] && [ "$DISTRO_MINOR_VERSION" -eq 10 ]); then if [ "$DISTRO_MAJOR_VERSION" -gt 12 ]; then
# Above Ubuntu 12.04 add-apt-repository is in a different package # Above Ubuntu 12.04 add-apt-repository is in a different package
__apt_get_install_noinput software-properties-common || return 1 __apt_get_install_noinput software-properties-common || return 1
else else
__apt_get_install_noinput python-software-properties || return 1 __apt_get_install_noinput python-software-properties || return 1
fi fi
if [ $_DISABLE_REPOS -eq $BS_FALSE ]; then if [ $_DISABLE_REPOS -eq $BS_FALSE ]; then
__enable_universe_repository || return 1 __enable_universe_repository || return 1
# Versions starting with 2015.5.6 and 2015.8.1 are hosted at repo.saltstack.com # Versions starting with 2015.5.6 and 2015.8.1 are hosted at repo.saltstack.com
if [ "$(echo "$STABLE_REV" | egrep '^(2015\.5|2015\.8|latest|archive\/)')" = "" ]; then if [ "$(echo "$STABLE_REV" | egrep '^(2015\.5|2015\.8|2016\.3|2016\.11|latest|archive\/)')" = "" ]; then
if [ "$DISTRO_MAJOR_VERSION" -lt 14 ]; then if [ "$DISTRO_MAJOR_VERSION" -lt 14 ]; then
echoinfo "Installing Python Requests/Chardet from Chris Lea's PPA repository" echoinfo "Installing Python Requests/Chardet from Chris Lea's PPA repository"
if [ "$DISTRO_MAJOR_VERSION" -gt 11 ] || ([ "$DISTRO_MAJOR_VERSION" -eq 11 ] && [ "$DISTRO_MINOR_VERSION" -gt 04 ]); then add-apt-repository -y "ppa:chris-lea/python-requests" || return 1
# Above Ubuntu 11.04 add a -y flag add-apt-repository -y "ppa:chris-lea/python-chardet" || return 1
add-apt-repository -y "ppa:chris-lea/python-requests" || return 1 add-apt-repository -y "ppa:chris-lea/python-urllib3" || return 1
add-apt-repository -y "ppa:chris-lea/python-chardet" || return 1 add-apt-repository -y "ppa:chris-lea/python-crypto" || return 1
add-apt-repository -y "ppa:chris-lea/python-urllib3" || return 1
add-apt-repository -y "ppa:chris-lea/python-crypto" || return 1
else
add-apt-repository "ppa:chris-lea/python-requests" || return 1
add-apt-repository "ppa:chris-lea/python-chardet" || return 1
add-apt-repository "ppa:chris-lea/python-urllib3" || return 1
add-apt-repository "ppa:chris-lea/python-crypto" || return 1
fi
fi fi
fi fi
@ -2337,7 +2332,7 @@ install_ubuntu_deps() {
# Minimal systems might not have upstart installed, install it # Minimal systems might not have upstart installed, install it
__PACKAGES="upstart" __PACKAGES="upstart"
if [ "$DISTRO_MAJOR_VERSION" -ge 15 ]; then if [ "$DISTRO_MAJOR_VERSION" -ge 16 ]; then
__PACKAGES="${__PACKAGES} python2.7" __PACKAGES="${__PACKAGES} python2.7"
fi fi
if [ "$_VIRTUALENV_DIR" != "null" ]; then if [ "$_VIRTUALENV_DIR" != "null" ]; then
@ -2349,6 +2344,9 @@ install_ubuntu_deps() {
# requests is still used by many salt modules # requests is still used by many salt modules
__PACKAGES="${__PACKAGES} python-requests" __PACKAGES="${__PACKAGES} python-requests"
# YAML module is used for generating custom master/minion configs
__PACKAGES="${__PACKAGES} python-yaml"
# Additionally install procps and pciutils which allows for Docker bootstraps. See 366#issuecomment-39666813 # Additionally install procps and pciutils which allows for Docker bootstraps. See 366#issuecomment-39666813
__PACKAGES="${__PACKAGES} procps pciutils" __PACKAGES="${__PACKAGES} procps pciutils"
@ -2365,7 +2363,7 @@ install_ubuntu_deps() {
} }
install_ubuntu_stable_deps() { install_ubuntu_stable_deps() {
if ([ "${_SLEEP}" -eq "${__DEFAULT_SLEEP}" ] && [ "$DISTRO_MAJOR_VERSION" -lt 15 ]); then if [ "${_SLEEP}" -eq "${__DEFAULT_SLEEP}" ] && [ "$DISTRO_MAJOR_VERSION" -lt 16 ]; then
# The user did not pass a custom sleep value as an argument, let's increase the default value # The user did not pass a custom sleep value as an argument, let's increase the default value
echodebug "On Ubuntu systems we increase the default sleep value to 10." echodebug "On Ubuntu systems we increase the default sleep value to 10."
echodebug "See https://github.com/saltstack/salt/issues/12248 for more info." echodebug "See https://github.com/saltstack/salt/issues/12248 for more info."
@ -2408,12 +2406,12 @@ install_ubuntu_stable_deps() {
fi fi
# Versions starting with 2015.5.6, 2015.8.1 and 2016.3.0 are hosted at repo.saltstack.com # Versions starting with 2015.5.6, 2015.8.1 and 2016.3.0 are hosted at repo.saltstack.com
if [ "$(echo "$STABLE_REV" | egrep '^(2015\.5|2015\.8|2016\.3|latest|archive\/)')" != "" ]; then if [ "$(echo "$STABLE_REV" | egrep '^(2015\.5|2015\.8|2016\.3|2016\.11|latest|archive\/)')" != "" ]; then
# Workaround for latest non-LTS ubuntu # Workaround for latest non-LTS ubuntu
if [ "$DISTRO_MAJOR_VERSION" -eq 15 ]; then if [ "$DISTRO_VERSION" = "16.10" ]; then
echowarn "Non-LTS Ubuntu detected, but stable packages requested. Trying packages from latest LTS release. You may experience problems." echowarn "Non-LTS Ubuntu detected, but stable packages requested. Trying packages from latest LTS release. You may experience problems."
UBUNTU_VERSION=14.04 UBUNTU_VERSION=16.04
UBUNTU_CODENAME=trusty UBUNTU_CODENAME=xenial
else else
UBUNTU_VERSION=$DISTRO_VERSION UBUNTU_VERSION=$DISTRO_VERSION
UBUNTU_CODENAME=$DISTRO_CODENAME UBUNTU_CODENAME=$DISTRO_CODENAME
@ -2439,12 +2437,7 @@ install_ubuntu_stable_deps() {
STABLE_PPA="saltstack/salt" STABLE_PPA="saltstack/salt"
fi fi
if [ "$DISTRO_MAJOR_VERSION" -gt 11 ] || ([ "$DISTRO_MAJOR_VERSION" -eq 11 ] && [ "$DISTRO_MINOR_VERSION" -gt 04 ]); then add-apt-repository -y "ppa:$STABLE_PPA" || return 1
# Above Ubuntu 11.04 add a -y flag
add-apt-repository -y "ppa:$STABLE_PPA" || return 1
else
add-apt-repository "ppa:$STABLE_PPA" || return 1
fi
fi fi
apt-get update apt-get update
@ -2456,24 +2449,17 @@ install_ubuntu_stable_deps() {
install_ubuntu_daily_deps() { install_ubuntu_daily_deps() {
install_ubuntu_stable_deps || return 1 install_ubuntu_stable_deps || return 1
if [ "$DISTRO_MAJOR_VERSION" -ge 12 ]; then if [ "$DISTRO_MAJOR_VERSION" -gt 12 ]; then
# Above Ubuntu 11.10 add-apt-repository is in a different package
__apt_get_install_noinput software-properties-common || return 1 __apt_get_install_noinput software-properties-common || return 1
else else
# Ubuntu 12.04 needs python-software-properties to get add-apt-repository binary
__apt_get_install_noinput python-software-properties || return 1 __apt_get_install_noinput python-software-properties || return 1
fi fi
if [ $_DISABLE_REPOS -eq $BS_FALSE ]; then if [ $_DISABLE_REPOS -eq $BS_FALSE ]; then
__enable_universe_repository || return 1 __enable_universe_repository || return 1
# for anything up to and including 11.04 do not use the -y option add-apt-repository -y ppa:saltstack/salt-daily || return 1
if [ "$DISTRO_MAJOR_VERSION" -gt 11 ] || ([ "$DISTRO_MAJOR_VERSION" -eq 11 ] && [ "$DISTRO_MINOR_VERSION" -gt 04 ]); then
# Above Ubuntu 11.04 add a -y flag
add-apt-repository -y ppa:saltstack/salt-daily || return 1
else
add-apt-repository ppa:saltstack/salt-daily || return 1
fi
apt-get update apt-get update
fi fi
@ -2486,7 +2472,15 @@ install_ubuntu_daily_deps() {
install_ubuntu_git_deps() { install_ubuntu_git_deps() {
apt-get update apt-get update
__apt_get_install_noinput git-core || return 1
if ! __check_command_exists git; then
__apt_get_install_noinput git-core || return 1
fi
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
__apt_get_install_noinput ca-certificates
fi
__git_clone_and_checkout || return 1 __git_clone_and_checkout || return 1
__PACKAGES="" __PACKAGES=""
@ -2569,12 +2563,6 @@ install_ubuntu_git() {
} }
install_ubuntu_stable_post() { install_ubuntu_stable_post() {
# Workaround for latest LTS packages on latest ubuntu. Normally packages on
# debian-based systems will automatically start the corresponding daemons
if [ "$DISTRO_MAJOR_VERSION" -lt 15 ]; then
return 0
fi
for fname in api master minion syndic; do for fname in api master minion syndic; do
# Skip salt-api since the service should be opt-in and not necessarily started on boot # Skip salt-api since the service should be opt-in and not necessarily started on boot
[ $fname = "api" ] && continue [ $fname = "api" ] && continue
@ -2607,7 +2595,7 @@ install_ubuntu_git_post() {
[ $fname = "minion" ] && [ "$_INSTALL_MINION" -eq $BS_FALSE ] && continue [ $fname = "minion" ] && [ "$_INSTALL_MINION" -eq $BS_FALSE ] && continue
[ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue [ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue
if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 15 ]; then if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 16 ]; then
__copyfile "${_SALT_GIT_CHECKOUT_DIR}/pkg/deb/salt-${fname}.service" "/lib/systemd/system/salt-${fname}.service" __copyfile "${_SALT_GIT_CHECKOUT_DIR}/pkg/deb/salt-${fname}.service" "/lib/systemd/system/salt-${fname}.service"
# Skip salt-api since the service should be opt-in and not necessarily started on boot # Skip salt-api since the service should be opt-in and not necessarily started on boot
@ -2652,7 +2640,7 @@ install_ubuntu_restart_daemons() {
[ $_START_DAEMONS -eq $BS_FALSE ] && return [ $_START_DAEMONS -eq $BS_FALSE ] && return
# Ensure upstart configs / systemd units are loaded # Ensure upstart configs / systemd units are loaded
if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 15 ]; then if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 16 ]; then
systemctl daemon-reload systemctl daemon-reload
elif [ -f /sbin/initctl ]; then elif [ -f /sbin/initctl ]; then
/sbin/initctl reload-configuration /sbin/initctl reload-configuration
@ -2667,7 +2655,7 @@ install_ubuntu_restart_daemons() {
[ $fname = "minion" ] && [ "$_INSTALL_MINION" -eq $BS_FALSE ] && continue [ $fname = "minion" ] && [ "$_INSTALL_MINION" -eq $BS_FALSE ] && continue
[ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue [ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue
if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 15 ]; then if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 16 ]; then
echodebug "There's systemd support while checking salt-$fname" echodebug "There's systemd support while checking salt-$fname"
systemctl stop salt-$fname > /dev/null 2>&1 systemctl stop salt-$fname > /dev/null 2>&1
systemctl start salt-$fname.service systemctl start salt-$fname.service
@ -2711,7 +2699,7 @@ install_ubuntu_check_services() {
[ $fname = "master" ] && [ "$_INSTALL_MASTER" -eq $BS_FALSE ] && continue [ $fname = "master" ] && [ "$_INSTALL_MASTER" -eq $BS_FALSE ] && continue
[ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue [ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue
if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 15 ]; then if [ -f /bin/systemctl ] && [ "$DISTRO_MAJOR_VERSION" -ge 16 ]; then
__check_services_systemd salt-$fname || return 1 __check_services_systemd salt-$fname || return 1
elif [ -f /sbin/initctl ] && [ -f /etc/init/salt-${fname}.conf ]; then elif [ -f /sbin/initctl ] && [ -f /etc/init/salt-${fname}.conf ]; then
__check_services_upstart salt-$fname || return 1 __check_services_upstart salt-$fname || return 1
@ -2755,6 +2743,9 @@ install_debian_deps() {
__PACKAGES="procps pciutils" __PACKAGES="procps pciutils"
__PIP_PACKAGES="" __PIP_PACKAGES=""
# YAML module is used for generating custom master/minion configs
__PACKAGES="${__PACKAGES} python-yaml"
# shellcheck disable=SC2086 # shellcheck disable=SC2086
__apt_get_install_noinput ${__PACKAGES} || return 1 __apt_get_install_noinput ${__PACKAGES} || return 1
@ -2817,7 +2808,7 @@ install_debian_7_deps() {
fi fi
# Versions starting with 2015.8.7 and 2016.3.0 are hosted at repo.saltstack.com # Versions starting with 2015.8.7 and 2016.3.0 are hosted at repo.saltstack.com
if [ "$(echo "$STABLE_REV" | egrep '^(2015\.8|2016\.3|latest|archive\/201[5-6]\.)')" != "" ]; then if [ "$(echo "$STABLE_REV" | egrep '^(2015\.8|2016\.3|2016\.11|latest|archive\/201[5-6]\.)')" != "" ]; then
# amd64 is just a part of repository URI, 32-bit pkgs are hosted under the same location # amd64 is just a part of repository URI, 32-bit pkgs are hosted under the same location
SALTSTACK_DEBIAN_URL="${HTTP_VAL}://repo.saltstack.com/apt/debian/${DISTRO_MAJOR_VERSION}/${__REPO_ARCH}/${STABLE_REV}" SALTSTACK_DEBIAN_URL="${HTTP_VAL}://repo.saltstack.com/apt/debian/${DISTRO_MAJOR_VERSION}/${__REPO_ARCH}/${STABLE_REV}"
echo "deb $SALTSTACK_DEBIAN_URL wheezy main" > "/etc/apt/sources.list.d/saltstack.list" echo "deb $SALTSTACK_DEBIAN_URL wheezy main" > "/etc/apt/sources.list.d/saltstack.list"
@ -2841,6 +2832,9 @@ install_debian_7_deps() {
# Additionally install procps and pciutils which allows for Docker bootstraps. See 366#issuecomment-39666813 # Additionally install procps and pciutils which allows for Docker bootstraps. See 366#issuecomment-39666813
__PACKAGES='procps pciutils' __PACKAGES='procps pciutils'
# YAML module is used for generating custom master/minion configs
__PACKAGES="${__PACKAGES} python-yaml"
# shellcheck disable=SC2086 # shellcheck disable=SC2086
__apt_get_install_noinput ${__PACKAGES} || return 1 __apt_get_install_noinput ${__PACKAGES} || return 1
@ -2896,7 +2890,7 @@ install_debian_8_deps() {
fi fi
# Versions starting with 2015.5.6, 2015.8.1 and 2016.3.0 are hosted at repo.saltstack.com # Versions starting with 2015.5.6, 2015.8.1 and 2016.3.0 are hosted at repo.saltstack.com
if [ "$(echo "$STABLE_REV" | egrep '^(2015\.5|2015\.8|2016\.3|latest|archive\/201[5-6]\.)')" != "" ]; then if [ "$(echo "$STABLE_REV" | egrep '^(2015\.5|2015\.8|2016\.3|2016\.11|latest|archive\/201[5-6]\.)')" != "" ]; then
SALTSTACK_DEBIAN_URL="${HTTP_VAL}://repo.saltstack.com/apt/debian/${DISTRO_MAJOR_VERSION}/${__REPO_ARCH}/${STABLE_REV}" SALTSTACK_DEBIAN_URL="${HTTP_VAL}://repo.saltstack.com/apt/debian/${DISTRO_MAJOR_VERSION}/${__REPO_ARCH}/${STABLE_REV}"
echo "deb $SALTSTACK_DEBIAN_URL jessie main" > "/etc/apt/sources.list.d/saltstack.list" echo "deb $SALTSTACK_DEBIAN_URL jessie main" > "/etc/apt/sources.list.d/saltstack.list"
@ -2920,9 +2914,8 @@ install_debian_8_deps() {
# shellcheck disable=SC2086 # shellcheck disable=SC2086
__apt_get_install_noinput ${__PACKAGES} || return 1 __apt_get_install_noinput ${__PACKAGES} || return 1
if [ "$_UPGRADE_SYS" -eq $BS_TRUE ]; then # YAML module is used for generating custom master/minion configs
__apt_get_upgrade_noinput || return 1 __PACKAGES="${__PACKAGES} python-yaml"
fi
if [ "${_EXTRA_PACKAGES}" != "" ]; then if [ "${_EXTRA_PACKAGES}" != "" ]; then
echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}" echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}"
@ -2938,10 +2931,14 @@ install_debian_git_deps() {
__apt_get_install_noinput git || return 1 __apt_get_install_noinput git || return 1
fi fi
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
__apt_get_install_noinput ca-certificates
fi
__git_clone_and_checkout || return 1 __git_clone_and_checkout || return 1
__PACKAGES="libzmq3 libzmq3-dev lsb-release python-apt python-backports.ssl-match-hostname python-crypto" __PACKAGES="libzmq3 libzmq3-dev lsb-release python-apt python-backports.ssl-match-hostname python-crypto"
__PACKAGES="${__PACKAGES} python-jinja2 python-msgpack python-requests python-tornado" __PACKAGES="${__PACKAGES} python-jinja2 python-msgpack python-requests"
__PACKAGES="${__PACKAGES} python-tornado python-yaml python-zmq" __PACKAGES="${__PACKAGES} python-tornado python-yaml python-zmq"
if [ "$_INSTALL_CLOUD" -eq $BS_TRUE ]; then if [ "$_INSTALL_CLOUD" -eq $BS_TRUE ]; then
@ -2975,9 +2972,14 @@ install_debian_8_git_deps() {
__apt_get_install_noinput git || return 1 __apt_get_install_noinput git || return 1
fi fi
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
__apt_get_install_noinput ca-certificates
fi
__git_clone_and_checkout || return 1 __git_clone_and_checkout || return 1
__PACKAGES='libzmq3 libzmq3-dev lsb-release python-apt python-crypto python-jinja2 python-msgpack python-requests python-yaml python-zmq' __PACKAGES="libzmq3 libzmq3-dev lsb-release python-apt python-crypto python-jinja2 python-msgpack"
__PACKAGES="${__PACKAGES} python-requests python-systemd python-yaml python-zmq"
if [ "$_INSTALL_CLOUD" -eq $BS_TRUE ]; then if [ "$_INSTALL_CLOUD" -eq $BS_TRUE ]; then
# Install python-libcloud if asked to # Install python-libcloud if asked to
@ -3184,16 +3186,7 @@ install_debian_check_services() {
# Fedora Install Functions # Fedora Install Functions
# #
FEDORA_PACKAGE_MANAGER="yum"
__fedora_get_package_manager() {
if [ "$DISTRO_MAJOR_VERSION" -ge 22 ] || __check_command_exists dnf; then
FEDORA_PACKAGE_MANAGER="dnf"
fi
}
install_fedora_deps() { install_fedora_deps() {
__fedora_get_package_manager
if [ $_DISABLE_REPOS -eq $BS_FALSE ]; then if [ $_DISABLE_REPOS -eq $BS_FALSE ]; then
if [ "$_ENABLE_EXTERNAL_ZMQ_REPOS" -eq $BS_TRUE ]; then if [ "$_ENABLE_EXTERNAL_ZMQ_REPOS" -eq $BS_TRUE ]; then
@ -3203,32 +3196,25 @@ install_fedora_deps() {
__install_saltstack_copr_salt_repository || return 1 __install_saltstack_copr_salt_repository || return 1
fi fi
__PACKAGES="yum-utils PyYAML libyaml python-crypto python-jinja2 python-zmq" __PACKAGES="yum-utils PyYAML libyaml python-crypto python-jinja2 python-zmq python2-msgpack python2-requests"
if [ "$DISTRO_MAJOR_VERSION" -ge 23 ]; then
__PACKAGES="${__PACKAGES} python2-msgpack python2-requests"
else
__PACKAGES="${__PACKAGES} python-msgpack python-requests"
fi
# shellcheck disable=SC2086 # shellcheck disable=SC2086
$FEDORA_PACKAGE_MANAGER install -y ${__PACKAGES} || return 1 dnf install -y ${__PACKAGES} || return 1
if [ "$_UPGRADE_SYS" -eq $BS_TRUE ]; then if [ "$_UPGRADE_SYS" -eq $BS_TRUE ]; then
$FEDORA_PACKAGE_MANAGER -y update || return 1 dnf -y update || return 1
fi fi
if [ "${_EXTRA_PACKAGES}" != "" ]; then if [ "${_EXTRA_PACKAGES}" != "" ]; then
echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}" echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}"
# shellcheck disable=SC2086 # shellcheck disable=SC2086
$FEDORA_PACKAGE_MANAGER install -y ${_EXTRA_PACKAGES} || return 1 dnf install -y ${_EXTRA_PACKAGES} || return 1
fi fi
return 0 return 0
} }
install_fedora_stable() { install_fedora_stable() {
__fedora_get_package_manager
__PACKAGES="" __PACKAGES=""
if [ "$_INSTALL_CLOUD" -eq $BS_TRUE ];then if [ "$_INSTALL_CLOUD" -eq $BS_TRUE ];then
@ -3245,7 +3231,7 @@ install_fedora_stable() {
fi fi
# shellcheck disable=SC2086 # shellcheck disable=SC2086
$FEDORA_PACKAGE_MANAGER install -y ${__PACKAGES} || return 1 dnf install -y ${__PACKAGES} || return 1
return 0 return 0
} }
@ -3267,11 +3253,15 @@ install_fedora_stable_post() {
} }
install_fedora_git_deps() { install_fedora_git_deps() {
__fedora_get_package_manager
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
dnf install -y ca-certificates || return 1
fi
install_fedora_deps || return 1 install_fedora_deps || return 1
if ! __check_command_exists git; then if ! __check_command_exists git; then
$FEDORA_PACKAGE_MANAGER install -y git || return 1 dnf install -y git || return 1
fi fi
__git_clone_and_checkout || return 1 __git_clone_and_checkout || return 1
@ -3299,7 +3289,7 @@ install_fedora_git_deps() {
fi fi
# shellcheck disable=SC2086 # shellcheck disable=SC2086
$FEDORA_PACKAGE_MANAGER install -y ${__PACKAGES} || return 1 dnf install -y ${__PACKAGES} || return 1
if [ "${__PIP_PACKAGES}" != "" ]; then if [ "${__PIP_PACKAGES}" != "" ]; then
# shellcheck disable=SC2086,SC2090 # shellcheck disable=SC2086,SC2090
@ -3449,7 +3439,13 @@ __install_saltstack_rhel_repository() {
repo_url="repo.saltstack.com" repo_url="repo.saltstack.com"
fi fi
base_url="${HTTP_VAL}://${repo_url}/yum/redhat/\$releasever/\$basearch/${repo_rev}/" # Cloud Linux $releasever = 7.x, which doesn't exist in repo.saltstack.com, we need this to be "7"
if [ "${DISTRO_NAME}" = "Cloud Linux" ] && [ "${DISTRO_MAJOR_VERSION}" = "7" ]; then
base_url="${HTTP_VAL}://${repo_url}/yum/redhat/${DISTRO_MAJOR_VERSION}/\$basearch/${repo_rev}/"
else
base_url="${HTTP_VAL}://${repo_url}/yum/redhat/\$releasever/\$basearch/${repo_rev}/"
fi
fetch_url="${HTTP_VAL}://${repo_url}/yum/redhat/${DISTRO_MAJOR_VERSION}/${CPU_ARCH_L}/${repo_rev}/" fetch_url="${HTTP_VAL}://${repo_url}/yum/redhat/${DISTRO_MAJOR_VERSION}/${CPU_ARCH_L}/${repo_rev}/"
if [ "${DISTRO_MAJOR_VERSION}" -eq 5 ]; then if [ "${DISTRO_MAJOR_VERSION}" -eq 5 ]; then
@ -3528,14 +3524,23 @@ install_centos_stable_deps() {
__PACKAGES="yum-utils chkconfig" __PACKAGES="yum-utils chkconfig"
if [ "${_EXTRA_PACKAGES}" != "" ]; then # YAML module is used for generating custom master/minion configs
echoinfo "Also installing the following extra packages as requested: ${_EXTRA_PACKAGES}" if [ "$DISTRO_MAJOR_VERSION" -eq 5 ]; then
__PACKAGES="${__PACKAGES} ${_EXTRA_PACKAGES}" __PACKAGES="${__PACKAGES} python26-PyYAML"
else
__PACKAGES="${__PACKAGES} PyYAML"
fi fi
# shellcheck disable=SC2086 # shellcheck disable=SC2086
__yum_install_noinput ${__PACKAGES} || return 1 __yum_install_noinput ${__PACKAGES} || return 1
if [ "${_EXTRA_PACKAGES}" != "" ]; then
echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}"
# shellcheck disable=SC2086
__yum_install_noinput ${_EXTRA_PACKAGES} || return 1
fi
return 0 return 0
} }
@ -3574,7 +3579,7 @@ install_centos_stable_post() {
[ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue [ $fname = "syndic" ] && [ "$_INSTALL_SYNDIC" -eq $BS_FALSE ] && continue
if [ -f /bin/systemctl ]; then if [ -f /bin/systemctl ]; then
/usr/systemctl is-enabled salt-${fname}.service > /dev/null 2>&1 || ( /bin/systemctl is-enabled salt-${fname}.service > /dev/null 2>&1 || (
/bin/systemctl preset salt-${fname}.service > /dev/null 2>&1 && /bin/systemctl preset salt-${fname}.service > /dev/null 2>&1 &&
/bin/systemctl enable salt-${fname}.service > /dev/null 2>&1 /bin/systemctl enable salt-${fname}.service > /dev/null 2>&1
) )
@ -3593,6 +3598,14 @@ install_centos_stable_post() {
} }
install_centos_git_deps() { install_centos_git_deps() {
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
if [ "$DISTRO_MAJOR_VERSION" -gt 5 ]; then
__yum_install_noinput ca-certificates || return 1
else
__yum_install_noinput "openssl.${CPU_ARCH_L}" || return 1
fi
fi
install_centos_stable_deps || return 1 install_centos_stable_deps || return 1
if ! __check_command_exists git; then if ! __check_command_exists git; then
@ -3604,10 +3617,10 @@ install_centos_git_deps() {
__PACKAGES="" __PACKAGES=""
if [ "$DISTRO_MAJOR_VERSION" -eq 5 ]; then if [ "$DISTRO_MAJOR_VERSION" -eq 5 ]; then
__PACKAGES="${__PACKAGES} python26-PyYAML python26 python26-requests" __PACKAGES="${__PACKAGES} python26 python26-crypto python26-jinja2 python26-msgpack python26-requests"
__PACKAGES="${__PACKAGES} python26-crypto python26-jinja2 python26-msgpack python26-tornado python26-zmq" __PACKAGES="${__PACKAGES} python26-tornado python26-zmq"
else else
__PACKAGES="${__PACKAGES} PyYAML python-crypto python-futures python-msgpack python-zmq python-jinja2" __PACKAGES="${__PACKAGES} python-crypto python-futures python-msgpack python-zmq python-jinja2"
__PACKAGES="${__PACKAGES} python-requests python-tornado" __PACKAGES="${__PACKAGES} python-requests python-tornado"
fi fi
@ -4082,6 +4095,69 @@ install_scientific_linux_check_services() {
# #
####################################################################################################################### #######################################################################################################################
#######################################################################################################################
#
# CloudLinux Install Functions
#
install_cloud_linux_stable_deps() {
install_centos_stable_deps || return 1
return 0
}
install_cloud_linux_git_deps() {
install_centos_git_deps || return 1
return 0
}
install_cloud_linux_testing_deps() {
install_centos_testing_deps || return 1
return 0
}
install_cloud_linux_stable() {
install_centos_stable || return 1
return 0
}
install_cloud_linux_git() {
install_centos_git || return 1
return 0
}
install_cloud_linux_testing() {
install_centos_testing || return 1
return 0
}
install_cloud_linux_stable_post() {
install_centos_stable_post || return 1
return 0
}
install_cloud_linux_git_post() {
install_centos_git_post || return 1
return 0
}
install_cloud_linux_testing_post() {
install_centos_testing_post || return 1
return 0
}
install_cloud_linux_restart_daemons() {
install_centos_restart_daemons || return 1
return 0
}
install_cloud_linux_check_services() {
install_centos_check_services || return 1
return 0
}
#
# End of CloudLinux Install Functions
#
#######################################################################################################################
####################################################################################################################### #######################################################################################################################
# #
# Amazon Linux AMI Install Functions # Amazon Linux AMI Install Functions
@ -4089,6 +4165,10 @@ install_scientific_linux_check_services() {
install_amazon_linux_ami_deps() { install_amazon_linux_ami_deps() {
# We need to install yum-utils before doing anything else when installing on
# Amazon Linux ECS-optimized images. See issue #974.
yum -y install yum-utils
ENABLE_EPEL_CMD="" ENABLE_EPEL_CMD=""
if [ $_DISABLE_REPOS -eq $BS_TRUE ]; then if [ $_DISABLE_REPOS -eq $BS_TRUE ]; then
ENABLE_EPEL_CMD="--enablerepo=${_EPEL_REPO}" ENABLE_EPEL_CMD="--enablerepo=${_EPEL_REPO}"
@ -4133,6 +4213,10 @@ _eof
} }
install_amazon_linux_ami_git_deps() { install_amazon_linux_ami_git_deps() {
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
yum -y install ca-certificates || return 1
fi
install_amazon_linux_ami_deps || return 1 install_amazon_linux_ami_deps || return 1
ENABLE_EPEL_CMD="" ENABLE_EPEL_CMD=""
@ -4238,6 +4322,9 @@ install_arch_linux_stable_deps() {
pacman-db-upgrade || return 1 pacman-db-upgrade || return 1
fi fi
# YAML module is used for generating custom master/minion configs
pacman -Sy --noconfirm --needed python2-yaml
if [ "$_UPGRADE_SYS" -eq $BS_TRUE ]; then if [ "$_UPGRADE_SYS" -eq $BS_TRUE ]; then
pacman -Syyu --noconfirm --needed || return 1 pacman -Syyu --noconfirm --needed || return 1
fi fi
@ -4262,7 +4349,7 @@ install_arch_linux_git_deps() {
fi fi
pacman -R --noconfirm python2-distribute pacman -R --noconfirm python2-distribute
pacman -Sy --noconfirm --needed python2-crypto python2-setuptools python2-jinja \ pacman -Sy --noconfirm --needed python2-crypto python2-setuptools python2-jinja \
python2-markupsafe python2-msgpack python2-psutil python2-yaml \ python2-markupsafe python2-msgpack python2-psutil \
python2-pyzmq zeromq python2-requests python2-systemd || return 1 python2-pyzmq zeromq python2-requests python2-systemd || return 1
__git_clone_and_checkout || return 1 __git_clone_and_checkout || return 1
@ -4293,7 +4380,7 @@ install_arch_linux_stable() {
pacman -S --noconfirm --needed bash || return 1 pacman -S --noconfirm --needed bash || return 1
pacman -Su --noconfirm || return 1 pacman -Su --noconfirm || return 1
# We can now resume regular salt update # We can now resume regular salt update
pacman -Syu --noconfirm salt-zmq || return 1 pacman -Syu --noconfirm salt || return 1
return 0 return 0
} }
@ -4515,6 +4602,10 @@ install_freebsd_9_stable_deps() {
# shellcheck disable=SC2086 # shellcheck disable=SC2086
/usr/local/sbin/pkg install ${FROM_FREEBSD} -y swig || return 1 /usr/local/sbin/pkg install ${FROM_FREEBSD} -y swig || return 1
# YAML module is used for generating custom master/minion configs
# shellcheck disable=SC2086
/usr/local/sbin/pkg install ${FROM_FREEBSD} -y py27-yaml || return 1
if [ "${_EXTRA_PACKAGES}" != "" ]; then if [ "${_EXTRA_PACKAGES}" != "" ]; then
echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}" echoinfo "Installing the following extra packages as requested: ${_EXTRA_PACKAGES}"
# shellcheck disable=SC2086 # shellcheck disable=SC2086
@ -5027,8 +5118,7 @@ __ZYPPER_REQUIRES_REPLACE_FILES=-1
__set_suse_pkg_repo() { __set_suse_pkg_repo() {
suse_pkg_url_path="${DISTRO_REPO}/systemsmanagement:saltstack.repo" suse_pkg_url_path="${DISTRO_REPO}/systemsmanagement:saltstack.repo"
if [ "$_DOWNSTREAM_PKG_REPO" -eq $BS_TRUE ]; then if [ "$_DOWNSTREAM_PKG_REPO" -eq $BS_TRUE ]; then
# FIXME: cleartext download over unsecure protocol (HTTP) suse_pkg_url_base="http://download.opensuse.org/repositories/systemsmanagement:/saltstack"
suse_pkg_url_base="http://download.opensuse.org/repositories/systemsmanagement:saltstack"
else else
suse_pkg_url_base="${HTTP_VAL}://repo.saltstack.com/opensuse" suse_pkg_url_base="${HTTP_VAL}://repo.saltstack.com/opensuse"
fi fi
@ -5127,6 +5217,10 @@ install_opensuse_stable_deps() {
} }
install_opensuse_git_deps() { install_opensuse_git_deps() {
if [ "$_INSECURE_DL" -eq $BS_FALSE ] && [ "${_SALT_REPO_URL%%://*}" = "https" ]; then
__zypper_install ca-certificates || return 1
fi
install_opensuse_stable_deps || return 1 install_opensuse_stable_deps || return 1
if ! __check_command_exists git; then if ! __check_command_exists git; then
@ -5917,7 +6011,7 @@ config_salt() {
# Copy the minions configuration if found # Copy the minions configuration if found
# Explicitly check for custom master config to avoid moving the minion config # Explicitly check for custom master config to avoid moving the minion config
elif [ -f "$_TEMP_CONFIG_DIR/minion" ] && [ "$_CUSTOM_MASTER_CONFIG" = "null" ]; then elif [ -f "$_TEMP_CONFIG_DIR/minion" ] && [ "$_CUSTOM_MASTER_CONFIG" = "null" ]; then
__movefile "$_TEMP_CONFIG_DIR/minion" "$_SALT_ETC_DIR" "$_CONFIG_ONLY" || return 1 __movefile "$_TEMP_CONFIG_DIR/minion" "$_SALT_ETC_DIR" "$_FORCE_OVERWRITE" || return 1
CONFIGURED_ANYTHING=$BS_TRUE CONFIGURED_ANYTHING=$BS_TRUE
fi fi
@ -6008,9 +6102,6 @@ config_salt() {
exit 0 exit 0
fi fi
# Create default logs directory if not exists
mkdir -p /var/log/salt
return 0 return 0
} }
# #
@ -6116,7 +6207,7 @@ for FUNC_NAME in $(__strip_duplicates "$DEP_FUNC_NAMES"); do
done done
echodebug "DEPS_INSTALL_FUNC=${DEPS_INSTALL_FUNC}" echodebug "DEPS_INSTALL_FUNC=${DEPS_INSTALL_FUNC}"
# Let's get the minion config function # Let's get the Salt config function
CONFIG_FUNC_NAMES="config_${DISTRO_NAME_L}${PREFIXED_DISTRO_MAJOR_VERSION}_${ITYPE}_salt" CONFIG_FUNC_NAMES="config_${DISTRO_NAME_L}${PREFIXED_DISTRO_MAJOR_VERSION}_${ITYPE}_salt"
CONFIG_FUNC_NAMES="$CONFIG_FUNC_NAMES config_${DISTRO_NAME_L}${PREFIXED_DISTRO_MAJOR_VERSION}${PREFIXED_DISTRO_MINOR_VERSION}_${ITYPE}_salt" CONFIG_FUNC_NAMES="$CONFIG_FUNC_NAMES config_${DISTRO_NAME_L}${PREFIXED_DISTRO_MAJOR_VERSION}${PREFIXED_DISTRO_MINOR_VERSION}_${ITYPE}_salt"
CONFIG_FUNC_NAMES="$CONFIG_FUNC_NAMES config_${DISTRO_NAME_L}${PREFIXED_DISTRO_MAJOR_VERSION}_salt" CONFIG_FUNC_NAMES="$CONFIG_FUNC_NAMES config_${DISTRO_NAME_L}${PREFIXED_DISTRO_MAJOR_VERSION}_salt"
@ -6265,6 +6356,16 @@ if [ "$_CUSTOM_MASTER_CONFIG" != "null" ] || [ "$_CUSTOM_MINION_CONFIG" != "null
if [ "$_TEMP_CONFIG_DIR" = "null" ]; then if [ "$_TEMP_CONFIG_DIR" = "null" ]; then
_TEMP_CONFIG_DIR="$_SALT_ETC_DIR" _TEMP_CONFIG_DIR="$_SALT_ETC_DIR"
fi fi
if [ "$_CONFIG_ONLY" -eq $BS_TRUE ]; then
# Execute function to satisfy dependencies for configuration step
echoinfo "Running ${DEPS_INSTALL_FUNC}()"
$DEPS_INSTALL_FUNC
if [ $? -ne 0 ]; then
echoerror "Failed to run ${DEPS_INSTALL_FUNC}()!!!"
exit 1
fi
fi
fi fi
# Configure Salt # Configure Salt
@ -6277,7 +6378,21 @@ if [ "$CONFIG_SALT_FUNC" != "null" ] && [ "$_TEMP_CONFIG_DIR" != "null" ]; then
fi fi
fi fi
# Pre-Seed master keys # Drop the master address if passed
if [ "$_SALT_MASTER_ADDRESS" != "null" ]; then
[ ! -d "$_SALT_ETC_DIR/minion.d" ] && mkdir -p "$_SALT_ETC_DIR/minion.d"
cat <<_eof > $_SALT_ETC_DIR/minion.d/99-master-address.conf
master: $_SALT_MASTER_ADDRESS
_eof
fi
# Drop the minion id if passed
if [ "$_SALT_MINION_ID" != "null" ]; then
[ ! -d "$_SALT_ETC_DIR" ] && mkdir -p "$_SALT_ETC_DIR"
echo "$_SALT_MINION_ID" > "$_SALT_ETC_DIR/minion_id"
fi
# Pre-seed master keys
if [ "$PRESEED_MASTER_FUNC" != "null" ] && [ "$_TEMP_KEYS_DIR" != "null" ]; then if [ "$PRESEED_MASTER_FUNC" != "null" ] && [ "$_TEMP_KEYS_DIR" != "null" ]; then
echoinfo "Running ${PRESEED_MASTER_FUNC}()" echoinfo "Running ${PRESEED_MASTER_FUNC}()"
$PRESEED_MASTER_FUNC $PRESEED_MASTER_FUNC
@ -6298,29 +6413,6 @@ if [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then
fi fi
fi fi
# Ensure that the cachedir exists
# (Workaround for https://github.com/saltstack/salt/issues/6502)
if [ "$_INSTALL_MINION" -eq $BS_TRUE ]; then
if [ ! -d "${_SALT_CACHE_DIR}/minion/proc" ]; then
echodebug "Creating salt's cachedir"
mkdir -p "${_SALT_CACHE_DIR}/minion/proc"
fi
fi
# Drop the master address if passed
if [ "$_SALT_MASTER_ADDRESS" != "null" ]; then
[ ! -d "$_SALT_ETC_DIR/minion.d" ] && mkdir -p "$_SALT_ETC_DIR/minion.d"
cat <<_eof > $_SALT_ETC_DIR/minion.d/99-master-address.conf
master: $_SALT_MASTER_ADDRESS
_eof
fi
# Drop the minion id if passed
if [ "$_SALT_MINION_ID" != "null" ]; then
[ ! -d "$_SALT_ETC_DIR" ] && mkdir -p "$_SALT_ETC_DIR"
echo "$_SALT_MINION_ID" > "$_SALT_ETC_DIR/minion_id"
fi
# Run any post install function. Only execute function if not in config mode only # Run any post install function. Only execute function if not in config mode only
if [ "$POST_INSTALL_FUNC" != "null" ] && [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then if [ "$POST_INSTALL_FUNC" != "null" ] && [ "$_CONFIG_ONLY" -eq $BS_FALSE ]; then
echoinfo "Running ${POST_INSTALL_FUNC}()" echoinfo "Running ${POST_INSTALL_FUNC}()"

13
salt/config/__init__.py
View File

@ -498,11 +498,6 @@ VALID_OPTS = {
# http://api.zeromq.org/3-2:zmq-setsockopt # http://api.zeromq.org/3-2:zmq-setsockopt
'pub_hwm': int, 'pub_hwm': int,
# ZMQ HWM for SaltEvent pub socket
'salt_event_pub_hwm': int,
# ZMQ HWM for EventPublisher pub socket
'event_publisher_pub_hwm': int,
# IPC buffer size # IPC buffer size
# Refs https://github.com/saltstack/salt/issues/34215 # Refs https://github.com/saltstack/salt/issues/34215
'ipc_write_buffer': int, 'ipc_write_buffer': int,
@ -1162,10 +1157,6 @@ DEFAULT_MINION_OPTS = {
'sudo_user': '', 'sudo_user': '',
'http_request_timeout': 1 * 60 * 60.0, # 1 hour 'http_request_timeout': 1 * 60 * 60.0, # 1 hour
'http_max_body': 100 * 1024 * 1024 * 1024, # 100GB 'http_max_body': 100 * 1024 * 1024 * 1024, # 100GB
# ZMQ HWM for SaltEvent pub socket - different for minion vs. master
'salt_event_pub_hwm': 2000,
# ZMQ HWM for EventPublisher pub socket - different for minion vs. master
'event_publisher_pub_hwm': 1000,
'event_match_type': 'startswith', 'event_match_type': 'startswith',
'minion_restart_command': [], 'minion_restart_command': [],
'pub_ret': True, 'pub_ret': True,
@ -1183,10 +1174,6 @@ DEFAULT_MASTER_OPTS = {
'publish_port': 4505, 'publish_port': 4505,
'zmq_backlog': 1000, 'zmq_backlog': 1000,
'pub_hwm': 1000, 'pub_hwm': 1000,
# ZMQ HWM for SaltEvent pub socket - different for minion vs. master
'salt_event_pub_hwm': 2000,
# ZMQ HWM for EventPublisher pub socket - different for minion vs. master
'event_publisher_pub_hwm': 1000,
'auth_mode': 1, 'auth_mode': 1,
'user': 'root', 'user': 'root',
'worker_threads': 5, 'worker_threads': 5,

15
salt/netapi/__init__.py
View File

@ -98,18 +98,6 @@ class NetapiClient(object):
local = salt.client.get_local_client(mopts=self.opts) local = salt.client.get_local_client(mopts=self.opts)
return local.cmd(*args, **kwargs) return local.cmd(*args, **kwargs)
def local_batch(self, *args, **kwargs):
'''
Run :ref:`execution modules <all-salt.modules>` against batches of minions
Wraps :py:meth:`salt.client.LocalClient.cmd_batch`
:return: Returns the result from the exeuction module for each batch of
returns
'''
local = salt.client.get_local_client(mopts=self.opts)
return local.cmd_batch(*args, **kwargs)
def local_subset(self, *args, **kwargs): def local_subset(self, *args, **kwargs):
''' '''
Run :ref:`execution modules <all-salt.modules>` against subsets of minions Run :ref:`execution modules <all-salt.modules>` against subsets of minions
@ -129,7 +117,8 @@ class NetapiClient(object):
:return: Returns the result from the salt-ssh command :return: Returns the result from the salt-ssh command
''' '''
ssh_client = salt.client.ssh.client.SSHClient(mopts=self.opts) ssh_client = salt.client.ssh.client.SSHClient(mopts=self.opts,
disable_custom_roster=True)
return ssh_client.cmd_sync(kwargs) return ssh_client.cmd_sync(kwargs)
def runner(self, fun, timeout=None, **kwargs): def runner(self, fun, timeout=None, **kwargs):

80
salt/netapi/rest_tornado/saltnado.py
View File

@ -191,7 +191,6 @@ a return like::
# Import Python libs # Import Python libs
import time import time
import math
import fnmatch import fnmatch
import logging import logging
from copy import copy from copy import copy
@ -230,7 +229,6 @@ logger = logging.getLogger()
# # all of these require coordinating minion stuff # # all of these require coordinating minion stuff
# - "local" (done) # - "local" (done)
# - "local_async" (done) # - "local_async" (done)
# - "local_batch" (done)
# # master side # # master side
# - "runner" (done) # - "runner" (done)
@ -252,7 +250,6 @@ class SaltClientsMixIn(object):
SaltClientsMixIn.__saltclients = { SaltClientsMixIn.__saltclients = {
'local': local_client.run_job_async, 'local': local_client.run_job_async,
# not the actual client we'll use.. but its what we'll use to get args # not the actual client we'll use.. but its what we'll use to get args
'local_batch': local_client.cmd_batch,
'local_async': local_client.run_job_async, 'local_async': local_client.run_job_async,
'runner': salt.runner.RunnerClient(opts=self.application.opts).cmd_async, 'runner': salt.runner.RunnerClient(opts=self.application.opts).cmd_async,
'runner_async': None, # empty, since we use the same client as `runner` 'runner_async': None, # empty, since we use the same client as `runner`
@ -390,30 +387,6 @@ class EventListener(object):
del self.timeout_map[future] del self.timeout_map[future]
# TODO: move to a utils function within salt-- the batching stuff is a bit tied together
def get_batch_size(batch, num_minions):
'''
Return the batch size that you should have
batch: string
num_minions: int
'''
# figure out how many we can keep in flight
partition = lambda x: float(x) / 100.0 * num_minions
try:
if '%' in batch:
res = partition(float(batch.strip('%')))
if res < 1:
return int(math.ceil(res))
else:
return int(res)
else:
return int(batch)
except ValueError:
print(('Invalid batch data sent: {0}\nData must be in the form'
'of %10, 10% or 3').format(batch))
class BaseSaltAPIHandler(tornado.web.RequestHandler, SaltClientsMixIn): # pylint: disable=W0223 class BaseSaltAPIHandler(tornado.web.RequestHandler, SaltClientsMixIn): # pylint: disable=W0223
ct_out_map = ( ct_out_map = (
('application/json', json.dumps), ('application/json', json.dumps),
@ -809,7 +782,7 @@ class SaltAPIHandler(BaseSaltAPIHandler, SaltClientsMixIn): # pylint: disable=W
Content-Type: application/json Content-Type: application/json
Content-Legnth: 83 Content-Legnth: 83
{"clients": ["local", "local_batch", "local_async", "runner", "runner_async"], "return": "Welcome"} {"clients": ["local", "local_async", "runner", "runner_async"], "return": "Welcome"}
''' '''
ret = {"clients": list(self.saltclients.keys()), ret = {"clients": list(self.saltclients.keys()),
"return": "Welcome"} "return": "Welcome"}
@ -927,57 +900,6 @@ class SaltAPIHandler(BaseSaltAPIHandler, SaltClientsMixIn): # pylint: disable=W
self.write(self.serialize({'return': ret})) self.write(self.serialize({'return': ret}))
self.finish() self.finish()
@tornado.gen.coroutine
def _disbatch_local_batch(self, chunk):
'''
Disbatch local client batched commands
'''
f_call = salt.utils.format_call(self.saltclients['local_batch'], chunk)
# ping all the minions (to see who we have to talk to)
# Don't catch any exception, since we won't know what to do, we'll
# let the upper level deal with this one
ping_ret = yield self._disbatch_local({'tgt': chunk['tgt'],
'fun': 'test.ping',
'expr_form': f_call['kwargs']['expr_form']})
chunk_ret = {}
if not isinstance(ping_ret, dict):
raise tornado.gen.Return(chunk_ret)
minions = list(ping_ret.keys())
maxflight = get_batch_size(f_call['kwargs']['batch'], len(minions))
inflight_futures = []
# override the expr_form
f_call['kwargs']['expr_form'] = 'list'
# do this batch
while len(minions) > 0 or len(inflight_futures) > 0:
# if you have more to go, lets disbatch jobs
while len(inflight_futures) < maxflight and len(minions) > 0:
minion_id = minions.pop(0)
batch_chunk = dict(chunk)
batch_chunk['tgt'] = [minion_id]
batch_chunk['expr_form'] = 'list'
future = self._disbatch_local(batch_chunk)
inflight_futures.append(future)
# if we have nothing to wait for, don't wait
if len(inflight_futures) == 0:
continue
# wait until someone is done
finished_future = yield Any(inflight_futures)
try:
b_ret = finished_future.result()
except TimeoutException:
break
chunk_ret.update(b_ret)
inflight_futures.remove(finished_future)
raise tornado.gen.Return(chunk_ret)
@tornado.gen.coroutine @tornado.gen.coroutine
def _disbatch_local(self, chunk): def _disbatch_local(self, chunk):
''' '''

42
salt/roster/__init__.py
View File

@ -19,19 +19,43 @@ log = logging.getLogger(__name__)
def get_roster_file(options): def get_roster_file(options):
if options.get('roster_file'): '''
template = options.get('roster_file') Find respective roster file.
elif 'config_dir' in options.get('__master_opts__', {}):
template = os.path.join(options['__master_opts__']['config_dir'], :param options:
'roster') :return:
elif 'config_dir' in options: '''
template = os.path.join(options['config_dir'], 'roster') template = None
else: # The __disable_custom_roster is always True if Salt SSH Client comes
template = os.path.join(salt.syspaths.CONFIG_DIR, 'roster') # from Salt API. In that case no way to define own 'roster_file', instead
# this file needs to be chosen from already validated rosters
# (see /etc/salt/master config).
if options.get('__disable_custom_roster') and options.get('roster_file'):
roster = options.get('roster_file').strip('/')
for roster_location in options.get('rosters'):
r_file = os.path.join(roster_location, roster)
if os.path.isfile(r_file):
template = r_file
break
del options['roster_file']
if not template:
if options.get('roster_file'):
template = options.get('roster_file')
elif 'config_dir' in options.get('__master_opts__', {}):
template = os.path.join(options['__master_opts__']['config_dir'],
'roster')
elif 'config_dir' in options:
template = os.path.join(options['config_dir'], 'roster')
else:
template = os.path.join(salt.syspaths.CONFIG_DIR, 'roster')
if not os.path.isfile(template): if not os.path.isfile(template):
raise IOError('No roster file found') raise IOError('No roster file found')
if not os.access(template, os.R_OK):
raise IOError('Access denied to roster "{0}"'.format(template))
return template return template

11
salt/states/file.py
View File

@ -113,12 +113,12 @@ There is more documentation about this feature in the
Special files can be managed via the ``mknod`` function. This function will Special files can be managed via the ``mknod`` function. This function will
create and enforce the permissions on a special file. The function supports the create and enforce the permissions on a special file. The function supports the
creation of character devices, block devices, and fifo pipes. The function will creation of character devices, block devices, and FIFO pipes. The function will
create the directory structure up to the special file if it is needed on the create the directory structure up to the special file if it is needed on the
minion. The function will not overwrite or operate on (change major/minor minion. The function will not overwrite or operate on (change major/minor
numbers) existing special files with the exception of user, group, and numbers) existing special files with the exception of user, group, and
permissions. In most cases the creation of some special files require root permissions. In most cases the creation of some special files require root
permisisons on the minion. This would require that the minion to be run as the permissions on the minion. This would require that the minion to be run as the
root user. Here is an example of a character device: root user. Here is an example of a character device:
.. code-block:: yaml .. code-block:: yaml
@ -1352,7 +1352,8 @@ def managed(name,
Default context passed to the template. Default context passed to the template.
backup backup
Overrides the default backup mode for this specific file. Overrides the default backup mode for this specific file. See
:ref:`backup_mode documentation <file-state-backups>` for more details.
show_changes show_changes
Output a unified diff of the old file and the new file. If ``False`` Output a unified diff of the old file and the new file. If ``False``
@ -2469,6 +2470,10 @@ def recurse(name,
Set this to True if empty directories should also be created Set this to True if empty directories should also be created
(default is False) (default is False)
backup
Overrides the default backup mode for all replaced files. See
:ref:`backup_mode documentation <file-state-backups>` for more details.
include_pat include_pat
When copying, include only this pattern from the source. Default When copying, include only this pattern from the source. Default
is glob match; if prefixed with 'E@', then regexp match. is glob match; if prefixed with 'E@', then regexp match.

77
tests/integration/netapi/rest_tornado/test_app.py
View File

@ -48,13 +48,12 @@ class TestSaltAPIHandler(SaltnadoTestCase):
) )
self.assertEqual(response.code, 200) self.assertEqual(response.code, 200)
response_obj = json.loads(response.body) response_obj = json.loads(response.body)
self.assertEqual(response_obj['clients'], self.assertItemsEqual(response_obj['clients'],
['runner', ['runner',
'runner_async', 'runner_async',
'local_async', 'local_async',
'local', 'local']
'local_batch'] )
)
self.assertEqual(response_obj['return'], 'Welcome') self.assertEqual(response_obj['return'], 'Welcome')
def test_post_no_auth(self): def test_post_no_auth(self):
@ -152,68 +151,6 @@ class TestSaltAPIHandler(SaltnadoTestCase):
) )
self.assertEqual(response.code, 400) self.assertEqual(response.code, 400)
# local_batch tests
@skipIf(True, 'to be reenabled when #23623 is merged')
def test_simple_local_batch_post(self):
'''
Basic post against local_batch
'''
low = [{'client': 'local_batch',
'tgt': '*',
'fun': 'test.ping',
}]
response = self.fetch('/',
method='POST',
body=json.dumps(low),
headers={'Content-Type': self.content_type_map['json'],
saltnado.AUTH_TOKEN_HEADER: self.token['token']},
connect_timeout=30,
request_timeout=30,
)
response_obj = json.loads(response.body)
self.assertEqual(response_obj['return'], [{'minion': True, 'sub_minion': True}])
# local_batch tests
@skipIf(True, 'to be reenabled when #23623 is merged')
def test_full_local_batch_post(self):
'''
Test full parallelism of local_batch
'''
low = [{'client': 'local_batch',
'tgt': '*',
'fun': 'test.ping',
'batch': '100%',
}]
response = self.fetch('/',
method='POST',
body=json.dumps(low),
headers={'Content-Type': self.content_type_map['json'],
saltnado.AUTH_TOKEN_HEADER: self.token['token']},
connect_timeout=30,
request_timeout=30,
)
response_obj = json.loads(response.body)
self.assertEqual(response_obj['return'], [{'minion': True, 'sub_minion': True}])
def test_simple_local_batch_post_no_tgt(self):
'''
Local_batch testing with no tgt
'''
low = [{'client': 'local_batch',
'tgt': 'minion_we_dont_have',
'fun': 'test.ping',
}]
response = self.fetch('/',
method='POST',
body=json.dumps(low),
headers={'Content-Type': self.content_type_map['json'],
saltnado.AUTH_TOKEN_HEADER: self.token['token']},
connect_timeout=30,
request_timeout=30,
)
response_obj = json.loads(response.body)
self.assertEqual(response_obj['return'], [{}])
# local_async tests # local_async tests
def test_simple_local_async_post(self): def test_simple_local_async_post(self):
low = [{'client': 'local_async', low = [{'client': 'local_async',
@ -435,7 +372,7 @@ class TestMinionSaltAPIHandler(SaltnadoTestCase):
make sure you get an error make sure you get an error
''' '''
# get a token for this test # get a token for this test
low = [{'client': 'local_batch', low = [{'client': 'local',
'tgt': '*', 'tgt': '*',
'fun': 'test.ping', 'fun': 'test.ping',
}] }]