valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-08 01:18:58 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #36506 from rallytime/merge-develop

Browse Source 
[develop] Merge forward from carbon to develop
This commit is contained in:
Nicole Thomas 2016-09-22 11:38:34 -06:00 committed by GitHub
commit 3e5218daea
17 changed files with 530 additions and 465 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

571
doc/topics/installation/windows.rst
View File

@ -11,7 +11,7 @@ You must connect Windows Salt minions to a Salt master on a supported operating
system to control your Salt Minions.
Many of the standard Salt modules have been ported to work on Windows and many
of the Salt States currently work on Windows, as well.
of the Salt States currently work on Windows as well.
.. _windows-installer:
@ -34,7 +34,7 @@ corresponding md5 file.
requires.
The 64bit installer has been tested on Windows 7 64bit and Windows Server
2008R2 64bit. The 32bit installer has been tested on Windows 2003 Server 32bit.
2008R2 64bit. The 32bit installer has been tested on Windows 2008 Server 32bit.
Please file a bug report on our GitHub repo if issues for other platforms are
found.
@ -54,9 +54,14 @@ minion config with these options.
The final page allows you to select which services to start. Checking the box
next to master or minion will start that service when the installer exits.
The `salt-minion` service will appear in the Windows Service Manager and can be
started and stopped there or with the command line program `sc` like any other
Windows service.
The ``salt-minion`` service will appear in the Windows Service Manager and can
be started and stopped there or with the command line program ``sc`` like any
other Windows service.
.. code-block:: bat
sc start salt-minion
net start salt-minion
If the minion won't start, try installing the Microsoft Visual C++ 2008 x64 SP1
redistributable. Allow all Windows updates to run salt-minion smoothly.
@ -66,7 +71,7 @@ redistributable. Allow all Windows updates to run salt-minion smoothly.
Silent Installer Options
========================
The installer can be run silently by providing the `/S` option at the command
The installer can be run silently by providing the ``/S`` option at the command
line. The installer also accepts the following options for configuring the Salt
Minion silently:
@ -105,11 +110,12 @@ Running the Salt Minion on Windows as an Unprivileged User
==========================================================
Notes:
- These instructions were tested with Windows Server 2008 R2
- They are generalizable to any version of Windows that supports a salt-minion
A. Create the Unprivileged User that the Salt Minion will Run As
----------------------------------------------------------------
Create the Unprivileged User that the Salt Minion will Run As
-------------------------------------------------------------
1. Click ``Start`` > ``Control Panel`` > ``User Accounts``.
@ -137,8 +143,8 @@ A. Create the Unprivileged User that the Salt Minion will Run As
12. Close the ``Change an Account`` window.
B. Add the New User to the Access Control List for the Salt Folder
------------------------------------------------------------------
Add the New User to the Access Control List for the Salt Folder
---------------------------------------------------------------
1. In a File Explorer window, browse to the path where Salt is installed (the default path is ``C:\Salt``).
@ -159,8 +165,8 @@ B. Add the New User to the Access Control List for the Salt Folder
9. Click the ``OK`` button to close the ``Salt Properties`` window.
C. Update the Windows Service User for the ``salt-minion`` Service
------------------------------------------------------------------
Update the Windows Service User for the ``salt-minion`` Service
---------------------------------------------------------------
1. Click ``Start`` > ``Administrative Tools`` > ``Services``.
@ -184,332 +190,238 @@ C. Update the Windows Service User for the ``salt-minion`` Service
10. Right-Click on ``salt-minion`` and select ``Start``.
Setting up a Windows build environment
======================================
.. _building-developing-windows:
This document will explain how to set up a development environment for salt on
Building and Developing on Windows
==================================
This document will explain how to set up a development environment for Salt on
Windows. The development environment allows you to work with the source code to
customize or fix bugs. It will also allow you to build your own installation.
The Easy Way
------------
There are several scripts to automate creating a Windows installer as well as
setting up an environment that facilitates developing and troubleshooting Salt
code. They are located in the ``pkg\windows`` directory in the Salt repo
`(here) <https://github.com/saltstack/salt/tree/develop/pkg/windows>`_.
Scripts:
--------
================= ===========
Script Description
================= ===========
``build_env.ps1`` A PowerShell script that sets up the build environment
``build_pkg.bat`` A batch file that builds a Windows installer based on the
contents of the ``C:\Python27`` directory
``build.bat`` A batch file that fully automates the building of the Windows
installer using the above two scripts
================= ===========
.. note::
The ``build.bat`` and ``build_pkg.bat`` scripts both accept a single
parameter to specify the version of Salt that will be displayed in the
Windows installer. If no version is passed, the version will be determined
using git.
Prerequisite Software
^^^^^^^^^^^^^^^^^^^^^
To do this the easy way you only need to install `Git for Windows <https://github.com/msysgit/msysgit/releases/download/Git-1.9.5-preview20150319/Git-1.9.5-preview20150319.exe/>`_.
Create the Build Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. Clone the `Salt-Windows-Dev <https://github.com/saltstack/salt-windows-dev/>`_
repo from github.
Open a command line and type:
.. code-block:: bat
git clone https://github.com/saltstack/salt-windows-dev
2. Build the Python Environment
Go into the salt-windows-dev directory. Right-click the file named
**dev_env.ps1** and select **Run with PowerShell**
If you get an error, you may need to change the execution policy.
Open a powershell window and type the following:
.. code-block:: powershell
Set-ExecutionPolicy RemoteSigned
This will download and install Python with all the dependencies needed to
develop and build salt.
3. Build the Salt Environment
Right-click on the file named **dev_env_salt.ps1** and select **Run with
Powershell**
This will clone salt into ``C:\Salt-Dev\salt`` and set it to the 2015.5
branch. You could optionally run the command from a powershell window with a
``-Version`` switch to pull a different version. For example:
.. code-block:: powershell
dev_env_salt.ps1 -Version '2014.7'
To view a list of available branches and tags, open a command prompt in your
`C:\Salt-Dev\salt` directory and type:
.. code-block:: bat
git branch -a
git tag -n
The Hard Way
------------
Prerequisite Software
^^^^^^^^^^^^^^^^^^^^^
Install the following software:
1. `Git for Windows <https://github.com/msysgit/msysgit/releases/download/Git-1.9.5-preview20150319/Git-1.9.5-preview20150319.exe/>`_
2. `Nullsoft Installer <http://downloads.sourceforge.net/project/nsis/NSIS%203%20Pre-release/3.0b1/nsis-3.0b1-setup.exe/>`_
Download the Prerequisite zip file for your CPU architecture from the
SaltStack download site:
* `Salt32.zip <http://repo.saltstack.com/windows/dependencies/Salt32.zip/>`_
* `Salt64.zip <http://repo.saltstack.com/windows/dependencies/Salt64.zip/>`_
These files contain all software required to build and develop salt. Unzip the
contents of the file to ``C:\Salt-Dev\temp``.
Create the Build Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. Build the Python Environment
* Install Python:
Browse to the ``C:\Salt-Dev\temp`` directory and find the Python
installation file for your CPU Architecture under the corresponding
subfolder. Double-click the file to install python.
Make sure the following are in your **PATH** environment variable:
.. code-block:: bat
C:\Python27
C:\Python27\Scripts
* Install Pip
Open a command prompt and navigate to ``C:\Salt-Dev\temp``
Run the following command:
.. code-block:: bat
python get-pip.py
* Easy Install compiled binaries.
M2Crypto, PyCrypto, and PyWin32 need to be installed using Easy Install.
Open a command prompt and navigate to ``C:\Salt-Dev\temp\<cpuarch>``.
Run the following commands:
.. code-block:: bat
easy_install -Z <M2Crypto file name>
easy_install -Z <PyCrypto file name>
easy_install -Z <PyWin32 file name>
.. note::
You can type the first part of the file name and then press the tab key
to auto-complete the name of the file.
* Pip Install Additional Prerequisites
All remaining prerequisites need to be pip installed. These prerequisites
are as follow:
* MarkupSafe
* Jinja
* MsgPack
* PSUtil
* PyYAML
* PyZMQ
* WMI
* Requests
* Certifi
Open a command prompt and navigate to ``C:\Salt-Dev\temp``. Run the following
commands:
.. code-block:: bat
pip install <cpuarch>\<MarkupSafe file name>
pip install <Jinja file name>
pip install <cpuarch>\<MsgPack file name>
pip install <cpuarch>\<psutil file name>
pip install <cpuarch>\<PyYAML file name>
pip install <cpuarch>\<pyzmq file name>
pip install <WMI file name>
pip install <requests file name>
pip install <certifi file name>
2. Build the Salt Environment
* Clone Salt
Open a command prompt and navigate to ``C:\Salt-Dev``. Run the following command
to clone salt:
.. code-block:: bat
git clone https://github.com/saltstack/salt
* Checkout Branch
Checkout the branch or tag of salt you want to work on or build. Open a
command prompt and navigate to ``C:\Salt-Dev\salt``. Get a list of
available tags and branches by running the following commands:
.. code-block:: bat
git fetch --all
To view a list of available branches:
git branch -a
To view a list of availabel tags:
git tag -n
Checkout the branch or tag by typing the following command:
.. code-block:: bat
git checkout <branch/tag name>
* Clean the Environment
When switching between branches residual files can be left behind that
will interfere with the functionality of salt. Therefore, after you check
out the branch you want to work on, type the following commands to clean
the salt environment:
.. code-block: bat
git clean -fxd
git reset --hard HEAD
Developing with Salt
====================
There are two ways to develop with salt. You can run salt's setup.py each time
you make a change to source code or you can use the setup tools develop mode.
Configure the Minion
--------------------
Both methods require that the minion configuration be in the ``C:\salt``
directory. Copy the conf and var directories from ``C:\Salt-Dev\salt\pkg\
windows\buildenv`` to ``C:\salt``. Now go into the ``C:\salt\conf`` directory
and edit the file name ``minion`` (no extension). You need to configure the
master and id parameters in this file. Edit the following lines:
.. code-block:: bat
master: <ip or name of your master>
id: <name of your minion>
Setup.py Method
---------------
Go into the ``C:\Salt-Dev\salt`` directory from a cmd prompt and type:
.. code-block:: bat
python setup.py install --force
This will install python into your python installation at ``C:\Python27``.
Everytime you make an edit to your source code, you'll have to stop the minion,
run the setup, and start the minion.
To start the salt-minion go into ``C:\Python27\Scripts`` from a cmd prompt and
type:
.. code-block:: bat
salt-minion
For debug mode type:
.. code-block:: bat
salt-minion -l debug
To stop the minion press Ctrl+C.
Setup Tools Develop Mode (Preferred Method)
-------------------------------------------
To use the Setup Tools Develop Mode go into ``C:\Salt-Dev\salt`` from a cmd
prompt and type:
.. code-block:: bat
pip install -e .
This will install pointers to your source code that resides at
``C:\Salt-Dev\salt``. When you edit your source code you only have to restart
the minion.
Build the windows installer
===========================
This is the method of building the installer as of version 2014.7.4.
Clean the Environment
---------------------
Make sure you don't have any leftover salt files from previous versions of salt
in your Python directory.
1. Remove all files that start with salt in the ``C:\Python27\Scripts``
directory
The only prerequisite is `Git for Windows <https://git-scm.com/download/win/>`_.
2. Remove all files and directories that start with salt in the
``C:\Python27\Lib\site-packages`` directory
.. _create-build-environment:
Install Salt
------------
Install salt using salt's setup.py. From the ``C:\Salt-Dev\salt`` directory
type the following command:
Create a Build Environment
--------------------------
1. Working Directory
^^^^^^^^^^^^^^^^^^^^
Create a ``Salt-Dev`` directory on the root of ``C:``. This will be our working
directory. Navigate to ``Salt-Dev`` and clone the
`Salt <https://github.com/saltstack/salt/>`_ repo from GitHub.
Open a command line and type:
.. code-block:: bat
python setup.py install --force
cd \
md Salt-Dev
cd Salt-Dev
git clone https://github.com/saltstack/salt
Build the Installer
-------------------
From cmd prompt go into the ``C:\Salt-Dev\salt\pkg\windows`` directory. Type
the following command for the branch or tag of salt you're building:
Go into the ``salt`` directory and checkout the version of salt to work with
(2016.3 or higher).
.. code-block:: bat
BuildSalt.bat <branch or tag>
cd salt
git checkout 2016.3
This will copy python with salt installed to the ``buildenv\bin`` directory,
make it portable, and then create the windows installer . The .exe for the
windows installer will be placed in the ``installer`` directory.
2. Setup the Python Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Navigate to the ``pkg\windows`` directory and execute the **build_env.ps1**
PowerShell script.
.. code-block:: bat
cd pkg\windows
powershell -file build_env.ps1
.. note::
You can also do this from Explorer by navigating to the ``pkg\windows``
directory, right clicking the **build_env.ps1** powershell script and
selecting **Run with PowerShell**
This will download and install Python with all the dependencies needed to
develop and build Salt.
.. note::
If you get an error or the script fails to run you may need to change the
execution policy. Open a powershell window and type the following command:
.. code-block:: powershell
Set-ExecutionPolicy RemoteSigned
3. Salt in Editable Mode
^^^^^^^^^^^^^^^^^^^^^^^^
Editable mode allows you to more easily modify and test the source code. For
more information see the `Pip documentation
<https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs>`_.
Navigate to the root of the ``salt`` directory and install Salt in editable mode
with ``pip``
.. code-block:: bat
cd \Salt-Dev\salt
pip install -e .
.. note::
The ``.`` is important
.. note::
If ``pip`` is not recognized, you may need to restart your shell to get the
updated path
4. Setup Salt Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Salt requires a minion configuration file and a few other directories. The
default config file is named ``minion`` located in ``C:\salt\conf``. The
easiest way to set this up is to copy the contents of the
``salt\pkg\windows\buildenv`` directory to ``C:\salt``.
.. code-block:: bat
cd \
md salt
xcopy /s /e \Salt-Dev\salt\pkg\windows\buildenv\* \salt\
Now go into the ``C:\salt\conf`` directory and edit the file name ``minion`` (no
extension). You need to configure the master and id parameters in this file.
Edit the following lines:
.. code-block:: bat
master: <ip or name of your master>
id: <name of your minion>
.. _create-windows-installer:
Create a Windows Installer
==========================
To create a Windows installer, follow steps 1 and 2 from
:ref:`Create a Build Environment <create-build-environment>` above. Then proceed
to 3 below:
3. Install Salt
---------------
To create the installer for Window we install Salt using Python instead of pip.
Navigate to the root ``salt`` directory and install Salt.
.. code-block:: bat
cd \Salt-Dev\salt
python setup.py install
4. Create the Windows Installer
-------------------------------
Navigate to the ``pkg\windows`` directory and run the ``build_pkg.bat``
with the build version (2016.3) script.
.. code-block:: bat
cd pkg\windows
build_pkg.bat 2016.3
.. note::
If no version is passed, the ``build_pkg.bat`` will guess the version number
using git.
.. _create-windows-installer-easy:
Creating a Windows Installer: Alternate Method (Easier)
=======================================================
Clone the `Salt <https://github.com/saltstack/salt/>`_ repo from GitHub into the
directory of your choice. We're going to use ``Salt-Dev``.
.. code-block:: bat
cd \
md Salt-Dev
cd Salt-Dev
git clone https://github.com/saltstack/salt
Go into the ``salt`` directory and checkout the version of Salt you want to
build.
.. code-block:: bat
cd salt
git checkout 2016.3
Then navigate to ``pkg\windows`` and run the ``build.bat`` script with the
version you're building.
.. code-block:: bat
cd pkg\windows
build.bat 2016.3
This will install everything needed to build a Windows installer for Salt. The
binary will be in the ``salt\pkg\windows\installer`` directory.
.. _test-salt-minion:
Testing the Salt minion
=======================
1. Create the directory C:\\salt (if it doesn't exist already)
1. Create the directory ``C:\salt`` (if it doesn't exist already)
2. Copy the example ``conf`` and ``var`` directories from
``pkg/windows/buildenv/`` into C:\\salt
2. Copy the example ``conf`` and ``var`` directories from
``pkg\windows\buildenv`` into ``C:\salt``
3. Edit C:\\salt\\conf\\minion
3. Edit ``C:\salt\conf\minion``
.. code-block:: bash
master: ipaddress or hostname of your salt-master
4. Start the salt-minion
4. Start the salt-minion
.. code-block:: bash
cd C:\Python27\Scripts
python salt-minion
python salt-minion -l debug
5. On the salt-master accept the new minion's key
5. On the salt-master accept the new minion's key
.. code-block:: bash
@ -518,7 +430,7 @@ Testing the Salt minion
This accepts all unaccepted keys. If you're concerned about security just
accept the key for this specific minion.
6. Test that your minion is responding
6. Test that your minion is responding
On the salt-master run:
@ -526,52 +438,15 @@ Testing the Salt minion
sudo salt '*' test.ping
You should get the following response: ``{'your minion hostname': True}``
Single command bootstrap script
===============================
On a 64 bit Windows host the following script makes an unattended install of
salt, including all dependencies:
.. admonition:: Not up to date.
This script is not up to date. Please use the installer found above
.. code-block:: powershell
# (All in one line.)
"PowerShell (New-Object System.Net.WebClient).DownloadFile('http://csa-net.dk/salt/bootstrap64.bat','C:\bootstrap.bat');(New-Object -com Shell.Application).ShellExecute('C:\bootstrap.bat');"
You can execute the above command remotely from a Linux host using winexe:
.. code-block:: bash
winexe -U "administrator" //fqdn "PowerShell (New-Object ......);"
For more info check `http://csa-net.dk/salt`_
Packages management under Windows 2003
Packages Management Under Windows 2003
======================================
On windows Server 2003, you need to install optional component "wmi windows
installer provider" to have full list of installed packages. If you don't have
Windows Server 2003 and Windows XP have both reached End of Support. Though Salt
is not officially supported on operating systems that are EoL, some
functionality may continue to work.
On Windows Server 2003, you need to install optional component "WMI Windows
Installer Provider" to get a full list of installed packages. If you don't have
this, salt-minion can't report some installed software.
.. _http://csa-net.dk/salt: http://csa-net.dk/salt
.. _msysgit: http://code.google.com/p/msysgit/downloads/list?can=3
.. _Python 2.7: http://www.python.org/downloads
.. _ez_setup.py: https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
.. _OpenSSL for Windows: http://slproweb.com/products/Win32OpenSSL.html
.. _M2Crypto: http://chandlerproject.org/Projects/MeTooCrypto
.. _pycrypto: http://www.voidspace.org.uk/python/modules.shtml#pycrypto
.. _pywin32: http://sourceforge.net/projects/pywin32/files/pywin32
.. _Cython: http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython
.. _jinja2: http://www.lfd.uci.edu/~gohlke/pythonlibs/#jinja2
.. _msgpack: http://www.lfd.uci.edu/~gohlke/pythonlibs/#msgpack

54
salt/engines/sqs_events.py
View File

@ -62,6 +62,15 @@ It's also possible to specify key, keyid and region via a profile:
key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
region: us-east-1
Additionally you can define cross account sqs:
.. code-block:: yaml
engines:
- sqs_events:
queue: prod
owner_acct_id: 111111111111
'''
# Import python libs
@ -122,9 +131,24 @@ def _get_sqs_conn(profile, region=None, key=None, keyid=None):
return conn
def start(queue, profile=None, tag='salt/engine/sqs'):
def _process_queue(q, q_name, fire_master, tag='salt/engine/sqs', owner_acct_id=None, message_format=None):
if not q:
log.warning('failure connecting to queue: {0}, '
'waiting 10 seconds.'.format(':'.join([_f for _f in (str(owner_acct_id), q_name) if _f])))
time.sleep(10)
else:
msgs = q.get_messages(wait_time_seconds=20)
for msg in msgs:
if message_format == "json":
fire_master(tag=tag, data={'message': json.loads(msg.get_body())})
else:
fire_master(tag=tag, data={'message': msg.get_body()})
msg.delete()
def start(queue, profile=None, tag='salt/engine/sqs', owner_acct_id=None):
'''
Listen to events and write them to a log file
Listen to sqs and fire message on event bus
'''
if __opts__.get('__role') == 'master':
fire_master = salt.utils.event.get_master_event(
@ -132,31 +156,13 @@ def start(queue, profile=None, tag='salt/engine/sqs'):
__opts__['sock_dir'],
listen=False).fire_event
else:
fire_master = None
fire_master = __salt__['event.send']
message_format = __opts__.get('sqs.message_format', None)
def fire(tag, msg):
if fire_master:
fire_master(msg, tag)
else:
__salt__['event.send'](tag, msg)
sqs = _get_sqs_conn(profile)
q = sqs.get_queue(queue)
q = None
while True:
if not q:
log.warning('failure connecting to queue: {0}, '
'waiting 10 seconds.'.format(queue))
time.sleep(10)
q = sqs.get_queue(queue)
if not q:
continue
msgs = q.get_messages(wait_time_seconds=20)
for msg in msgs:
if message_format == "json":
fire(tag, {'message': json.loads(msg.get_body())})
else:
fire(tag, {'message': msg.get_body()})
msg.delete()
q = sqs.get_queue(queue, owner_acct_id=owner_acct_id)
_process_queue(q, queue, fire_master, tag=tag, owner_acct_id=owner_acct_id, message_format=message_format)

60
salt/grains/core.py
View File

@ -37,6 +37,9 @@ import salt.utils
import salt.utils.network
import salt.utils.dns
if salt.utils.is_windows():
import salt.utils.win_osinfo
# Solve the Chicken and egg problem where grains need to run before any
# of the modules are loaded and are generally available for any usage.
import salt.modules.cmdmod
@ -872,6 +875,10 @@ def _windows_platform_data():
Use the platform module for as much as we can.
'''
# Provides:
# kernelrelease
# osversion
# osrelease
# osservicepack
# osmanufacturer
# manufacturer
# productname
@ -882,6 +889,7 @@ def _windows_platform_data():
# windowsdomain
# motherboard.productname
# motherboard.serialnumber
# virtual
if not HAS_WMI:
return {}
@ -890,7 +898,7 @@ def _windows_platform_data():
wmi_c = wmi.WMI()
# http://msdn.microsoft.com/en-us/library/windows/desktop/aa394102%28v=vs.85%29.aspx
systeminfo = wmi_c.Win32_ComputerSystem()[0]
# http://msdn.microsoft.com/en-us/library/windows/desktop/aa394239%28v=vs.85%29.aspx
# https://msdn.microsoft.com/en-us/library/aa394239(v=vs.85).aspx
osinfo = wmi_c.Win32_OperatingSystem()[0]
# http://msdn.microsoft.com/en-us/library/windows/desktop/aa394077(v=vs.85).aspx
biosinfo = wmi_c.Win32_BIOS()[0]
@ -898,9 +906,8 @@ def _windows_platform_data():
timeinfo = wmi_c.Win32_TimeZone()[0]
# http://msdn.microsoft.com/en-us/library/windows/desktop/aa394072(v=vs.85).aspx
motherboard = {}
motherboard['product'] = None
motherboard['serial'] = None
motherboard = {'product': None,
'serial': None}
try:
motherboardinfo = wmi_c.Win32_BaseBoard()[0]
motherboard['product'] = motherboardinfo.Product
@ -908,13 +915,40 @@ def _windows_platform_data():
except IndexError:
log.debug('Motherboard info not available on this system')
# the name of the OS comes with a bunch of other data about the install
# location. For example:
# 'Microsoft Windows Server 2008 R2 Standard |C:\\Windows|\\Device\\Harddisk0\\Partition2'
(osfullname, _) = osinfo.Name.split('|', 1)
osfullname = osfullname.strip()
os_release = platform.release()
info = salt.utils.win_osinfo.get_os_version_info()
# Starting with Python 2.7.12 and 3.5.2 the `platform.uname()` function
# started reporting the Desktop version instead of the Server version on
# Server versions of Windows, so we need to look those up
# Check for Python >=2.7.12 or >=3.5.2
ver = pythonversion()['pythonversion']
if ((six.PY2 and
salt.utils.compare_versions(ver, '>=', [2, 7, 12, 'final', 0]))
or
(six.PY3 and
salt.utils.compare_versions(ver, '>=', [3, 5, 2, 'final', 0]))):
# (Product Type 1 is Desktop, Everything else is Server)
if info['ProductType'] > 1:
server = {'Vista': '2008Server',
'7': '2008ServerR2',
'8': '2012Server',
'8.1': '2012ServerR2',
'10': '2016Server'}
os_release = server.get(os_release,
'Grain not found. Update lookup table '
'in the `_windows_platform_data` '
'function in `grains\\core.py`')
service_pack = None
if info['ServicePackMajor'] > 0:
service_pack = ''.join(['SP', str(info['ServicePackMajor'])])
grains = {
'kernelrelease': osinfo.Version,
'osversion': osinfo.Version,
'osrelease': os_release,
'osservicepack': service_pack,
'osmanufacturer': osinfo.Manufacturer,
'manufacturer': systeminfo.Manufacturer,
'productname': systeminfo.Model,
@ -922,12 +956,12 @@ def _windows_platform_data():
# 'PhoenixBIOS 4.0 Release 6.0 '
'biosversion': biosinfo.Name.strip(),
'serialnumber': biosinfo.SerialNumber,
'osfullname': osfullname,
'osfullname': osinfo.Caption,
'timezone': timeinfo.Description,
'windowsdomain': systeminfo.Domain,
'motherboard': {
'productname': motherboard['product'],
'serialnumber': motherboard['serial']
'serialnumber': motherboard['serial'],
}
}
@ -1159,10 +1193,6 @@ def os_data():
grains['os_family'] = 'proxy'
grains['osfullname'] = 'proxy'
elif salt.utils.is_windows():
with salt.utils.winapi.Com():
wmi_c = wmi.WMI()
grains['osrelease'] = grains['kernelrelease']
grains['osversion'] = grains['kernelrelease'] = wmi_c.Win32_OperatingSystem()[0].Version
grains['os'] = 'Windows'
grains['os_family'] = 'Windows'
grains.update(_memdata(grains))

6
salt/modules/file.py
View File

@ -4061,8 +4061,10 @@ def check_file_meta(
if contents is not None:
# Write a tempfile with the static contents
tmp = salt.utils.mkstemp(text=True)
with salt.utils.fopen(tmp, 'wb') as tmp_:
tmp_.write(salt.utils.to_bytes(str(contents)))
if salt.utils.is_windows():
contents = os.linesep.join(contents.splitlines())
with salt.utils.fopen(tmp, 'w') as tmp_:
tmp_.write(str(contents))
# Compare the static contents with the named file
with salt.utils.fopen(tmp, 'r') as src:
slines = src.readlines()

31
salt/states/git.py
View File

@ -140,8 +140,11 @@ def _strip_exc(exc):
return re.sub(r'^Command [\'"].+[\'"] failed: ', '', exc.strerror)
def _uptodate(ret, target, comments=None):
def _uptodate(ret, target, comments=None, local_changes=False):
ret['comment'] = 'Repository {0} is up-to-date'.format(target)
if local_changes:
ret['comment'] += ', but with local changes. Set \'force_reset\' to ' \
'True to purge local changes.'
if comments:
# Shouldn't be making any changes if the repo was up to date, but
# report on them so we are alerted to potential problems with our
@ -789,6 +792,30 @@ def latest(name,
)
local_changes = False
if local_changes and revs_match:
if force_reset:
msg = (
'{0} is up-to-date, but with local changes. Since '
'\'force_reset\' is enabled, these local changes '
'would be reset.'.format(target)
)
if __opts__['test']:
ret['changes']['forced update'] = True
if comments:
msg += _format_comments(comments)
return _neutral_test(ret, msg)
log.debug(msg.replace('would', 'will'))
else:
log.debug(
'%s up-to-date, but with local changes. Since '
'\'force_reset\' is disabled, no changes will be '
'made.', target
)
return _uptodate(ret,
target,
_format_comments(comments),
local_changes)
if remote_rev_type == 'sha1' \
and base_rev is not None \
and base_rev.startswith(remote_rev):
@ -882,7 +909,7 @@ def latest(name,
has_remote_rev = True
# If fast_forward is not boolean, then we don't know if this will
# be a fast forward or not, because a fetch is requirde.
# be a fast forward or not, because a fetch is required.
fast_forward = None if not local_changes else False
if has_remote_rev:

5
salt/utils/mac_utils.py
View File

@ -16,6 +16,7 @@ import os
import salt.utils
import salt.utils.timed_subprocess
import salt.grains.extra
from salt.ext import six
from salt.exceptions import CommandExecutionError, SaltInvocationError,\
TimedProcTimeoutError
@ -50,6 +51,10 @@ def _run_all(cmd):
if not isinstance(cmd, list):
cmd = salt.utils.shlex_split(cmd, posix=False)
for idx, item in enumerate(cmd):
if not isinstance(cmd[idx], six.string_types):
cmd[idx] = str(cmd[idx])
cmd = ' '.join(cmd)
run_env = os.environ.copy()

4
salt/utils/vmware.py
View File

@ -601,7 +601,7 @@ def get_content(service_instance, obj_type, property_list=None,
traversal_spec = vmodl.query.PropertyCollector.TraversalSpec(
name='traverseEntities',
path='view',
skip=True,
skip=False,
type=vim.view.ContainerView
)
@ -615,7 +615,7 @@ def get_content(service_instance, obj_type, property_list=None,
# Create object spec to navigate content
obj_spec = vmodl.query.PropertyCollector.ObjectSpec(
obj=obj_ref,
skip=False,
skip=True if not local_properties else False,
selectSet=[traversal_spec] if not local_properties else None
)

56
salt/utils/win_osinfo.py Normal file
View File

@ -0,0 +1,56 @@
# -*- coding: utf-8 -*-
'''
Get Versioning information from Windows
'''
# http://stackoverflow.com/questions/32300004/python-ctypes-getting-0-with-getversionex-function
from __future__ import absolute_import
import ctypes
from ctypes.wintypes import BYTE, WORD, DWORD, WCHAR
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
class OSVERSIONINFO(ctypes.Structure):
_fields_ = (('dwOSVersionInfoSize', DWORD),
('dwMajorVersion', DWORD),
('dwMinorVersion', DWORD),
('dwBuildNumber', DWORD),
('dwPlatformId', DWORD),
('szCSDVersion', WCHAR * 128))
def __init__(self, *args, **kwds):
super(OSVERSIONINFO, self).__init__(*args, **kwds)
self.dwOSVersionInfoSize = ctypes.sizeof(self)
kernel32.GetVersionExW(ctypes.byref(self))
class OSVERSIONINFOEX(OSVERSIONINFO):
_fields_ = (('wServicePackMajor', WORD),
('wServicePackMinor', WORD),
('wSuiteMask', WORD),
('wProductType', BYTE),
('wReserved', BYTE))
def errcheck_bool(result, func, args):
if not result:
raise ctypes.WinError(ctypes.get_last_error())
return args
kernel32.GetVersionExW.errcheck = errcheck_bool
kernel32.GetVersionExW.argtypes = (ctypes.POINTER(OSVERSIONINFO),)
def get_os_version_info():
info = OSVERSIONINFOEX()
ret = {'MajorVersion': info.dwMajorVersion,
'MinorVersion': info.dwMinorVersion,
'BuildNumber': info.dwBuildNumber,
'PlatformID': info.dwPlatformId,
'ServicePackMajor': info.wServicePackMajor,
'ServicePackMinor': info.wServicePackMinor,
'SuiteMask': info.wSuiteMask,
'ProductType': info.wProductType}
return ret

4
tests/integration/modules/mac_service.py
View File

@ -65,10 +65,10 @@ class MacServiceModuleTest(integration.ModuleCase):
# Expected Functionality
self.assertTrue(
self.run_function('service.launchctl',
['error', 'bootstrap', '64']))
['error', 'bootstrap', 64]))
self.assertEqual(
self.run_function('service.launchctl',
['error', 'bootstrap', '64'],
['error', 'bootstrap', 64],
return_stdout=True),
'64: unknown error code')

11
tests/integration/states/git.py
View File

@ -175,16 +175,21 @@ class GitTest(integration.ModuleCase, integration.SaltReturnAssertsMixIn):
# Make sure that we now have uncommitted changes
self.assertTrue(self.run_function('git.diff', [name, 'HEAD']))
# Re-run state with force_reset=False, this should fail
# Re-run state with force_reset=False
ret = self.run_state(
'git.latest',
name='https://{0}/saltstack/salt-test-repo.git'.format(self.__domain),
target=name,
force_reset=False
)
self.assertSaltFalseReturn(ret)
self.assertSaltTrueReturn(ret)
self.assertEqual(
ret[next(iter(ret))]['comment'],
('Repository {0} is up-to-date, but with local changes. Set '
'\'force_reset\' to True to purge local changes.'.format(name))
)
# Now run the state with force_reset=True, this should succeed
# Now run the state with force_reset=True
ret = self.run_state(
'git.latest',
name='https://{0}/saltstack/salt-test-repo.git'.format(self.__domain),

76
tests/integration/states/keystone.py
View File

@ -40,7 +40,7 @@ class KeystoneStateTest(integration.ModuleCase,
token = 'administrator'
@destructiveTest
def test_aaa_setup_keystone_endpoint(self):
def setUp(self):
ret = self.run_state('keystone.service_present',
name='keystone',
description='OpenStack Identity',
@ -105,58 +105,14 @@ class KeystoneStateTest(integration.ModuleCase,
connection_token=self.token)
self.assertTrue(ret['keystone_|-demo_|-demo_|-user_present']['result'])
@destructiveTest
def test_zzz_teardown_keystone_endpoint(self):
ret = self.run_state('keystone.user_absent',
name='admin',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-admin_|-admin_|-user_absent']['result'])
ret = self.run_state('keystone.user_absent',
name='demo',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-demo_|-demo_|-user_absent']['result'])
ret = self.run_state('keystone.role_absent',
name='admin',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-admin_|-admin_|-role_absent']['result'])
ret = self.run_state('keystone.role_absent',
name='user',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-user_|-user_|-role_absent']['result'])
ret = self.run_state('keystone.tenant_absent',
name='admin',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-admin_|-admin_|-tenant_absent']['result'])
ret = self.run_state('keystone.tenant_absent',
name='demo',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-demo_|-demo_|-tenant_absent']['result'])
ret = self.run_state('keystone.service_absent',
name='keystone',
connection_endpoint=self.endpoint,
connection_token=self.token)
self.assertTrue(ret['keystone_|-keystone_|-keystone_|-service_absent']['result'])
@destructiveTest
def test_keystone_v2(self):
ret = self.run_state('keystone.service_present',
name='nova',
name='testv2',
description='Nova Service',
service_type='compute',
profile='adminv2')
self.assertTrue(ret['keystone_|-nova_|-nova_|-service_present']['result'])
self.assertTrue(ret['keystone_|-testv2_|-testv2_|-service_present']['result'])
ret = self.run_state('keystone.endpoint_present',
name='nova',
@ -188,41 +144,46 @@ class KeystoneStateTest(integration.ModuleCase,
profile='adminv2')
self.assertTrue(ret['keystone_|-test_|-test_|-user_present']['result'])
ret = self.run_state('keystone.service_absent',
name='testv2',
profile='adminv2')
self.assertTrue(ret['keystone_|-testv2_|-testv2_|-service_absent']['result'])
@destructiveTest
def test_keystone_v3(self):
ret = self.run_state('keystone.service_present',
name='glance',
name='testv3',
description='Image Service',
service_type='image',
profile='adminv3')
self.assertTrue(ret['keystone_|-glance_|-glance_|-service_present']['result'])
self.assertTrue(ret['keystone_|-testv3_|-testv3_|-service_present']['result'])
ret = self.run_state('keystone.endpoint_present',
name='glance',
name='testv3',
description='Glance Service',
interface='public',
url='http://localhost:9292',
region='RegionOne',
profile='adminv3')
self.assertTrue(ret['keystone_|-glance_|-glance_|-endpoint_present']['result'])
self.assertTrue(ret['keystone_|-testv3_|-testv3_|-endpoint_present']['result'])
ret = self.run_state('keystone.endpoint_present',
name='glance',
name='testv3',
description='Glance Service',
interface='internal',
url='http://localhost:9292',
region='RegionOne',
profile='adminv3')
self.assertTrue(ret['keystone_|-glance_|-glance_|-endpoint_present']['result'])
self.assertTrue(ret['keystone_|-testv3_|-testv3_|-endpoint_present']['result'])
ret = self.run_state('keystone.endpoint_present',
name='glance',
name='testv3',
description='Glance Service',
interface='admin',
url='http://localhost:9292',
region='RegionOne',
profile='adminv3')
self.assertTrue(ret['keystone_|-glance_|-glance_|-endpoint_present']['result'])
self.assertTrue(ret['keystone_|-testv3_|-testv3_|-endpoint_present']['result'])
ret = self.run_state('keystone.project_present',
name='testv3',
@ -244,6 +205,11 @@ class KeystoneStateTest(integration.ModuleCase,
profile='adminv3')
self.assertTrue(ret['keystone_|-testv3_|-testv3_|-user_present']['result'])
ret = self.run_state('keystone.service_absent',
name='testv3',
profile='adminv3')
self.assertTrue(ret['keystone_|-testv3_|-testv3_|-service_absent']['result'])
if __name__ == '__main__':
from integration import run_tests

5
tests/unit/cloud/clouds/dimensiondata_test.py
View File

@ -8,6 +8,8 @@
# Import Python libs
from __future__ import absolute_import
from distutils.version import LooseVersion
import mock
try:
import libcloud.security
@ -150,7 +152,8 @@ class DimensionDataTestCase(ExtendedTestCase):
with patch('salt.config.check_driver_dependencies', return_value=True) as p:
get_deps = dimensiondata.get_dependencies()
self.assertEqual(get_deps, True)
p.assert_called_once()
if LooseVersion(mock.__version__) >= LooseVersion('2.0.0'):
p.assert_called_once()
def test_provider_matches(self):
"""

5
tests/unit/cloud/clouds/gce_test.py
View File

@ -8,6 +8,8 @@
# Import Python libs
from __future__ import absolute_import
from distutils.version import LooseVersion
import mock
try:
import libcloud.security
@ -95,7 +97,8 @@ class GCETestCase(TestCase):
with patch('salt.config.check_driver_dependencies', return_value=True) as p:
get_deps = gce.get_dependencies()
self.assertEqual(get_deps, True)
p.assert_called_once()
if LooseVersion(mock.__version__) >= LooseVersion('2.0.0'):
p.assert_called_once()
def test_provider_matches(self):
"""

1
tests/unit/engines/__init__.py Normal file
View File

@ -0,0 +1 @@
# -*- coding: utf-8 -*-

84
tests/unit/engines/sqs_events_test.py Normal file
View File

@ -0,0 +1,84 @@
# -*- coding: utf-8 -*-
'''
unit tests for the sqs_events engine
'''
# Import Python libs
from __future__ import absolute_import
# Import Salt Testing Libs
from salttesting import skipIf, TestCase
from salttesting.mock import (
NO_MOCK,
NO_MOCK_REASON,
MagicMock,
patch)
from salttesting.helpers import ensure_in_syspath
ensure_in_syspath('../../')
# Import Salt Libs
from salt.engines import sqs_events
sqs_events.__salt__ = {}
sqs_events.__opts__ = {}
@skipIf(NO_MOCK, NO_MOCK_REASON)
@patch('salt.engines.sqs_events.boto.sqs')
class EngineSqsEventTestCase(TestCase):
'''
Test cases for salt.engine.sqs_events
'''
def sample_msg(self):
fake_msg = MagicMock()
fake_msg.get_body.return_value = "This is a test message"
fake_msg.delete.return_value = True
return fake_msg
# 'present' function tests: 1
@patch('salt.engines.sqs_events.log')
@patch('time.sleep', return_value=None)
def test_no_queue_present(self, mock_sleep, mock_logging, mock_sqs):
'''
Test to ensure the SQS engine logs a warning when queue not present
'''
q = None
q_name = 'mysqs'
mock_fire = MagicMock(return_value=True)
sqs_events._process_queue(q, q_name, mock_fire)
self.assertTrue(mock_logging.warning.called)
self.assertFalse(mock_sqs.queue.Queue().get_messages.called)
def test_minion_message_fires(self, mock_sqs):
'''
Test SQS engine correctly gets and fires messages on minion
'''
msgs = [self.sample_msg(), self.sample_msg()]
mock_sqs.queue.Queue().get_messages.return_value = msgs
q = mock_sqs.queue.Queue()
q_name = 'mysqs'
mock_event = MagicMock(return_value=True)
mock_fire = MagicMock(return_value=True)
with patch.dict(sqs_events.__salt__, {'event.send': mock_event}):
sqs_events._process_queue(q, q_name, mock_fire)
self.assertTrue(mock_sqs.queue.Queue().get_messages.called)
self.assertTrue(all(x.delete.called for x in msgs))
def test_master_message_fires(self, mock_sqs):
'''
Test SQS engine correctly gets and fires messages on master
'''
msgs = [self.sample_msg(), self.sample_msg()]
mock_sqs.queue.Queue().get_messages.return_value = msgs
q = mock_sqs.queue.Queue()
q_name = 'mysqs'
mock_fire = MagicMock(return_value=True)
sqs_events._process_queue(q, q_name, mock_fire)
self.assertTrue(mock_sqs.queue.Queue().get_messages.called, len(msgs))
self.assertTrue(mock_fire.called, len(msgs))
if __name__ == '__main__':
from integration import run_tests
run_tests(EngineSqsEventTestCase, needs_daemon=False)

12
tests/unit/modules/pacman_test.py
View File

@ -40,16 +40,17 @@ class PacmanTestCase(TestCase):
cmdmock = MagicMock(return_value='A 1.0\nB 2.0')
sortmock = MagicMock()
stringifymock = MagicMock()
mock_ret = {'A': ['1.0'], 'B': ['2.0']}
with patch.dict(pacman.__salt__, {
'cmd.run': cmdmock,
'pkg_resource.add_pkg': lambda pkgs, name, version: pkgs.setdefault(name, []).append(version),
'pkg_resource.sort_pkglist': sortmock,
'pkg_resource.stringify': stringifymock
}):
self.assertDictEqual(pacman.list_pkgs(), {'A': ['1.0'], 'B': ['2.0']})
self.assertDictEqual(pacman.list_pkgs(), mock_ret)
sortmock.assert_called_once()
stringifymock.assert_called_once()
sortmock.assert_called_with(mock_ret)
stringifymock.assert_called_with(mock_ret)
def test_list_pkgs_as_list(self):
'''
@ -58,15 +59,16 @@ class PacmanTestCase(TestCase):
cmdmock = MagicMock(return_value='A 1.0\nB 2.0')
sortmock = MagicMock()
stringifymock = MagicMock()
mock_ret = {'A': ['1.0'], 'B': ['2.0']}
with patch.dict(pacman.__salt__, {
'cmd.run': cmdmock,
'pkg_resource.add_pkg': lambda pkgs, name, version: pkgs.setdefault(name, []).append(version),
'pkg_resource.sort_pkglist': sortmock,
'pkg_resource.stringify': stringifymock
}):
self.assertDictEqual(pacman.list_pkgs(True), {'A': ['1.0'], 'B': ['2.0']})
self.assertDictEqual(pacman.list_pkgs(True), mock_ret)
sortmock.assert_called_once()
sortmock.assert_called_with(mock_ret)
stringifymock.assert_not_called()
def test_group_list(self):

10
tests/unit/utils/vmware_test/common_test.py
View File

@ -403,11 +403,11 @@ class GetContentTestCase(TestCase):
self.create_container_view_mock.assert_called_once_with(
self.root_folder_mock, [self.obj_type_mock], True)
self.traversal_spec_mock.assert_called_once_with(
name='traverseEntities', path='view', skip=True,
name='traverseEntities', path='view', skip=False,
type=vim.view.ContainerView)
self.obj_spec_mock.assert_called_once_with(
obj=self.container_view_mock,
skip=False,
skip=True,
selectSet=[self.traversal_spec_ret_mock])
# check destroy is called
self.assertEqual(self.destroy_mock.call_count, 1)
@ -423,7 +423,7 @@ class GetContentTestCase(TestCase):
traversal_spec=traversal_spec_obj_mock)
self.obj_spec_mock.assert_called_once_with(
obj=self.root_folder_mock,
skip=False,
skip=True,
selectSet=[traversal_spec_obj_mock])
# Check local traversal methods are not called
self.assertEqual(self.create_container_view_mock.call_count, 0)
@ -441,12 +441,12 @@ class GetContentTestCase(TestCase):
self.si_mock,
self.obj_type_mock)
self.traversal_spec_mock.assert_called_once_with(
name='traverseEntities', path='view', skip=True,
name='traverseEntities', path='view', skip=False,
type=vim.view.ContainerView)
self.property_spec_mock.assert_called_once_with(
type=self.obj_type_mock, all=True, pathSet=None)
self.obj_spec_mock.assert_called_once_with(
obj=self.container_view_mock, skip=False,
obj=self.container_view_mock, skip=True,
selectSet=[self.traversal_spec_ret_mock])
self.retrieve_contents_mock.assert_called_once_with(
[self.filter_spec_ret_mock])