Merge remote-tracking branch 'upstream/develop' into sam_raet_21

This commit is contained in:
Samuel M Smith 2014-03-12 16:55:31 -06:00
commit d6c833d5af
119 changed files with 1693 additions and 1075 deletions

View File

@ -216,6 +216,34 @@ Set up an initial profile at ``/etc/salt/cloud.profiles``:
- { size: 10, device: /dev/sdf }
- { size: 10, device: /dev/sdg, type: io1, iops: 1000 }
- { size: 10, device: /dev/sdh, type: io1, iops: 1000 }
# optionally add tags to profile:
tag: {'Environment': 'production', 'Role': 'database'}
# force grains to sync after install
sync_after_install: grains
base_ec2_vpc:
provider: my-ec2-southeast-public-ips
image: ami-a73264ce
size: m1.xlarge
ssh_username: ec2-user
script: /etc/salt/cloud.deploy.d/user_data.sh
network_interfaces:
- DeviceIndex: 0
PrivateIpAddresses:
- Primary: True
#auto assign public ip (not EIP)
AssociatePublicIpAddress: True
SubnetId: subnet-813d4bbf
SecurityGroupId:
- sg-750af413
volumes:
- { size: 10, device: /dev/sdf }
- { size: 10, device: /dev/sdg, type: io1, iops: 1000 }
- { size: 10, device: /dev/sdh, type: io1, iops: 1000 }
del_root_vol_on_destroy: True
del_all_vol_on_destroy: True
tag: {'Environment': 'production', 'Role': 'database'}
sync_after_install: grains
The profile can now be realized with a salt command:

View File

@ -0,0 +1,132 @@
==============================
Getting Started With HP Cloud
==============================
HP Cloud is a major public cloud platform and uses the libcloud
`openstack` driver. The current version of OpenStack that HP Cloud
uses is Grizzly. When an instance is booted, it must have a
floating IP added to it in order to connect to it and further below
you will see an example that adds context to this statement.
To use the `openstack` driver for HP Cloud, set up the cloud
provider configuration file as in the example shown below:
``/etc/salt/cloud.providers.d/hpcloud.conf``:
.. code-block:: yaml
hpcloud-config:
# Set the location of the salt-master
#
minion:
master: saltmaster.example.com
# Configure HP Cloud using the OpenStack plugin
#
identity_url: https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0/tokens
compute_name: Compute
protocol: ipv4
# Set the compute region:
#
compute_region: region-b.geo-1
# Configure HP Cloud authentication credentials
#
user: myname
tenant: myname-project1
password: xxxxxxxxx
# keys to allow connection to the instance launched
#
ssh_key_name: yourkey
ssh_key_file: /path/to/key/yourkey.priv
provider: openstack
The subsequent example that follows is using the openstack driver.
Compute Region
==============
Originally, HP Cloud, in its OpenStack Essex version (1.0), had 3
availability zones in one region, US West (region-a.geo-1), which
each behaved each as a region.
This has since changed, and the current OpenStack Grizzly version of
HP Cloud (1.1) now has simplified this and now has two regions to choose from:
.. code-block:: bash
region-a.geo-1 -> US West
region-b.geo-1 -> US East
Authentication
==============
The ``user`` is the same user as is used to log into the HP Cloud management
UI. The ``tenant`` can be found in the upper left under "Project/Region/Scope".
It is often named the same as ``user`` albeit with a ``-project1`` appended.
The ``password`` is of course what you created your account with. The management
UI also has other information such as being able to select US East or US West.
The profile shown below is a know working profile for an Ubuntu instance. The
profile configuration file is stored in the following location:
``/etc/salt/cloud.profiles.d/hp_ae1_ubuntu.conf``:
.. code-block:: yaml
hp_ae1_ubuntu:
provider: hp_ae1
image: 9302692b-b787-4b52-a3a6-daebb79cb498
ignore_cidr: 10.0.0.1/24
networks:
- floating: Ext-Net
size: standard.small
ssh_key_file: /root/keys/test.key
ssh_key_name: test
ssh_username: ubuntu
Some important things about the example above:
* The ``image`` parameter can use either the image name or image ID which you can obtain by running in the example below (this case US East):
.. code-block:: bash
# salt-cloud --list-images hp_ae1
* The parameter ``ignore_cidr`` specifies a range of addresses to ignore when trying to connect to the instance. In this case, it's the range of IP addresses used for an internal IP of the instance.
* The parameter ``networks`` is very important to include. This is what makes it possible for salt-cloud to be able to attach a floating IP to the instance in order to connect to the instance and set up the minion
* The ``ssh_key_file`` and ``ssh_key_name`` are the keys that will make it possible to connect to the instance to set up the minion
* The ``ssh_username`` parameter, in this case, being that the image used will be ubuntu, will make it possible to not only log in but install the minion
To instantiate a machine based on this profile (example):
.. code-block:: bash
# salt-cloud -p hp_ae1_ubuntu ubuntu_instance_1
After several minutes, this will create an instance named ubuntu_instance_1
running in HP Cloud in the US East region and will set up the minion and then
return information about the instance once completed.
Once the instance has been created with salt-minion installed, connectivity to
it can be verified with Salt:
.. code-block:: bash
# salt ubuntu_instance_1 ping
Additionally, the instance can be acessed via SSH using the floating IP assigned to it
.. code-block:: bash
# ssh ubuntu@<floating ip>

View File

@ -22,6 +22,7 @@ Getting Started
Getting Started With OpenStack <openstack>
Getting Started With Parallels <parallels>
Getting Started With Rackspace <rackspace>
Getting Started With HP Cloud <hpcloud>
Getting Started With SoftLayer <softlayer>
Core Configuration

View File

@ -1,111 +1,17 @@
Install Salt Cloud
==================
Salt Cloud has only two dependencies:
* ``salt``
* ``apache-libcloud``
Of course, ``salt`` has it's own set of dependencies and the same applies to
``apache-libcloud``.
Salt Cloud is now part of Salt proper. It was merged in as of
:doc:`Salt version 2014.1.0 </topics/releases/2014.1.0>`.
Salt Cloud depends on ``apache-libcloud``. Libcloud can be installed via pip
with ``pip install apache-libcloud``.
Installing Salt Cloud for development
-------------------------------------
Clone the repository using:
Installing Salt for development enables Salt Cloud development as well, just
make sure ``apache-libcloud`` is installed as per above paragraph.
.. code-block:: bash
See these instructions: :doc:`Installing Salt for development </topics/development/hacking>`.
git clone https://github.com/saltstack/salt-cloud.git
Create a new `virtualenv`_:
.. code-block:: bash
virtualenv /path/to/your/virtualenv
.. _`virtualenv`: https://pypi.python.org/pypi/virtualenv
On Arch Linux, where Python 3 is the default installation of Python, use the
``virtualenv2`` command instead of ``virtualenv``.
.. note:: Using system Python modules in the virtualenv
To use already-installed python modules in virtualenv (instead of having pip
download and compile new ones), run ``virtualenv --system-site-packages``
Using this method eliminates the requirement to install the salt
dependencies again, although it does assume that the listed modules are all
installed in the system ``PYTHONPATH`` at the time of virtualenv creation.
Activate the virtualenv:
.. code-block:: bash
source /path/to/your/virtualenv/bin/activate
.. _dependencies:
Install Salt Cloud (and dependencies) into the virtualenv:
.. code-block:: bash
pip install M2Crypto # Don't install on Debian/Ubuntu (see below)
pip install pyzmq PyYAML pycrypto msgpack-python jinja2 psutil salt
pip install apache-libcloud
pip install -e ./salt-cloud # the path to the salt-cloud git clone
.. note:: Installing M2Crypto
``swig`` and ``libssl-dev`` are required to build M2Crypto. To fix the
error ``command 'swig' failed with exit status 1`` while installing
M2Crypto, try installing it with the following command:
.. code-block:: bash
env SWIG_FEATURES="-cpperraswarn -includeall -D__`uname -m`__ -I/usr/include/openssl" pip install M2Crypto
Debian and Ubuntu systems have modified openssl libraries and mandate that
a patched version of M2Crypto be installed. This means that M2Crypto
needs to be installed via apt:
.. code-block:: bash
apt-get install python-m2crypto
This also means that pulling in the M2Crypto installed using apt requires
using ``--system-site-packages`` when creating the virtualenv.
Or using a pre-patched M2Crypto
.. code-block:: bash
pip install http://dl.dropbox.com/u/174789/m2crypto-0.20.1.tar.gz
Using easy_install to Install Salt Cloud
----------------------------------------
If you are installing using ``easy_install``, you will need to define a
:strong:`USE_SETUPTOOLS` environment variable, otherwise dependencies will not
be installed:
.. code-block:: bash
USE_SETUPTOOLS=1 easy_install salt-cloud
Installing Salt Cloud from Git
------------------------------
To install salt cloud from ``git`` without any development purposes in mind,
install the required dependencies_ replacing the last step with:
.. code-block:: bash
pip install git+https://github.com/saltstack/salt-cloud.git#egg=salt_cloud

View File

@ -13,33 +13,6 @@ Quick Install
On most distributions, you can set up a **Salt Minion** with the bootstrap script:
.. code-block:: bash
curl -L http://bootstrap.saltstack.org | sudo sh
or, to connect immediately to a running Salt Master:
.. code-block:: bash
curl -L http://bootstrap.saltstack.org | sudo sh -s -- -A saltmaster.example.com
To set up a **Salt Master**:
.. code-block:: bash
curl -L http://bootstrap.saltstack.org | sudo sh -s -- -M -N
Currently the install script has been tested to work on:
* Ubuntu 10.x/11.x/12.x
* Debian 6.x
* CentOS 6.3
* Fedora
* Arch
* FreeBSD 9.0
See `Salt Bootstrap`_ for more information.
.. _`Salt Bootstrap`: https://github.com/saltstack/salt-bootstrap

View File

@ -113,6 +113,14 @@ Due to the fact that the targeting approach differs in salt-ssh, only glob
and regex targets are supported as of this writing, the remaining target
systems still need to be implemented.
Configuring Salt SSH
====================
Salt SSH takes its configuration from a master configuration file. Normally, this
file is in ``/etc/salt/master``. If one wishes to use a customized configuration file,
the ``-c`` option to Salt SSH facilitates passing in a directory to look inside for a
configuration file named ``master``.
Running Salt SSH as non-root user
=================================

View File

@ -114,10 +114,7 @@ def __virtual__():
'''
Only load on Linux systems
'''
if HAS_PAM:
return 'pam'
else:
return False
return HAS_PAM
def authenticate(username, password, service='login'):

View File

@ -35,7 +35,7 @@ def __virtual__():
Requires newer pycrypto and pyOpenSSL
'''
if HAS_DEPS:
return 'pki'
return True
return False

View File

@ -112,7 +112,8 @@ class Batch(object):
# create a new iterator for this batch of minions
new_iter = self.local.cmd_iter_no_block(
*args,
raw=self.opts.get('raw', False))
raw=self.opts.get('raw', False),
ret=self.opts.get('return', ''))
# add it to our iterators and to the minion_tracker
iters.append(new_iter)
minion_tracker[new_iter] = {}

View File

@ -700,6 +700,7 @@ class LocalClient(object):
pub_data['minions'],
timeout,
tgt,
expr_form,
**kwargs):
yield fn_ret

View File

@ -59,12 +59,12 @@ SSH_SHIM = r'''/bin/sh << 'EOF'
MISS_PKG="$MISS_PKG tar"
fi
for py_candidate in \\
python27 \\
python2.7 \\
python26 \\
python2.6 \\
python2 \\
for py_candidate in \
python27 \
python2.7 \
python26 \
python2.6 \
python2 \
python ;
do
command -v $py_candidate >/dev/null
@ -83,9 +83,9 @@ SSH_SHIM = r'''/bin/sh << 'EOF'
SALT="/tmp/.salt/salt-call"
if [ "{{2}}" = "md5" ]
then
for md5_candidate in \\
md5sum \\
md5 \\
for md5_candidate in \
md5sum \
md5 \
csum ;
do
command -v $md5_candidate >/dev/null

View File

@ -2744,9 +2744,17 @@ def copy_snapshot(kwargs=None, call=None):
return data
def describe_snapshot(kwargs=None, call=None):
def describe_snapshots(kwargs=None, call=None):
'''
Describe a snapshot
Describe a snapshots
Options: snapshot_id - One or more snapshot IDs.
Multiple IDs must be separated by ",".
owner - Returns the snapshots owned by the specified owner. Multiple owners can be specified.
Valid values: self | amazon | AWS Account ID
Multiple values must be separated by ",".
restorable_by - One or more AWS accounts IDs that can create volumes from the snapshot.
Multiple aws account IDs must be separated by ",".
TODO: Add all of the filters.
'''
if call != 'function':
@ -2755,14 +2763,22 @@ def describe_snapshot(kwargs=None, call=None):
)
return False
if 'snapshot_id' not in kwargs:
log.error('A snapshot_id must be specified to describe a snapshot.')
return False
params = {'Action': 'DescribeSnapshots'}
if 'snapshot_id' in kwargs:
params['SnapshotId'] = kwargs['snapshot_id']
if 'snapshot_ids' in kwargs:
snapshot_ids = kwargs['snapshot_ids'].split(',')
for snapshot_index, snapshot_id in enumerate(snapshot_ids):
params['SnapshotId.{0}'.format(snapshot_index)] = snapshot_id
if 'owner' in kwargs:
owners = kwargs['owner'].split(',')
for owner_index, owner in enumerate(owners):
params['Owner.{0}'.format(owner_index)] = owner
if 'restorable_by' in kwargs:
restorable_bys = kwargs['restorable_by'].split(',')
for restorable_by_index, restorable_by in enumerate(restorable_bys):
params['RestorableBy.{0}'.format(restorable_by_index)] = restorable_by
log.debug(params)

View File

@ -67,7 +67,7 @@ def _minion_opts(cfg='minion'):
default_dir = salt.syspaths.CONFIG_DIR,
cfg = os.environ.get(
'SALT_MINION_CONFIG', os.path.join(default_dir, cfg))
opts = config.minion_conf(cfg)
opts = config.minion_config(cfg)
return opts
@ -220,16 +220,16 @@ def list_nodes(conn=None, call=None):
hide = False
names = __opts__.get('names', [])
profile = __opts__.get('profile', [])
destroy = __opts__.get('destroy', False)
destroy_opt = __opts__.get('destroy', False)
action = __opts__.get('action', '')
for opt in ['full_query', 'select_query', 'query']:
if __opts__.get(opt, False):
call = 'full'
if destroy:
if destroy_opt:
call = 'full'
if action and not call:
call = 'action'
if profile and names and not destroy:
if profile and names and not destroy_opt:
hide = True
if not get_configured_provider():
return
@ -247,7 +247,7 @@ def list_nodes(conn=None, call=None):
}
# in creation mode, we need to go inside the create method
# so we hide the running vm from being seen as already installed
# do not also mask half configured nodes which are explictly asked
# do not also mask half configured nodes which are explicitly asked
# to be acted on, on the command line
if (
(call in ['full'] or not hide)
@ -320,9 +320,9 @@ last message: {comment}'''.format(**ret)
def destroy(vm_, call=None):
'''Destroy a lxc container'''
destroy = __opts__.get('destroy', False)
destroy_opt = __opts__.get('destroy', False)
action = __opts__.get('action', '')
if action != 'destroy' and not destroy:
if action != 'destroy' and not destroy_opt:
raise SaltCloudSystemExit(
'The destroy action must be called with -d, --destroy, '
'-a or --action.'

View File

@ -62,12 +62,33 @@ examples could be set up in the cloud configuration at
For local installations that only use private IP address ranges, the
following option may be useful. Using the old syntax:
Note: For api use, you will need an auth plugin. The base novaclient does not
support apikeys, but some providers such as rackspace have extended keystone to
accept them
.. code-block:: yaml
my-openstack-config:
# Ignore IP addresses on this network for bootstrap
ignore_cidr: 192.168.50.0/24
my-nova:
identity_url: 'https://identity.api.rackspacecloud.com/v2.0/'
compute_region: IAD
user: myusername
password: mypassword
tenant: <userid>
provider: nova
my-api:
identity_url: 'https://identity.api.rackspacecloud.com/v2.0/'
compute_region: IAD
user: myusername
api_key: <api_key>
os_auth_plugin: rackspace
tenant: <userid>
provider: nova
'''
# pylint: disable=E0102
@ -88,6 +109,7 @@ from salt.utils.openstack import nova
HASNOVA = False
try:
from novaclient.v1_1 import client # pylint: disable=W0611
import novaclient.exceptions
HASNOVA = True
except ImportError:
pass
@ -122,8 +144,6 @@ log = logging.getLogger(__name__)
# Some of the libcloud functions need to be in the same namespace as the
# functions defined in the module, so we create new function objects inside
# this module namespace
get_size = namespaced_function(get_size, globals())
get_image = namespaced_function(get_image, globals())
avail_locations = namespaced_function(avail_locations, globals())
script = namespaced_function(script, globals())
destroy = namespaced_function(destroy, globals())
@ -162,13 +182,15 @@ def get_conn():
'''
vm_ = get_configured_provider()
kwargs = {
'username': vm_['user'],
'api_key': vm_['password'],
'project_id': vm_['tenant'],
'auth_url': vm_['identity_url'],
'region_name': vm_['compute_region']
}
kwargs = vm_.copy()
kwargs['username'] = vm_['user']
kwargs['project_id'] = vm_['tenant']
kwargs['auth_url'] = vm_['identity_url']
kwargs['region_name'] = vm_['compute_region']
if 'password' in vm_:
kwargs['password'] = vm_['password']
return nova.SaltNova(**kwargs)
@ -187,12 +209,19 @@ def get_image(conn, vm_):
if vm_image in (image_list[img]['id'], img):
return image_list[img]['id']
raise SaltCloudNotFound(
'The specified image, {0!r}, could not be found.'.format(vm_image)
)
try:
image = conn.image_show(vm_image)
return image['id']
except novaclient.exceptions.NotFound as exc:
raise SaltCloudNotFound(
'The specified image, {0!r}, could not be found: {1}'.format(
vm_image,
exc.message
)
)
def show_instance(name, call=None, **kwargs):
def show_instance(name, call=None):
'''
Show the details from the provider concerning an instance
'''
@ -329,8 +358,8 @@ def destroy(name, conn=None, call=None):
flush_mine_on_destroy = profiles[profile]['flush_mine_on_destroy']
if flush_mine_on_destroy:
log.info('Clearing Salt Mine: {0}'.format(name))
client = salt.client.LocalClient(__opts__['conf_file'])
minions = client.cmd(name, 'mine.flush')
salt_client = salt.client.LocalClient(__opts__['conf_file'])
minions = salt_client.cmd(name, 'mine.flush')
log.info('Clearing Salt Mine: {0}, {1}'.format(
name,
@ -891,14 +920,13 @@ def volume_delete(name, **kwargs):
return conn.volume_delete(name)
def volume_detach(name, server_name, **kwargs):
def volume_detach(name, **kwargs):
'''
Detach block volume
'''
conn = get_conn()
return conn.volume_detach(
name,
server_name,
timeout=300
)

View File

@ -444,6 +444,7 @@ DEFAULT_MASTER_OPTS = {
'ssh_timeout': 60,
'ssh_user': 'root',
'master_floscript': os.path.join(FLO_DIR, 'master.flo'),
'worker_floscript': os.path.join(FLO_DIR, 'worker.flo'),
'ioflo_verbose': 3,
'ioflo_period': 0.01,
'ioflo_realtime': True,

View File

@ -17,11 +17,17 @@ opts['ioflo_realtime']
opts['ioflo_verbose']
'''
# Import modules
from . import master
from . import minion
# Import python libs
import multiprocessing
__all__ = ['master', 'minion']
# Import modules
from . import core
from . import worker
__all__ = ['core', 'worker']
# Import salt libs
import salt.daemons.masterapi
# Import ioflo libs
import ioflo.app.run
@ -48,6 +54,42 @@ class IofloMaster(object):
Assign self.opts
'''
self.opts = opts
self.preloads = explode_opts(self.opts)
self.access_keys = salt.daemons.masterapi.access_keys(self.opts)
def _make_workers(self):
'''
Spin up a process for each worker thread
'''
for ind in range(int(self.opts['worker_threads'])):
proc = multiprocessing.Process(
target=self._worker, kwargs={'yid': ind + 1}
)
proc.start()
def _worker(self, yid):
'''
Spin up a worker, do this in s multiprocess
'''
behaviors = ['salt.transport.road.raet', 'salt.daemons.flo']
self.preloads.append(('.salt.yid', dict(value=yid)))
self.preloads.append(
('.salt.access_keys', dict(value=self.access_keys)))
ioflo.app.run.start(
name='worker{0}'.format(yid),
period=float(self.opts['ioflo_period']),
stamp=0.0,
real=self.opts['ioflo_realtime'],
filepath=self.opts['worker_floscript'],
behaviors=behaviors,
username="",
password="",
mode=None,
houses=None,
metas=None,
preloads=self.preloads,
verbose=int(self.opts['ioflo_verbose']),
)
def start(self):
'''
@ -55,8 +97,8 @@ class IofloMaster(object):
port = self.opts['raet_port']
'''
self._make_workers()
behaviors = ['salt.transport.road.raet', 'salt.daemons.flo']
preloads = explode_opts(self.opts)
ioflo.app.run.start(
name='master',
period=float(self.opts['ioflo_period']),
@ -69,7 +111,7 @@ class IofloMaster(object):
mode=None,
houses=None,
metas=None,
preloads=preloads,
preloads=self.preloads,
verbose=int(self.opts['ioflo_verbose']),
)

View File

@ -11,6 +11,7 @@ import types
import logging
import multiprocessing
import traceback
import itertools
from collections import deque
# Import salt libs
@ -121,6 +122,7 @@ class Setup(ioflo.base.deeding.Deed):
'fun': '.salt.local.fun',
'event': '.salt.event.events',
'event_req': '.salt.event.event_req',
'workers': '.salt.track.workers',
'uxd_stack': '.salt.uxd.stack.stack'}
def postinitio(self):
@ -128,6 +130,7 @@ class Setup(ioflo.base.deeding.Deed):
Set up required objects and queues
'''
self.uxd_stack.value = stacking.StackUxd(
name='yard',
lanename=self.opts.value['id'],
yid=0,
dirpath=self.opts.value['sock_dir'])
@ -138,6 +141,11 @@ class Setup(ioflo.base.deeding.Deed):
self.event.value = deque()
self.event_req.value = deque()
self.publish.value = deque()
if self.opts.value.get('worker_threads'):
worker_seed = []
for ind in range(self.opts.value['worker_threads']):
worker_seed.append('yard{0}'.format(ind + 1))
self.workers.value = itertools.cycle(worker_seed)
class Rx(ioflo.base.deeding.Deed):
@ -187,6 +195,7 @@ class Router(ioflo.base.deeding.Deed):
'fun': '.salt.local.fun',
'event': '.salt.event.events',
'event_req': '.salt.event.event_req',
'workers': '.salt.track.workers',
'uxd_stack': '.salt.uxd.stack.stack',
'udp_stack': '.raet.udp.stack.stack'}
@ -219,8 +228,9 @@ class Router(ioflo.base.deeding.Deed):
# Refuse local commands over the wire
log.error('Received local command remotely! Ignoring: {0}'.format(msg))
return
l_stack = getattr(self, d_share)
l_stack.value.append(msg)
if d_share == 'remote_cmd':
# Send it to a remote worker
self.uxd_stack.value.transmit(msg, next(self.workers.value))
def _process_uxd_rxmsg(self, msg):
'''
@ -251,8 +261,8 @@ class Router(ioflo.base.deeding.Deed):
# No queue destination!
log.error('Received message without share: {0}'.format(msg))
return
l_stack = getattr(self, d_share)
l_stack.value.append(msg)
if d_share == 'local_cmd':
self.uxd_stack.value.transmit(msg, next(self.workers.value))
def action(self):
'''

View File

@ -6,17 +6,32 @@ init .raet.udp.stack.local to eid 1 name "master" host "" port 7530
init port in .raet.udp.stack.local from value in .salt.etc.raet_port
framer masterudpstack be active first start
framer masterudpstack be active first setup
frame setup
enter
do setup
go start
frame start
do raet udp stack per inode ".raet.udp.stack"
exit
do raet udp stack closer per inode ".raet.udp.stack."
framer inbound be active first start
frame start
do rx
framer uxdrouter be active first start
frame start
do master router
#timeout 20
do router
framer localcmd be active first start
framer events be active first start
frame start
do cmd local
do eventer
framer publish be active first start
frame start
do publisher
framer outbound be active first start
frame start
do tx

View File

@ -1,220 +0,0 @@
# -*- coding: utf-8 -*-
'''
The behaviors to run the salt master via ioflo
'''
# Import python libs
from collections import deque
# Import salt libs
import salt.daemons.masterapi
from salt.transport.road.raet import stacking
from salt.transport.road.raet import yarding
# Import ioflo libs
import ioflo.base.deeding
@ioflo.base.deeding.deedify('master_keys', ioinits={'opts': '.salt.etc.opts', 'keys': '.salt.etc.keys.master'})
def master_keys(self):
'''
Return the master keys
'''
self.keys.value = salt.daemons.masterapi.master_keys(self.opts.value)
@ioflo.base.deeding.deedify('clean_old_jobs', ioinits={'opts': '.salt.etc.opts'})
def clean_old_jobs(self):
'''
Call the clan old jobs routine
'''
salt.daemons.masterapi.clean_old_jobs(self.opts.value)
@ioflo.base.deeding.deedify('access_keys', ioinits={'opts': '.salt.etc.opts'})
def access_keys(self):
'''
Build the access keys
'''
salt.daemons.masterapi.access_keys(self.opts.value)
@ioflo.base.deeding.deedify('fileserver_update', ioinits={'opts': '.salt.etc.opts'})
def fileserver_update(self):
'''
Update the fileserver backends
'''
salt.daemons.masterapi.fileserver_update(self.opts.value)
class RouterMaster(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Routes the communication in and out of uxd connections
'''
Ioinits = {'opts': '.salt.opts',
'event_yards': '.salt.uxd.yards.event',
'com_yards': '.salt.uxd.yards.com',
'local_cmd': '.salt.uxd.local_cmd',
'local_ret': '.salt.uxd.local_ret',
'events': '.salt.uxd.events',
'publish': '.salt.net.publish',
'stack': '.salt.uxd.stack.stack',
'udp_stack': '.raet.udp.stack.stack'}
def postinitio(self):
'''
Set up required objects
'''
self.stack.value = stacking.StackUxd(
name='router',
lanename='master',
yid=0,
dirpath=self.opts.value['sock_dir'])
self.event_yards.value = set()
self.com_yards.value = set()
self.local_cmd.value = deque()
self.local_ret.value = deque()
self.events.value = deque()
if not self.publish.value:
self.publish.value = deque()
def _register_event_yard(self, msg):
'''
register an incoming event request with the requesting yard id
'''
ev_yard = yarding.Yard(
yid=msg['load']['yid'],
prefix='master',
dirpath=msg['load']['dirpath'])
self.event_yards.value.add(ev_yard.name)
def _fire_event(self, event):
'''
Fire an event to all subscribed yards
'''
for y_name in self.event_yards.value:
route = {'src': ('router', self.stack.value.yard.name, None),
'dst': ('router', y_name, None)}
msg = {'route': route, 'event': event}
self.stack.value.transmit(msg, y_name)
def _process_rxmsg(self, msg):
'''
Send the message to the correct location
'''
try:
if msg['route']['dst'][2] == 'local_cmd':
self.local_cmd.value.append(msg)
elif msg['route']['dst'][2] == 'event_req':
# Register the event interface
self._register_event_yard(msg)
elif msg['route']['dst'][2] == 'event_fire':
# Register the event interface
self.events.value.append(
{'tag': msg['tag'],
'data': msg['data']})
except Exception:
return
def _publish(self, pub_msg):
'''
Publish the message out to the targetted minions
'''
import pprint
pprint.pprint(self.udp_stack.value.eids)
pprint.pprint(pub_msg)
for minion in self.udp_stack.value.eids:
eid = self.udp_stack.value.eids.get(minion)
if eid:
route = {'dst': (minion, None, 'fun')}
msg = {'route': route, 'pub': pub_msg['pub']}
self.udp_stack.value.message(msg, eid)
def action(self):
'''
Process the messages!
'''
self.stack.value.serviceAll()
# Process inboud communication stack
while self.stack.value.rxMsgs:
self._process_rxmsg(self.stack.value.rxMsgs.popleft())
while self.events.value:
self._fire_event(self.events.value.popleft())
while self.local_ret.value:
msg = self.local_ret.value.popleft()
self.stack.value.transmit(msg, msg['route']['dst'][1])
while self.publish.value:
pub_msg = self.publish.value.popleft()
self._publish(pub_msg)
self.stack.value.serviceAll()
class RemoteMaster(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Abstract access to the core salt master api
'''
Ioinits = {'opts': '.salt.opts',
'ret_in': '.salt.net.ret_in',
'ret_out': '.salt.net.ret_out'}
def postinitio(self):
'''
Set up required objects
'''
self.remote = salt.daemons.masterapi.RemoteFuncs(self.opts.value)
def action(self):
'''
Perform an action
'''
if self.ret_in.value:
exchange = self.ret_in.value.pop()
load = exchange.get('load')
# If the load is invalid, just ignore the request
if not 'cmd' in load:
return False
if load['cmd'].startswith('__'):
return False
exchange['ret'] = getattr(self.remote, load['cmd'])(load)
self.ret_out.value.append(exchange)
class LocalCmd(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Abstract access to the core salt master api
'''
Ioinits = {'opts': '.salt.opts',
'local_cmd': '.salt.uxd.local_cmd',
'local_ret': '.salt.uxd.local_ret',
'publish': '.salt.net.publish',
'stack': '.salt.uxd.stack.stack'}
def postinitio(self):
'''
Set up required objects
'''
self.access_keys = salt.daemons.masterapi.access_keys(self.opts.value)
self.local = salt.daemons.masterapi.LocalFuncs(self.opts.value, self.access_keys)
if not self.publish.value:
self.publish.value = deque()
def action(self):
'''
Perform an action
'''
while self.local_cmd.value:
cmd = self.local_cmd.value.popleft()
ret = {}
load = cmd.get('load')
# If the load is invalid, just ignore the request
if not 'cmd' in load:
return
if load['cmd'].startswith('__'):
return
if hasattr(self.local, load['cmd']):
ret['return'] = getattr(self.local, load['cmd'])(load)
ret['route'] = {'src': ('router', self.stack.value.yard.name, None),
'dst': cmd['route']['src']}
if load['cmd'] == 'publish':
self.publish.value.append(ret['return'])
self.local_ret.value.append(ret)

View File

@ -7,12 +7,20 @@ init port in .raet.udp.stack.local from value in .salt.etc.raet_port
init name in .raet.udp.stack.local from value in .salt.etc.id
framer minionudpstack be active first start
framer minionudpstack be active first setup
frame setup
enter
do setup
go start
frame start
do raet udp stack per inode ".raet.udp.stack"
exit
do raet udp stack closer per inode ".raet.udp.stack."
framer inbound be active first start
frame start
do rx
framer bootstrap be active first join
frame join
print Joining...
@ -43,24 +51,25 @@ framer bootstrap be active first join
print Allowed
go next
frame message
print Messaging...
enter
do raet udp stack messenger to contents "Minion 1 Hello" code 15 \
per inode ".raet.udp.stack."
go next
frame loading
enter
do load modules
go next
frame router
do minion router
do router
frame abort
bid stop all
framer eventing be active first event
frame event
do eventer
framer functionmanager be active first checkexec
frame checkexec
do nix function
do nix executor
framer outbound be active first start
frame start
do tx

View File

@ -1,361 +0,0 @@
# -*- coding: utf-8 -*-
'''
The behaviors to run the salt minion via ioflo
'''
# Import python libs
import os
import logging
import sys
import types
import traceback
import multiprocessing
from collections import deque
# Import salt libs
import salt.minion
import salt.payload
import salt.utils
import salt.utils.event
import salt.daemons.masterapi
import salt.utils.schedule
from salt.exceptions import (
CommandExecutionError, CommandNotFoundError, SaltInvocationError)
from salt.transport.road.raet import yarding
from salt.transport.road.raet import stacking
# Import ioflo libs
import ioflo.base.deeding
# Import Third Party Libs
HAS_PSUTIL = False
try:
import psutil
HAS_PSUTIL = True
except ImportError:
pass
HAS_RESOURCE = False
try:
import resource
HAS_RESOURCE = True
except ImportError:
pass
log = logging.getLogger(__name__)
class RouterMinion(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Route packaets from raet into minion proessing bins
'''
Ioinits = {'opts': '.salt.opts',
'udp_stack': '.raet.udp.stack.stack',
'uxd_stack': '.salt.uxd.stack.stack',
'fun_in': '.salt.net.fun_in',
}
def postinitio(self):
'''
Map opts for convenience
'''
self.uxd_stack.value = stacking.StackUxd(
lanename=self.opts.value['id'],
yid=0,
dirpath=self.opts.value['sock_dir'])
self.fun_in.value = deque()
def action(self):
'''
Empty the queues into process management queues
'''
# Start on the udp_in:
# TODO: Route UXD messages
while self.udp_stack.value.rxMsgs:
data = self.udp_stack.value.rxMsgs.popleft()
if data['route']['dst'][2] == 'fun':
self.fun_in.value.append(data)
if data['route']['dst'][1] is not None:
if data['route']['dst'][1] in self.uxd_stack.value.yards:
self.uxd_stack.value.transmit(data, data['route']['dst'][1])
self.uxd_stack.value.serviceAll()
while self.uxd_stack.value.rxMsgs:
msg = self.uxd_stack.value.rxMsgs.popleft()
estate = msg['route']['dst'][0]
if estate is not None:
if estate != self.opts.value['id']:
self.udp_stack.value.message(
msg,
self.udp_stack.value.eids[estate])
class ModulesLoad(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Reload the minion modules
'''
Ioinits = {'opts_store': '.salt.opts',
'grains': '.salt.loader.grains',
'modules': '.salt.loader.modules',
'returners': '.salt.loader.returners'}
def postinitio(self):
'''
Map opts for convenience
'''
self.opts = self.opts_store.value
def action(self):
'''
Return the functions and the returners loaded up from the loader
module
'''
# if this is a *nix system AND modules_max_memory is set, lets enforce
# a memory limit on module imports
# this feature ONLY works on *nix like OSs (resource module doesn't work on windows)
modules_max_memory = False
if self.opts.get('modules_max_memory', -1) > 0 and HAS_PSUTIL and HAS_RESOURCE:
log.debug(
'modules_max_memory set, enforcing a maximum of {0}'.format(
self.opts['modules_max_memory'])
)
modules_max_memory = True
old_mem_limit = resource.getrlimit(resource.RLIMIT_AS)
rss, vms = psutil.Process(os.getpid()).get_memory_info()
mem_limit = rss + vms + self.opts['modules_max_memory']
resource.setrlimit(resource.RLIMIT_AS, (mem_limit, mem_limit))
elif self.opts.get('modules_max_memory', -1) > 0:
if not HAS_PSUTIL:
log.error('Unable to enforce modules_max_memory because psutil is missing')
if not HAS_RESOURCE:
log.error('Unable to enforce modules_max_memory because resource is missing')
self.opts['grains'] = salt.loader.grains(self.opts)
self.grains.value = self.opts['grains']
self.modules.value = salt.loader.minion_mods(self.opts)
self.returners.value = salt.loader.returners(self.opts, self.modules.value)
# we're done, reset the limits!
if modules_max_memory is True:
resource.setrlimit(resource.RLIMIT_AS, old_mem_limit)
class Schedule(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Evaluates the scedule
'''
Ioinits = {'opts_store': '.salt.opts',
'grains': '.salt.grains',
'modules': '.salt.loader.modules',
'returners': '.salt.loader.returners',
'master_ret': '.salt.net.master_out'}
def postinitio(self):
'''
Map opts and make the scedule object
'''
self.scedule = salt.utils.schedule.Schedule(
self.opts.value,
self.modules.value,
self.returners.value)
def action(self):
'''
Eval the schedule
'''
self.scedule.eval()
class FunctionNix(ioflo.base.deeding.Deed): # pylint: disable=W0232
'''
Execute a function call
'''
Ioinits = {'opts_store': '.salt.opts',
'grains': '.salt.grains',
'modules': '.salt.loader.modules',
'returners': '.salt.loader.returners',
'fun_ack': '.salt.net.fun_ack',
'fun_in': '.salt.net.fun_in',
'master_ret': '.salt.net.master_out',
'uxd_stack': '.salt.uxd.stack.stack',
'executors': '.salt.track.executors'}
def postinitio(self):
'''
Map opts for convenience
'''
self.opts = self.opts_store.value
self.matcher = salt.minion.Matcher(
self.opts,
self.modules.value)
self.proc_dir = salt.minion.get_proc_dir(self.opts['cachedir'])
self.serial = salt.payload.Serial(self.opts)
self.executors.value = {}
def _return_pub(self, ret):
'''
Send the return data back via the uxd socket
'''
ret_stack = stacking.StackUxd(
lanename=self.opts['id'],
yid=ret['jid'],
dirpath=self.opts['sock_dir'])
main_yard = yarding.Yard(
yid=0,
prefix=self.opts['id'],
dirpath=self.opts['sock_dir']
)
ret_stack.addRemoteYard(main_yard)
route = {'src': (self.opts['id'], ret_stack.yard.name, 'jid_ret'),
'dst': ('master', None, 'return')}
msg = {'route': route, 'return': ret}
ret_stack.transmit(msg, 'yard0')
ret_stack.serviceAll()
def action(self):
'''
Pull the queue for functions to execute
'''
if not self.fun_in.value:
return
exchange = self.fun_in.value.popleft()
data = exchange.get('pub')
# convert top raw strings - take this out once raet is using msgpack
for key, val in data.items():
if isinstance(val, basestring):
data[str(key)] = str(val)
else:
data[str(key)] = val
match = getattr(
self.matcher,
'{0}_match'.format(
data.get('tgt_type', 'glob')
)
)(data['tgt'])
if not match:
return
if 'user' in data:
log.info(
'User {0[user]} Executing command {0[fun]} with jid '
'{0[jid]}'.format(data))
else:
log.info(
'Executing command {0[fun]} with jid {0[jid]}'.format(data)
)
log.debug('Command details {0}'.format(data))
ex_yard = yarding.Yard(
yid=data['jid'],
prefix=self.opts['id'],
dirpath=self.opts['sock_dir'])
self.uxd_stack.value.addRemoteYard(ex_yard)
process = multiprocessing.Process(
target=self.proc_run,
kwargs={'exchange': exchange}
)
process.start() # Don't join this process! The process daemonizes
# itself and init will clean it up
def proc_run(self, exchange):
'''
Execute the run in a dedicated process
'''
data = exchange['pub']
fn_ = os.path.join(self.proc_dir, data['jid'])
self.opts['__ex_id'] = data['jid']
salt.utils.daemonize_if(self.opts)
sdata = {'pid': os.getpid()}
sdata.update(data)
with salt.utils.fopen(fn_, 'w+') as fp_:
fp_.write(self.serial.dumps(sdata))
ret = {'success': False}
function_name = data['fun']
if function_name in self.modules.value:
try:
func = self.modules.value[data['fun']]
args, kwargs = salt.minion.parse_args_and_kwargs(func, data['arg'], data)
sys.modules[func.__module__].__context__['retcode'] = 0
return_data = func(*args, **kwargs)
if isinstance(return_data, types.GeneratorType):
ind = 0
iret = {}
for single in return_data:
if isinstance(single, dict) and isinstance(iret, list):
iret.update(single)
else:
if not iret:
iret = []
iret.append(single)
tag = salt.utils.event.tagify(
[data['jid'], 'prog', self.opts['id'], str(ind)],
'job')
event_data = {'return': single}
self._fire_master(event_data, tag) # Need to look into this
ind += 1
ret['return'] = iret
else:
ret['return'] = return_data
ret['retcode'] = sys.modules[func.__module__].__context__.get(
'retcode',
0
)
ret['success'] = True
except CommandNotFoundError as exc:
msg = 'Command required for {0!r} not found'.format(
function_name
)
log.debug(msg, exc_info=True)
ret['return'] = '{0}: {1}'.format(msg, exc)
except CommandExecutionError as exc:
log.error(
'A command in {0!r} had a problem: {1}'.format(
function_name,
exc
),
exc_info=log.isEnabledFor(logging.DEBUG)
)
ret['return'] = 'ERROR: {0}'.format(exc)
except SaltInvocationError as exc:
log.error(
'Problem executing {0!r}: {1}'.format(
function_name,
exc
),
exc_info=log.isEnabledFor(logging.DEBUG)
)
ret['return'] = 'ERROR executing {0!r}: {1}'.format(
function_name, exc
)
except TypeError as exc:
aspec = salt.utils.get_function_argspec(
self.modules.value[data['fun']]
)
msg = ('TypeError encountered executing {0}: {1}. See '
'debug log for more info. Possibly a missing '
'arguments issue: {2}').format(function_name,
exc,
aspec)
log.warning(msg, exc_info=log.isEnabledFor(logging.DEBUG))
ret['return'] = msg
except Exception:
msg = 'The minion function caused an exception'
log.warning(msg, exc_info=log.isEnabledFor(logging.DEBUG))
ret['return'] = '{0}: {1}'.format(msg, traceback.format_exc())
else:
ret['return'] = '{0!r} is not available.'.format(function_name)
ret['jid'] = data['jid']
ret['fun'] = data['fun']
ret['fun_args'] = data['arg']
self._return_pub(ret)
if data['ret']:
ret['id'] = self.opts['id']
for returner in set(data['ret'].split(',')):
try:
self.returners.value['{0}.returner'.format(
returner
)](ret)
except Exception as exc:
log.error(
'The return failed for job {0} {1}'.format(
data['jid'],
exc
)
)

View File

@ -0,0 +1,7 @@
# Raet Test FloScript
house worker
framer uxdrouter be active first start
frame start
do worker router

View File

@ -0,0 +1,66 @@
# -*- coding: utf-8 -*-
'''
The core bahaviuors ued by minion and master
'''
# pylint: disable=W0232
# Import salt libs
import salt.daemons.masterapi
from salt.transport.road.raet import stacking
from salt.transport.road.raet import yarding
# Import ioflo libs
import ioflo.base.deeding
class RouterWorker(ioflo.base.deeding.Deed):
Ioinits = {
'uxd_stack': '.salt.uxd.stack.stack',
'opts': '.salt.opts',
'yid': '.salt.yid',
'access_keys': '.salt.access_keys',
}
def postinitio(self):
'''
Set up the uxd stack
'''
self.uxd_stack.value = stacking.StackUxd(
lanename=self.opts.value['id'],
yid=self.yid.value,
dirpath=self.opts.value['sock_dir'])
manor_yard = yarding.Yard(
yid=0,
prefix=self.opts.value['id'],
dirpath=self.opts.value['sock_dir'])
self.uxd_stack.value.addRemoteYard(manor_yard)
self.remote = salt.daemons.masterapi.RemoteFuncs(self.opts.value)
self.local = salt.daemons.masterapi.LocalFuncs(
self.opts.value,
self.access_keys.value)
def action(self):
'''
Read in a command and execute it, send the return back up to the
main master process
'''
self.uxd_stack.value.serviceAll()
while self.uxd_stack.value.rxMsgs:
msg = self.uxd_stack.value.rxMsgs.popleft()
if 'load' in msg:
cmd = msg['load'].get['cmd']
if not cmd:
continue
elif cmd.startswith('__'):
continue
ret = {}
if hasattr(self.remote, cmd):
ret['return'] = getattr(self.remote, cmd)(msg['load'])
elif hasattr(self.local, cmd):
ret['return'] = getattr(self.local, cmd)(msg['load'])
ret['route'] = {
'src': (self.opts.value['id'], self.yid.value, None),
'dst': (msg['route']['src'][0], msg['route']['src'][1], 'ret')
}
self.uxd_stack.value.transmit(ret, 'yard0')
self.uxd_stack.value.serviceAll()

View File

@ -174,7 +174,7 @@ class RemoteFuncs(object):
# Create the tops dict for loading external top data
self.tops = salt.loader.tops(self.opts)
# Make a client
self.local = salt.client.LocalClient(self.opts['conf_file'])
self.local = salt.client.get_local_client(mopts=self.opts)
# Create the master minion to access the external job cache
self.mminion = salt.minion.MasterMinion(
self.opts,
@ -761,7 +761,7 @@ class LocalFuncs(object):
# Create the event manager
self.event = salt.utils.event.MasterEvent(self.opts['sock_dir'])
# Make a client
self.local = salt.client.LocalClient(self.opts['conf_file'])
self.local = salt.client.get_local_client(mopts=self.opts)
# Make an minion checker object
self.ckminions = salt.utils.minions.CkMinions(opts)
# Make an Auth object

View File

@ -836,12 +836,27 @@ class Loader(object):
)
continue
if virtual is not True and module_name != virtual:
# If __virtual__ returned True the module will
# be loaded with the same name, if it returned
# other value than `True`, it should be a new
# name for the module.
# Update the module name with the new name
# At this point, __virtual__ did not return a
# boolean value, let's check for deprecated usage
# or module renames
if virtual is not True and module_name == virtual:
# The module was not renamed, it should
# have returned True instead
#salt.utils.warn_until(
# 'Helium',
# 'The {0!r} module is NOT renaming itself '
# 'and is returning a string. In this case '
# 'the __virtual__() function should simply '
# 'return `True`. This usage will become an '
# 'error in Salt Helium'.format(
# mod.__name__,
# )
#)
pass
elif virtual is not True and module_name != virtual:
# The module is renaming itself. Updating the
# module name with the new name
log.debug(
'Loaded {0} as virtual {1}'.format(
module_name, virtual
@ -862,10 +877,31 @@ class Loader(object):
virtual
)
)
module_name = virtual
elif virtual and hasattr(mod, '__virtualname__'):
module_name = mod.__virtualname__
# Get the module's virtual name
virtualname = getattr(
mod, '__virtualname__', virtual
)
if virtualname != virtual:
# The __virtualname__ attribute does not
# match what's being returned by the
# __virtual__() function. This should be
# considered an error.
log.error(
'The module {0!r} is showing some bad '
'usage. It\'s __virtualname__ '
'attribute is set to {1!r} yet the '
'__virtual__() function is returning '
'{2!r}. These values should '
'match!'.format(
mod.__name__,
virtualname,
virtual
)
)
module_name = virtualname
except KeyError:
# Key errors come out of the virtual function when passing

View File

@ -30,7 +30,7 @@ def __virtual__():
Only if alternatives dir is available
'''
if os.path.isdir('/etc/alternatives'):
return 'alternatives'
return True
return False

View File

@ -24,7 +24,7 @@ def __virtual__():
# If none of the above commands are in $PATH this module is a no-go
if not any(_which(cmd) for cmd in commands):
return False
return 'archive'
return True
@decorators.which('tar')

View File

@ -86,15 +86,18 @@ def list_pkgs(versions_as_list=False, **kwargs):
__salt__['pkg_resource.stringify'](ret)
return ret
ret = {}
cmd = 'brew list --versions'
ret = {}
out = __salt__['cmd.run'](cmd, output_loglevel='debug')
for line in out.splitlines():
try:
name, version_num = line.split(' ')[0:2]
name_and_versions = line.split(' ')
name = name_and_versions[0]
installed_versions = name_and_versions[1:]
newest_version = sorted(installed_versions, cmp=salt.utils.version_cmp).pop()
except ValueError:
continue
__salt__['pkg_resource.add_pkg'](ret, name, version_num)
__salt__['pkg_resource.add_pkg'](ret, name, newest_version)
__salt__['pkg_resource.sort_pkglist'](ret)
__context__['pkg.list_pkgs'] = copy.deepcopy(ret)
@ -346,3 +349,39 @@ def upgrade_available(pkg):
salt '*' pkg.upgrade_available <package name>
'''
return pkg in list_upgrades()
def upgrade(refresh=True):
'''
Upgrade outdated, unpinned brews.
refresh
Fetch the newest version of Homebrew and all formulae from GitHub before installing.
Return a dict containing the new package names and versions::
{'<package>': {'old': '<old-version>',
'new': '<new-version>'}}
CLI Example:
.. code-block:: bash
salt '*' pkg.upgrade
'''
old = list_pkgs()
if salt.utils.is_true(refresh):
refresh_db()
cmd = 'brew upgrade'
user = __salt__['file.get_user'](_homebrew_bin())
__salt__['cmd.run'](
cmd,
runas=user if user != __opts__['user'] else __opts__['user'],
output_loglevel='debug'
)
__context__.pop('pkg.list_pkgs', None)
new = list_pkgs()
return salt.utils.compare_dicts(old, new)

View File

@ -25,7 +25,7 @@ def __virtual__():
cur_os = __grains__['kernel']
for _os in supported_os:
if cur_os == _os and salt.utils.which(supported_os[cur_os]):
return 'bridge'
return True
return False

View File

@ -20,9 +20,9 @@ __func_alias__ = {
def __virtual__():
'''
Ensure correct name
Always load
'''
return 'composer'
return True
def _valid_composer(composer):

View File

@ -19,6 +19,8 @@ SALT_CRON_NO_IDENTIFIER = 'NO ID SET'
def _encode(string):
if isinstance(string, unicode):
string = string.encode('utf-8')
elif not string:
string = ''
return "{0}".format(string)
@ -45,14 +47,31 @@ def _cron_matched(cron, cmd, identifier=None):
ret, id_matched = False, None
cid = _cron_id(cron)
if cid:
if not identifier:
identifier = SALT_CRON_NO_IDENTIFIER
eidentifier = _encode(identifier)
# old style second round
# after saving crontab, we must check that if
# we have not the same command, but the default id
# to not set that as a match
if (
cron.get('cmd', None) == cmd
cron.get('cmd', None) != cmd
and cid == SALT_CRON_NO_IDENTIFIER
and identifier
and eidentifier == SALT_CRON_NO_IDENTIFIER
):
cid = eidentifier
id_matched = eidentifier == cid
id_matched = False
else:
# on saving, be sure not to overwrite a cron
# with specific identifier but also track
# crons where command is the same
# but with the default if that we gonna overwrite
if (
cron.get('cmd', None) == cmd
and cid == SALT_CRON_NO_IDENTIFIER
and identifier
):
cid = eidentifier
id_matched = eidentifier == cid
if (
((id_matched is None) and cmd == cron.get('cmd', None))
or id_matched
@ -393,6 +412,11 @@ def set_job(user,
):
if identifier:
cid = identifier
if (
cid == SALT_CRON_NO_IDENTIFIER
and cron['identifier'] is None
):
cid = None
cron['identifier'] = cid
if not cid or (
cid and not _needs_change(cid, identifier)

View File

@ -20,7 +20,7 @@ def __virtual__():
'''
Only load module if dig binary is present
'''
return __virtualname__ if salt.utils.which('dig') else False
return True if salt.utils.which('dig') else False
def check_ip(addr):

View File

@ -21,7 +21,7 @@ def __virtual__():
'''
if salt.utils.is_windows():
return False
return 'disk'
return True
def _clean_flags(args, caller):

View File

@ -19,7 +19,7 @@ def __virtual__():
'''
if salt.utils.is_windows():
return False
return 'dnsmasq'
return True
def version():

View File

@ -18,7 +18,7 @@ def __virtual__():
Generic, should work on any platform (including Windows). Functionality
which requires dependencies outside of Python do not belong in this module.
'''
return 'dnsutil'
return True
def parse_hosts(hostsfile='/etc/hosts', hosts=None):

View File

@ -1424,11 +1424,15 @@ def build(path=None,
valid(status, id=image_id, out=out, comment='Image built')
else:
invalid(status, id=image_id, out=out)
else:
raise NotImplementedError(
'Unknown response type for build() {0!r}'.format(ret))
except Exception:
invalid(status,
out=traceback.format_exc(),
comment='Unexpected error while building an image')
return status
else:
invalid(status, comment='`path` or `fileobj` must be given')
return status

View File

@ -18,7 +18,7 @@ def __virtual__():
'''
if salt.utils.is_windows():
return False
return 'extfs'
return True
def mkfs(device, fs_type, **kwargs):

View File

@ -64,7 +64,7 @@ def __virtual__():
# win_file takes care of windows
if salt.utils.is_windows():
return False
return 'file'
return True
def __clean_tmp(sfn):
@ -894,6 +894,9 @@ def replace(path,
count=0,
flags=0,
bufsize=1,
append_if_not_found=False,
prepend_if_not_found=False,
not_found_content=None,
backup='.bak',
dry_run=False,
search_only=False,
@ -923,6 +926,15 @@ def replace(path,
before processing. Note: multiline searches must specify ``file``
buffering.
:type bufsize: int or str
:param append_if_not_found If pattern is not found and set to ``True``
then, the content will be appended to the file.
:param prepend_if_not_found If pattern is not found and set to ``True``
then, the content will be appended to the file.
:param not_found_content Content to use for append/prepend if not found. If
None (default), uses repl. Useful when repl uses references to group in
pattern.
:param backup: The file extension to use for a backup of the file before
editing. Set to ``False`` to skip making a backup.
:param dry_run: Don't make any edits to the file
@ -952,6 +964,12 @@ def replace(path,
.format(path)
)
if search_only and (append_if_not_found or prepend_if_not_found):
raise SaltInvocationError('Choose between search_only and append/prepend_if_not_found')
if append_if_not_found and prepend_if_not_found:
raise SaltInvocationError('Choose between append or prepend_if_not_found')
flags_num = _get_flags(flags)
cpattern = re.compile(pattern, flags_num)
if bufsize == 'file':
@ -973,6 +991,7 @@ def replace(path,
backup=False if dry_run else backup,
bufsize=bufsize,
mode='rb')
found = False
for line in fi_file:
if search_only:
@ -982,7 +1001,11 @@ def replace(path,
if result:
return True
else:
result = re.sub(cpattern, repl, line, count)
result, nrepl = re.subn(cpattern, repl, line, count)
# found anything? (even if no change)
if nrepl > 0:
found = True
# Identity check each potential change until one change is made
if has_changes is False and not result is line:
@ -996,6 +1019,27 @@ def replace(path,
print(result, end='', file=sys.stdout)
fi_file.close()
if not found and (append_if_not_found or prepend_if_not_found):
if None == not_found_content:
not_found_content = repl
if prepend_if_not_found:
new_file.insert(not_found_content + '\n')
else:
# append_if_not_found
# Make sure we have a newline at the end of the file
if 0 != len(new_file):
if not new_file[-1].endswith('\n'):
new_file[-1] += '\n'
new_file.append(not_found_content + '\n')
has_changes = True
if not dry_run:
# backup already done in filter part
# write new content in the file while avoiding partial reads
f = salt.utils.atomicfile.atomic_open(path, 'wb')
for line in new_file:
f.write(line)
f.close()
if not dry_run and not salt.utils.is_windows():
check_perms(path, None, pre_user, pre_group, pre_mode)
@ -1150,6 +1194,10 @@ def blockreplace(path,
new_file.insert(0, marker_start + '\n')
done = True
elif append_if_not_found:
# Make sure we have a newline at the end of the file
if 0 != len(new_file):
if not new_file[-1].endswith('\n'):
new_file[-1] += '\n'
# add the markers and content at the end of file
new_file.append(marker_start + '\n')
new_file.append(content + '\n')

View File

@ -22,7 +22,7 @@ def __virtual__():
'''
if not all((utils.which('git'), HAS_PIPES)):
return False
return 'git'
return True
def _git_ssh_helper(identity):

View File

@ -25,8 +25,8 @@ def __virtual__():
comment_regexp = re.compile(r'^\s*#\s*(.*)')
section_regexp = re.compile(r'\s*\[(.+)\]\s*')
option_regexp1 = re.compile(r'\s*(.+?)\s*(=)\s*(.+)\s*')
option_regexp2 = re.compile(r'\s*(.+?)\s*(:)\s*(.+)\s*')
option_regexp1 = re.compile(r'\s*(.+?)\s*(=)(.*)')
option_regexp2 = re.compile(r'\s*(.+?)\s*(:)(.*)')
def set_option(file_name, sections=None, summary=True):

View File

@ -19,7 +19,7 @@ def __virtual__():
Only load the module if iptables is installed
'''
if salt.utils.which('iptables'):
return 'iptables'
return True
return False

View File

@ -15,9 +15,8 @@ def __virtual__():
Only work on supported POSIX-like systems
'''
if __grains__['os_family'] in ('Arch', 'Redhat', 'Debian', 'Gentoo'):
return 'keyboard'
else:
return False
return True
return False
def get_sys():

View File

@ -15,7 +15,7 @@ def __virtual__():
'''
Only runs on Linux systems
'''
return 'kmod' if __grains__['kernel'] == 'Linux' else False
return __grains__['kernel'] == 'Linux'
def _new_mods(pre_mods, post_mods):

View File

@ -18,7 +18,7 @@ def __virtual__():
'''
if salt.utils.is_windows():
return False
return 'locate'
return True
def version():

View File

@ -26,7 +26,7 @@ def __virtual__():
'''
if salt.utils.is_windows():
return False
return 'logrotate'
return True
def _parse_conf(conf_file=default_conf):

View File

@ -164,8 +164,8 @@ def init(name,
salt 'minion' lxc.init name [cpuset=cgroups_cpuset] \\
[cpushare=cgroups_cpushare] [memory=cgroups_memory] \\
[nic=nic_profile] [profile=lxc_profile] \\
[nic_opts=nic_opts] [start=(true|false)] \\
[seed=(true|false)] [install=(true|false)] \\
[nic_opts=nic_opts] [start=(True|False)] \\
[seed=(True|False)] [install=(True|False)] \\
[config=minion_config]
name
@ -194,10 +194,10 @@ def init(name,
Start the newly created container.
seed
Seed the container with the minion config. Default: true
Seed the container with the minion config. Default: ``True``
install
If salt-minion is not already installed, install it. Default: true
If salt-minion is not already installed, install it. Default: ``True``
config
Optional config parameters. By default, the id is set to the name of the
@ -628,7 +628,7 @@ def destroy(name, stop=True):
.. code-block:: bash
salt '*' lxc.destroy name [stop=(true|false)]
salt '*' lxc.destroy name [stop=(True|False)]
'''
if stop:
_change_state('lxc-stop', name, 'stopped')
@ -1018,16 +1018,16 @@ def bootstrap(name, config=None, approve_key=True, install=True):
.. code-block:: bash
salt 'minion' lxc.bootstrap name [config=config_data] \\
[approve_key=(true|false)] [install=(true|false)]
[approve_key=(True|False)] [install=(True|False)]
config
Minion configuration options. By default, the 'master' option is set to
the target host's 'master'.
Minion configuration options. By default, the ``master`` option is set
to the target host's master.
approve_key
Request a pre-approval of the generated minion key. Requires
that the salt-master be configured to either auto-accept all keys or
expect a signing request from the target host. Default: true.
expect a signing request from the target host. Default: ``True``
install
Whether to attempt a full installation of salt-minion if needed.
@ -1067,9 +1067,9 @@ def run_cmd(name, cmd, no_start=False, preserve_state=True,
.. code-block:: bash
salt 'minion' name command [no_start=(true|false)] \\
[preserve_state=(true|false)] [stdout=(true|alse)] \\
[stderr=(true|false)]
salt 'minion' name command [no_start=(True|False)] \\
[preserve_state=(True|False)] [stdout=(True|False)] \\
[stderr=(True|False)]
name
Name of the container on which to operate.
@ -1078,22 +1078,25 @@ def run_cmd(name, cmd, no_start=False, preserve_state=True,
Command to run
no_start
If the container is not running, don't start it. Default: false.
If the container is not running, don't start it. Default: ``False``
preserve_state
After running the command, return the container to its previous
state. Default: true.
state. Default: ``True``
stdout:
Return stdout. Default: true
Return stdout. Default: ``True``
stderr:
Return stderr. Default: false
Return stderr. Default: ``False``
Note: If stderr and stdout are both false, the return code is returned. If
stderr and stdout are both true, the pid and return code are also returned.
.. note::
If stderr and stdout are both ``False``, the return code is returned.
If stderr and stdout are both ``True``, the pid and return code are
also returned.
'''
prior_state = _ensure_running(name)
prior_state = _ensure_running(name, no_start=no_start)
if not prior_state:
return prior_state

View File

@ -40,8 +40,7 @@ def __virtual__():
'''
Always load
'''
return 'modjk'
return True
def _auth(url, user, passwd, realm):

View File

@ -153,7 +153,7 @@ def __virtual__():
Only load this module if the mysql libraries exist
'''
if HAS_MYSQLDB:
return 'mysql'
return True
return False
@ -208,19 +208,22 @@ def _connect(**kwargs):
'''
connargs = dict()
def _connarg(name, key=None):
def _connarg(name, key=None, get_opts=True):
'''
Add key to connargs, only if name exists in our kwargs or as
mysql.<name> in __opts__ or __pillar__ Evaluate in said order - kwargs,
opts then pillar. To avoid collision with other functions, kwargs-based
connection arguments are prefixed with 'connection_' (i.e.
'connection_host', 'connection_user', etc.).
Add key to connargs, only if name exists in our kwargs or,
if get_opts is true, as mysql.<name> in __opts__ or __pillar__
If get_opts is true, evaluate in said order - kwargs, opts
then pillar. To avoid collision with other functions,
kwargs-based connection arguments are prefixed with 'connection_'
(i.e. 'connection_host', 'connection_user', etc.).
'''
if key is None:
key = name
if name in kwargs:
connargs[key] = kwargs[name]
else:
elif get_opts:
prefix = 'connection_'
if name.startswith(prefix):
try:
@ -231,15 +234,22 @@ def _connect(**kwargs):
if val is not None:
connargs[key] = val
_connarg('connection_host', 'host')
_connarg('connection_user', 'user')
_connarg('connection_pass', 'passwd')
_connarg('connection_port', 'port')
_connarg('connection_db', 'db')
_connarg('connection_conv', 'conv')
_connarg('connection_unix_socket', 'unix_socket')
_connarg('connection_default_file', 'read_default_file')
_connarg('connection_default_group', 'read_default_group')
# If a default file is explicitly passed to kwargs, don't grab the
# opts/pillar settings, as it can override info in the defaults file
if 'connection_default_file' in kwargs:
get_opts = False
else:
get_opts = True
_connarg('connection_host', 'host', get_opts)
_connarg('connection_user', 'user', get_opts)
_connarg('connection_pass', 'passwd', get_opts)
_connarg('connection_port', 'port', get_opts)
_connarg('connection_db', 'db', get_opts)
_connarg('connection_conv', 'conv', get_opts)
_connarg('connection_unix_socket', 'unix_socket', get_opts)
_connarg('connection_default_file', 'read_default_file', get_opts)
_connarg('connection_default_group', 'read_default_group', get_opts)
# MySQLdb states that this is required for charset usage
# but in fact it's more than it's internally activated
# when charset is used, activating use_unicode here would

View File

@ -18,7 +18,7 @@ def __virtual__():
'''
if not salt.utils.which('showmount'):
return False
return 'nfs3'
return True
def list_exports(exports='/etc/exports'):

View File

@ -75,15 +75,20 @@ def _auth(profile=None):
tenant = credentials['keystone.tenant']
auth_url = credentials['keystone.auth_url']
region_name = credentials.get('keystone.region_name', None)
api_key = credentials.get('keystone.api_key', None)
os_auth_system = credentials.get('keystone.os_auth_system', None)
else:
user = __salt__['config.option']('keystone.user')
password = __salt__['config.option']('keystone.password')
tenant = __salt__['config.option']('keystone.tenant')
auth_url = __salt__['config.option']('keystone.auth_url')
region_name = __salt__['config.option']('keystone.region_name')
api_key = __salt__['config.option']('keystone.api_key')
os_auth_system = __salt__['config.option']('keystone.os_auth_system')
kwargs = {
'username': user,
'api_key': password,
'password': password,
'api_key': api_key,
'project_id': tenant,
'auth_url': auth_url,
'region_name': region_name
@ -231,7 +236,6 @@ def volume_delete(name, profile=None):
def volume_detach(name,
server_name,
profile=None,
timeout=300):
'''
@ -256,7 +260,6 @@ def volume_detach(name,
conn = _auth(profile)
return conn.volume_detach(
name,
server_name,
timeout
)
@ -386,7 +389,7 @@ def flavor_list(profile=None):
def flavor_create(name, # pylint: disable=C0103
id=0, # pylint: disable=C0103
flavor_id=0, # pylint: disable=C0103
ram=0,
disk=0,
vcpus=1,
@ -397,7 +400,7 @@ def flavor_create(name, # pylint: disable=C0103
name
Name of the new flavor (must be first)
id
flavor_id
Unique integer ID for the new flavor
ram
Memory size in MB
@ -410,19 +413,20 @@ def flavor_create(name, # pylint: disable=C0103
.. code-block:: bash
salt '*' nova.flavor_create myflavor id=6 ram=4096 disk=10 vcpus=1
salt '*' nova.flavor_create myflavor flavor_id=6 \
ram=4096 disk=10 vcpus=1
'''
conn = _auth(profile)
return conn.flavor_create(
name,
id,
flavor_id,
ram,
disk,
vcpus
)
def flavor_delete(id, profile=None): # pylint: disable=C0103
def flavor_delete(flavor_id, profile=None): # pylint: disable=C0103
'''
Delete a flavor from nova by id (nova flavor-delete)
@ -433,7 +437,7 @@ def flavor_delete(id, profile=None): # pylint: disable=C0103
salt '*' nova.flavor_delete 7
'''
conn = _auth(profile)
return conn.flavor_delete(id)
return conn.flavor_delete(flavor_id)
def keypair_list(profile=None):
@ -499,7 +503,7 @@ def image_list(name=None, profile=None):
return conn.image_list(name)
def image_meta_set(id=None,
def image_meta_set(image_id=None,
name=None,
profile=None,
**kwargs): # pylint: disable=C0103
@ -510,19 +514,19 @@ def image_meta_set(id=None,
.. code-block:: bash
salt '*' nova.image_meta_set id=6f52b2ff-0b31-4d84-8fd1-af45b84824f6 \
salt '*' nova.image_meta_set 6f52b2ff-0b31-4d84-8fd1-af45b84824f6 \
cheese=gruyere
salt '*' nova.image_meta_set name=myimage salad=pasta beans=baked
'''
conn = _auth(profile)
return conn.image_meta_set(
id,
image_id,
name,
**kwargs
)
def image_meta_delete(id=None, # pylint: disable=C0103
def image_meta_delete(image_id=None, # pylint: disable=C0103
name=None,
keys=None,
profile=None):
@ -535,12 +539,12 @@ def image_meta_delete(id=None, # pylint: disable=C0103
.. code-block:: bash
salt '*' nova.image_meta_delete \
id=6f52b2ff-0b31-4d84-8fd1-af45b84824f6 keys=cheese
6f52b2ff-0b31-4d84-8fd1-af45b84824f6 keys=cheese
salt '*' nova.image_meta_delete name=myimage keys=salad,beans
'''
conn = _auth(profile)
return conn.image_meta_data(
id,
return conn.image_meta_delete(
image_id,
name,
keys
)

View File

@ -37,7 +37,7 @@ __func_alias__ = {
def __virtual__():
if _quote is None and not HAS_DEPS:
return False
return 'openstack_config'
return True
def _fallback(*args, **kw):

View File

@ -16,6 +16,7 @@ reference the man page for ``sfdisk(8)``.
# Import python libs
import os
import stat
import string
import logging
@ -696,3 +697,24 @@ def toggle(device, partition, flag):
cmd = 'parted -m -s {0} toggle {1} {2} {3}'.format(device, partition, flag)
out = __salt__['cmd.run'](cmd).splitlines()
return out
def exists(device=''):
'''
partition.exists device
Check to see if the partition exists
CLI Example:
.. code-block:: bash
salt '*' partition.exists /dev/sdb1
'''
if os.path.exists(device):
dev = os.stat(device).st_mode
if stat.S_ISBLK(dev):
return True
return False

View File

@ -57,7 +57,7 @@ def __virtual__():
Only load this module if the psql bin exists
'''
if all((salt.utils.which('psql'), HAS_ALL_IMPORTS)):
return 'postgres'
return True
return False

View File

@ -20,9 +20,6 @@ try:
except ImportError:
HAS_PSUTIL = False
# Define the module's virtual name
__virtualname__ = 'ps'
def __virtual__():
if not HAS_PSUTIL:
@ -36,7 +33,7 @@ def __virtual__():
# most distributions have already moved to later versions (for example,
# as of Dec. 2013 EPEL is on 0.6.1, Debian 7 is on 0.5.1, etc.).
if psutil.version_info >= (0, 3, 0):
return __virtualname__
return True
return False

View File

@ -16,7 +16,7 @@ def __virtual__():
Only available on systems with Riak installed.
'''
if salt.utils.which('riak'):
return 'riak'
return True
return False

View File

@ -46,7 +46,7 @@ def __virtual__():
'''
Should work on any modern Python installation
'''
return 's3'
return True
def delete(bucket, path=None, action=None, key=None, keyid=None,

View File

@ -15,7 +15,7 @@ import salt.utils
def __virtual__():
return 'shadow' if __grains__.get('kernel', '') == 'Linux' else False
return __grains__.get('kernel', '') == 'Linux'
def default_hash():

View File

@ -16,7 +16,7 @@ except ImportError:
def __virtual__():
if not HAS_SQLITE3:
return False
return 'sqlite3'
return True
def _connect(db=None):

View File

@ -719,6 +719,7 @@ def single(fun, name, test=None, queue=False, **kwargs):
__context__['retcode'] = 1
return err
st_._mod_init(kwargs)
ret = {'{0[state]}_|-{0[__id__]}_|-{0[name]}_|-{0[fun]}'.format(kwargs):
st_.call(kwargs)}
_set_retcode(ret)

View File

@ -21,7 +21,7 @@ __opts__ = {}
def __virtual__():
if salt.utils.is_windows():
return False
return 'status'
return True
def _number(text):

View File

@ -19,7 +19,7 @@ def __virtual__():
Only load if svn is installed
'''
if utils.which('svn'):
return 'svn'
return True
return False
@ -441,6 +441,7 @@ def export(cwd,
user=None,
username=None,
password=None,
revision='HEAD',
*opts):
'''
Create an unversioned copy of a tree.
@ -475,4 +476,6 @@ def export(cwd,
opts += (remote,)
if target:
opts += (target,)
revision_args = '-r'
opts += (revision_args, str(revision),)
return _run_svn('export', cwd, user, username, password, opts)

View File

@ -12,7 +12,7 @@ def __virtual__():
'''
if salt.utils.is_windows() or not salt.utils.which('shutdown'):
return False
return 'system'
return True
def halt():

View File

@ -20,7 +20,7 @@ def __virtual__():
'''
if salt.utils.is_windows():
return False
return 'timezone'
return True
def get_zone():

View File

@ -38,7 +38,7 @@ def __virtual__():
Only load this module if the ca config options are set
'''
if HAS_SSL:
return 'tls'
return True
return False

View File

@ -93,7 +93,7 @@ def install(feature, recurse=False):
sub = ''
if recurse:
sub = '-IncludeAllSubFeature'
out = _srvmgr('"Add-WindowsFeature -Name {0} {1} -erroraction silentlycontinue | format-list"'.format(feature, sub))
out = _srvmgr('Add-WindowsFeature -Name {0} {1} -erroraction silentlycontinue | format-list'.format(feature, sub))
return _parse_powershell_list(out)
@ -114,5 +114,5 @@ def remove(feature):
salt -t 600 '*' win_servermanager.remove Telnet-Client
'''
out = _srvmgr('"Remove-WindowsFeature -Name {0} -erroraction silentlycontinue | format-list"'.format(feature))
out = _srvmgr('Remove-WindowsFeature -Name {0} -erroraction silentlycontinue | format-list'.format(feature))
return _parse_powershell_list(out)

View File

@ -225,7 +225,8 @@ def latest_version(*names, **kwargs):
# Get updates for specified package(s)
repo_arg = _get_repo_options(**kwargs)
updates = _repoquery_pkginfo(
'{0} --pkgnarrow=available {1}'.format(repo_arg, ' '.join(names))
'{0} --pkgnarrow=available --plugins {1}'
.format(repo_arg, ' '.join(names))
)
for name in names:
@ -356,7 +357,7 @@ def list_repo_pkgs(*args, **kwargs):
ret = {}
for repo in repos:
repoquery_cmd = '--all --repoid="{0}"'.format(repo)
repoquery_cmd = '--all --repoid="{0}" --plugins'.format(repo)
for arg in args:
repoquery_cmd += ' "{0}"'.format(arg)
all_pkgs = _repoquery_pkginfo(repoquery_cmd)
@ -382,7 +383,9 @@ def list_upgrades(refresh=True, **kwargs):
refresh_db()
repo_arg = _get_repo_options(**kwargs)
updates = _repoquery_pkginfo('{0} --all --pkgnarrow=updates'.format(repo_arg))
updates = _repoquery_pkginfo(
'{0} --all --pkgnarrow=updates --plugins'.format(repo_arg)
)
return dict([(x.name, x.version) for x in updates])
@ -409,7 +412,8 @@ def check_db(*names, **kwargs):
salt '*' pkg.check_db <package1> <package2> <package3> fromrepo=epel-testing
'''
repo_arg = _get_repo_options(**kwargs)
repoquery_base = '{0} --all --quiet --whatprovides'.format(repo_arg)
repoquery_base = \
'{0} --all --quiet --whatprovides --plugins'.format(repo_arg)
if 'pkg._avail' in __context__:
avail = __context__['pkg._avail']
@ -417,7 +421,7 @@ def check_db(*names, **kwargs):
# get list of available packages
avail = []
lines = _repoquery(
'{0} --pkgnarrow=all --all'.format(repo_arg),
'{0} --pkgnarrow=all --all --plugins'.format(repo_arg),
query_format='%{NAME}_|-%{ARCH}'
)
for line in lines:
@ -435,7 +439,7 @@ def check_db(*names, **kwargs):
for name in names:
ret.setdefault(name, {})['found'] = name in avail
if not ret[name]['found']:
repoquery_cmd = repoquery_base + ' {0!r}'.format(name)
repoquery_cmd = repoquery_base + ' {0}'.format(name)
provides = set(x.name for x in _repoquery_pkginfo(repoquery_cmd))
if provides:
for pkg in provides:

View File

@ -488,14 +488,15 @@ class Pillar(object):
)
return pillar
def compile_pillar(self):
def compile_pillar(self, ext=True):
'''
Render the pillar data and return
'''
top, terrors = self.get_top()
matches = self.top_matches(top)
pillar, errors = self.render_pillar(matches)
self.ext_pillar(pillar)
if ext:
self.ext_pillar(pillar)
errors.extend(terrors)
if self.opts.get('pillar_opts', True):
mopts = dict(self.opts)

View File

@ -108,7 +108,10 @@ log = logging.getLogger(__name__)
def __virtual__():
return 'django_orm'
'''
Always load
'''
return True
def ext_pillar(minion_id,

View File

@ -171,7 +171,7 @@ except ImportError:
def __virtual__():
if not HAS_MYSQL:
return False
return 'mysql'
return True
def _get_options():

View File

@ -80,7 +80,7 @@ log = logging.getLogger(__name__)
def __virtual__():
if not HAS_MYSQL:
return False
return 'mysql'
return True
def _get_options():

View File

@ -14,7 +14,10 @@ import salt.wheel
def __virtual__():
return 'doc'
'''
Always load
'''
return True
def runner():

View File

@ -2,13 +2,20 @@
'''
Execute overstate functions
'''
# Import pytohn libs
from __future__ import print_function
import fnmatch
import json
import logging
# Import salt libs
import salt.overstate
import salt.output
import salt.syspaths
import salt.utils.event
logger = logging.getLogger(__name__)
def over(saltenv='base', os_fn=None):
@ -92,3 +99,64 @@ def show_stages(saltenv='base', os_fn=None):
'overstatestage',
opts=__opts__)
return overstate.over
def event(tagmatch='*', count=1, quiet=False, sock_dir=None):
'''
Watch Salt's event bus and block until the given tag is matched
This is useful for taking some simple action after an event is fired via
the CLI without having to use Salt's Reactor.
:param tagmatch: the event is written to stdout for each tag that matches
this pattern; uses the same matching semantics as Salt's Reactor.
:param count: this number is decremented for each event that matches the
``tagmatch`` parameter; pass ``-1`` to listen forever.
:param quiet: do not print to stdout; just block
:param sock_dir: path to the Salt master's event socket file.
CLI Examples:
.. code-block:: bash
# Reboot a minion and run highstate when it comes back online
salt 'jerry' system.reboot && \\
salt-run state.event 'salt/minion/jerry/start' quiet=True && \\
salt 'jerry' state.highstate
# Reboot multiple minions and run highstate when all are back online
salt -L 'kevin,stewart,dave' system.reboot && \\
salt-run state.event 'salt/minion/*/start' count=3 quiet=True && \\
salt -L 'kevin,stewart,dave' state.highstate
# Watch the event bus forever in a shell for-loop;
# note, slow-running tasks here will fill up the input buffer.
salt-run state.event count=-1 | while read -r tag data; do
echo $tag
echo $data | jq -colour-output .
done
wait
Enable debug logging to see ignored events.
'''
sevent = salt.utils.event.SaltEvent('master',
sock_dir or __opts__['sock_dir'])
while True:
ret = sevent.get_event(full=True)
if ret is None:
continue
if fnmatch.fnmatch(ret['tag'], tagmatch):
if not quiet:
print('{0}\t{1}'.format(ret['tag'], json.dumps(ret['data'])))
count -= 1
logger.debug('Remaining event matches: {0}'.format(count))
if count == 0:
break
else:
logger.debug('Skipping event tag: {0}'.format(ret['tag']))
continue

View File

@ -17,7 +17,7 @@ def __virtual__():
'''
Only work on apt-based platforms with pkg.get_selections
'''
return 'apt' if 'pkg.get_selections' in __salt__ else False
return 'pkg.get_selections' in __salt__
def held(name):

View File

@ -23,7 +23,7 @@ def __virtual__():
'''
Only load if the cloud module is available in __salt__
'''
return 'cloud' if 'cloud.profile' in __salt__ else False
return 'cloud.profile' in __salt__
def _check_name(name):
@ -314,6 +314,40 @@ def volume_present(name, provider=None, **kwargs):
return ret
def volume_absent(name, provider=None, **kwargs):
'''
Check that a block volume exists.
'''
ret = _check_name(name)
if not ret['result']:
return ret
volumes = __salt__['cloud.volume_list'](provider=provider)
if not name in volumes.keys():
ret['comment'] = 'Volume is absent.'
ret['result'] = True
return ret
elif __opts__['test']:
ret['comment'] = 'Volume {0} will be deleted.'.format(name)
ret['result'] = None
return ret
response = __salt__['cloud.volume_delete'](
names=name,
provider=provider,
**kwargs
)
if response:
ret['result'] = True
ret['comment'] = 'Volume {0} was deleted'.format(name)
ret['changes'] = {'old': volumes[name], 'new': response}
else:
ret['result'] = False
ret['comment'] = 'Volume {0} failed to delete.'.format(name)
return ret
def volume_attached(name, server_name, provider=None, **kwargs):
'''
Check if a block volume is attached.
@ -367,3 +401,60 @@ def volume_attached(name, server_name, provider=None, **kwargs):
ret['result'] = False
ret['comment'] = 'Volume {0} failed to attach.'.format(name)
return ret
def volume_detached(name, server_name=None, provider=None, **kwargs):
'''
Check if a block volume is attached.
'''
ret = _check_name(name)
if not ret['result']:
return ret
if not server_name is None:
ret = _check_name(server_name)
if not ret['result']:
return ret
volumes = __salt__['cloud.volume_list'](provider=provider)
if server_name:
instance = __salt__['cloud.action'](fun='show_instance', names=[name])
else:
instance = None
if name in volumes.keys() and not volumes[name]['attachments']:
volume = volumes[name]
ret['comment'] = (
'Volume {name} is not currently attached to anything.'
).format(**volumes[name])
ret['result'] = True
return ret
elif not name in volumes.keys():
ret['comment'] = 'Volume {0} does not exist'.format(name)
ret['result'] = False
return ret
elif not instance and not server_name is None:
ret['comment'] = 'Server {0} does not exist'.format(server_name)
ret['result'] = False
return ret
elif __opts__['test']:
ret['comment'] = 'Volume {0} will be will be detached.'.format(
name
)
ret['result'] = None
return ret
response = __salt__['cloud.volume_detach'](
provider=provider,
names=name,
server_name=server_name,
**kwargs
)
if response:
ret['result'] = True
ret['comment'] = 'Volume {0} was created'.format(name)
ret['changes'] = {'old': volumes[name], 'new': response}
else:
ret['result'] = False
ret['comment'] = 'Volume {0} failed to detach.'.format(name)
return ret

View File

@ -45,7 +45,7 @@ def __virtual__():
'''
Only load if the composer module is available in __salt__
'''
return 'composer' if 'composer.install' in __salt__ else False
return 'composer.install' in __salt__
def installed(name,

View File

@ -464,7 +464,7 @@ def file(name,
# If the source is a list then find which file exists
source, source_hash = __salt__['file.source_list'](source,
source_hash,
env)
__env__)
# Gather the source file from the server
try:
@ -476,7 +476,7 @@ def file(name,
owner,
group,
mode,
env,
__env__,
context,
defaults,
**kwargs
@ -503,7 +503,7 @@ def file(name,
owner,
group,
mode,
env,
__env__,
backup
)
except Exception as exc:

View File

@ -116,15 +116,6 @@ def __virtual__():
INVALID_RESPONSE = 'We did not get an acceptable answer from docker'
VALID_RESPONSE = ''
NOTSET = object()
#Use a proxy mapping to allow queries & updates after the initial grain load
MAPPING_CACHE = {}
FN_CACHE = {}
def __salt(fn):
if not fn in FN_CACHE:
FN_CACHE[fn] = __salt__[fn]
return FN_CACHE[fn]
def _ret_status(exec_status=None,
@ -188,8 +179,7 @@ def mod_watch(name, sfun=None, *args, **kw):
name=name)
installed_status = installed(name=name, **kw)
result = installed_status['result'] and remove_status['result']
comment = '\n'.join((remove_status['comment'],
installed_status['comment']))
comment = remove_status['comment']
status = _ret_status(installed_status, name=name,
result=result,
changes={name: result},
@ -230,17 +220,18 @@ def pulled(name, force=False, *args, **kwargs):
force
Pull even if the image is already pulled
'''
ins = __salt('docker.inspect_image')
iinfos = ins(name)
if iinfos['status'] and not force:
inspect_image = __salt__['docker.inspect_image']
image_infos = inspect_image(name)
if image_infos['status'] and not force:
return _valid(
name=name,
comment='Image already pulled: {0}'.format(name))
previous_id = iinfos['out']['id'] if iinfos['status'] else None
func = __salt('docker.pull')
returned = func(name)
previous_id = image_infos['out']['id'] if image_infos['status'] else None
pull = __salt__['docker.pull']
returned = pull(name)
if previous_id != returned['id']:
changes = {name: True}
changes = {name: {'old': previous_id,
'new': returned['id']}}
else:
changes = {}
return _ret_status(returned, name, changes=changes)
@ -265,15 +256,15 @@ def built(name,
or filesystem path to the dockerfile
'''
ins = __salt('docker.inspect_image')
iinfos = ins(name)
if iinfos['status'] and not force:
inspect_image = __salt__['docker.inspect_image']
image_infos = inspect_image(name)
if image_infos['status'] and not force:
return _valid(
name=name,
comment='Image already built: {0}, id: {1}'.format(
name, iinfos['out']['id']))
previous_id = iinfos['out']['id'] if iinfos['status'] else None
func = __salt('docker.build')
name, image_infos['out']['id']))
previous_id = image_infos['out']['id'] if image_infos['status'] else None
build = __salt__['docker.build']
kw = dict(tag=name,
path=path,
quiet=quiet,
@ -281,12 +272,15 @@ def built(name,
rm=rm,
timeout=timeout,
)
returned = func(**kw)
returned = build(**kw)
if previous_id != returned['id']:
changes = {name: True}
changes = {name: {'old': previous_id,
'new': returned['id']}}
else:
changes = {}
return _ret_status(returned, name, changes=changes)
return _ret_status(exec_status=returned,
name=name,
changes=changes)
def installed(name,
@ -336,9 +330,9 @@ def installed(name,
This command does not verify that the named container
is running the specified image.
'''
ins_image = __salt('docker.inspect_image')
ins_container = __salt('docker.inspect_container')
create = __salt('docker.create_container')
ins_image = __salt__['docker.inspect_image']
ins_container = __salt__['docker.inspect_container']
create = __salt__['docker.create_container']
iinfos = ins_image(image)
if not iinfos['status']:
return _invalid(comment='image "{0}" does not exist'.format(image))
@ -417,7 +411,7 @@ def absent(name):
# destroy if we found meat to do
if is_running:
__salt__['docker.stop'](cid)
is_running = __salt('docker.is_running')(cid)
is_running = __salt__['docker.is_running'](cid)
if is_running:
return _invalid(
comment=('Container {!r}'
@ -441,7 +435,7 @@ def present(name):
name:
container id
'''
ins_container = __salt('docker.inspect_container')
ins_container = __salt__['docker.inspect_container']
cinfos = ins_container(name)
if cinfos['status']:
cid = cinfos['id']
@ -588,7 +582,7 @@ def running(name, container=None, port_bindings=None, binds=None,
HostIp: ""
HostPort: "5000"
'''
is_running = __salt('docker.is_running')(container)
is_running = __salt__['docker.is_running'](container)
if is_running:
return _valid(
comment='Container {!r} is started'.format(container))

View File

@ -1923,6 +1923,9 @@ def replace(name,
count=0,
flags=0,
bufsize=1,
append_if_not_found=False,
prepend_if_not_found=False,
not_found_content=None,
backup='.bak',
show_changes=True):
'''
@ -1945,6 +1948,9 @@ def replace(name,
count=count,
flags=flags,
bufsize=bufsize,
append_if_not_found=append_if_not_found,
prepend_if_not_found=prepend_if_not_found,
not_found_content=not_found_content,
backup=backup,
dry_run=__opts__['test'],
show_changes=show_changes)

View File

@ -23,7 +23,7 @@ def __virtual__():
'''
Only load if gem module is available in __salt__
'''
return 'gem' if 'gem.list' in __salt__ else False
return 'gem.list' in __salt__
def installed(name, # pylint: disable=C0103

View File

@ -30,7 +30,7 @@ def __virtual__():
'''
Only load if git is available
'''
return 'git' if __salt__['cmd.has_exec']('git') else False
return __salt__['cmd.has_exec']('git')
def latest(name,

View File

@ -110,7 +110,7 @@ def __virtual__():
'''
Only load if the locale module is available in __salt__
'''
return 'iptables' if 'iptables.version' in __salt__ else False
return 'iptables.version' in __salt__
def chain_present(name, table='filter', family='ipv4'):

View File

@ -23,7 +23,7 @@ def __virtual__():
'''
Only load if the keyboard module is available in __salt__
'''
return 'keyboard' if 'keyboard.get_sys' in __salt__ else False
return 'keyboard.get_sys' in __salt__
def system(name):

View File

@ -19,7 +19,7 @@ def __virtual__():
'''
Only load if the kmod module is available in __salt__
'''
return 'kmod' if 'kmod.available' in __salt__ else False
return 'kmod.available' in __salt__
def present(name, persist=False):

View File

@ -16,7 +16,7 @@ def __virtual__():
'''
Only load if the locale module is available in __salt__
'''
return 'locale' if 'locale.get_locale' in __salt__ else False
return 'locale.get_locale' in __salt__
def system(name):

View File

@ -14,7 +14,7 @@ def __virtual__():
Load this state if modjk is loaded
'''
return 'modjk' if 'modjk.workers' in __salt__ else False
return 'modjk.workers' in __salt__
def _bulk_state(saltfunc, lbn, workers, profile):

View File

@ -24,7 +24,7 @@ def __virtual__():
'''
Check if we have peer access ?
'''
return 'modjk_worker'
return True
def _send_command(cmd,

View File

@ -22,7 +22,7 @@ def __virtual__():
'''
Only load if the mysql module is available in __salt__
'''
return 'mysql_database' if 'mysql.db_exists' in __salt__ else False
return 'mysql.db_exists' in __salt__
def _get_mysql_error():

View File

@ -49,7 +49,7 @@ def __virtual__():
'''
Only load if the mysql module is available
'''
return 'mysql_grants' if 'mysql.grant_exists' in __salt__ else False
return 'mysql.grant_exists' in __salt__
def _get_mysql_error():

View File

@ -26,7 +26,7 @@ def __virtual__():
'''
Only load if the mysql module is available in __salt__
'''
return 'mysql_query' if 'mysql.query' in __salt__ else False
return 'mysql.query' in __salt__
def _get_mysql_error():

View File

@ -47,7 +47,7 @@ def __virtual__():
'''
Only load if the mysql module is in __salt__
'''
return 'mysql_user' if 'mysql.user_create' in __salt__ else False
return 'mysql.user_create' in __salt__
def _get_mysql_error():

View File

@ -161,9 +161,6 @@ import salt.utils
import salt.utils.network
from salt.loader import _create_loader
# Define the module's virtual name
__virtualname__ = 'network'
# Set up logging
import logging
log = logging.getLogger(__name__)
@ -175,7 +172,7 @@ def __virtual__():
module available.
'''
if not salt.utils.is_windows() and 'ip.get_interface' in __salt__:
return __virtualname__
return True
return False

View File

@ -23,7 +23,7 @@ def __virtual__():
return False
if 'openstack_config.delete' not in __salt__:
return False
return 'openstack_config'
return True
def present(name, filename, section, value, parameter=None):

View File

@ -76,7 +76,7 @@ def __virtual__():
Only make these states available if a pkg provider has been detected or
assigned for this minion
'''
return 'pkg' if 'pkg.install' in __salt__ else False
return 'pkg.install' in __salt__
def __gen_rtag():

View File

@ -69,7 +69,7 @@ def __virtual__():
'''
Only load if modifying repos is available for this package type
'''
return 'pkgrepo' if 'pkg.mod_repo' in __salt__ else False
return 'pkg.mod_repo' in __salt__
def managed(name, **kwargs):

View File

@ -20,7 +20,7 @@ def __virtual__():
'''
Only load if the postgres module is present
'''
return 'postgres_database' if 'postgres.user_exists' in __salt__ else False
return 'postgres.user_exists' in __salt__
def present(name,

View File

@ -24,9 +24,7 @@ def __virtual__():
'''
Only load if the postgres module is present
'''
return 'postgres_extension' if (
'postgres.create_extension' in __salt__
) else False
return 'postgres.create_extension' in __salt__
def present(name,

View File

@ -28,9 +28,7 @@ def __virtual__():
'''
Only load if the postgres module is present
'''
return 'postgres_group' if (
'postgres.group_create' in __salt__
) else False
return 'postgres.group_create' in __salt__
def present(name,

View File

@ -28,9 +28,7 @@ def __virtual__():
'''
Only load if the postgres module is present
'''
return 'postgres_user' if (
'postgres.user_exists' in __salt__
) else False
return 'postgres.user_exists' in __salt__
def present(name,

View File

@ -14,7 +14,7 @@ Ensure a process matching a given pattern is absent.
def __virtual__():
return 'process' if 'ps.pkill' in __salt__ else False
return 'ps.pkill' in __salt__
def absent(name):

View File

@ -16,6 +16,9 @@ Example:
# Import python libs
import logging
# Import salt libs
import salt.utils
log = logging.getLogger(__name__)
@ -23,10 +26,7 @@ def __virtual__():
'''
Only load if RabbitMQ is installed.
'''
name = 'rabbitmq_cluster'
if not __salt__['cmd.has_exec']('rabbitmqctl'):
name = False
return name
return salt.utils.which('rabbitmqctl') is not None
def join(name, host, user='rabbit', runas=None):

Some files were not shown because too many files have changed in this diff Show More