valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-07 00:55:19 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Typos, white-space, unused/missing imports, wrapping, PEP-8.

Browse Source
This commit is contained in:
Pedro Algarvio 2011-05-26 12:17:02 +01:00
parent 3f157549c2
commit 7ccb5e4cec
38 changed files with 262 additions and 194 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

52
README
View File

@ -1,12 +1,50 @@
What is Salt?
We’re not just talking about NaCl.
Salt is a distributed remote execution system used to execute commands and query data. It was developed in order to bring the best solutions found in the world of remote execution together and make them better, faster and more malleable. Salt accomplishes this via its ability to handle larger loads of information, and not just dozens, but hundreds or even thousands of individual servers, handle them quickly and through a simple and manageable interface.
Versatility between massive scale deployments and smaller systems may seem daunting, but Salt is very simple to set up and maintain, regardless of the size of the project. The architecture of Salt is designed to work with any number of servers, from a handful of local network systems to international deployments across disparate datacenters. The topology is a simple server/client model with the needed functionality built into a single set of daemons. While the default configuration will work with little to no modification, salt can be fine tuned to meet specific needs.
The core function of salt is to enable remote commands to be called in parallel rather than in serial, to use a secure and encrypted protocol, the smallest and fastest network payloads possible and with a simple programmer interface. Salt also introduces more granular controls to the realm of remote execution, allowing for commands to be executed in parallel and for systems to be targeted based on more than just hostname, but by system properties.
Salt takes advantage of a number of technologies and techniques. The networking layer is built with the excellent zeromq networking library, so salt itself contains a viable, and transparent, amq broker inside the daemon. Salt uses public keys for authentication with the master daemon, then uses faster aes encryption for payload communication, this means that authentication and encryption are also built into Salt. Salt takes advantage of communication via python pickles, enabling fast and light network traffic.
In order to allow for simple expansion, Salt execution routines can be written as plain python modules, and the data collected from salt executions can be sent back to the master server, or to any arbitrary program. Salt can be called from a simple python api, or from the command line, so that salt can be used to execute one-off commands as well as operate as an integral part of a larger application.
The result is a system that can execute commands across groups of varying size, from very few to very many servers at considerably high speed. A system that is very fast, easy to set up and amazingly malleable, able to suit the needs of any number of servers working within the same system. Salt’s unique architecture brings together the best of the remote execution world, amplifies its capabilities and expands its range, resulting in this system that is as versatile as it is practical, able to suit any network.
Salt is developed under the Apache 2.0 licence, and can be used for open and proprietary projects. Please submit your expansions to back so that we can all benefit together as Salt grows. So, please feel free to sprinkle some of this around your systems and let the deliciousness come forth.
Salt is a distributed remote execution system used to execute commands and query
data. It was developed in order to bring the best solutions found in the world
of remote execution together and make them better, faster and more malleable.
Salt accomplishes this via its ability to handle larger loads of information,
and not just dozens, but hundreds or even thousands of individual servers,
handle them quickly and through a simple and manageable interface.
Versatility between massive scale deployments and smaller systems may seem
daunting, but Salt is very simple to set up and maintain, regardless of the
size of the project. The architecture of Salt is designed to work with any
number of servers, from a handful of local network systems to international
deployments across disparate data-centers. The topology is a simple server/client
model with the needed functionality built into a single set of daemons.
While the default configuration will work with little to no modification, salt
can be fine tuned to meet specific needs.
The core function of salt is to enable remote commands to be called in parallel
rather than in serial, to use a secure and encrypted protocol, the smallest and
fastest network payloads possible and with a simple programmer interface.
Salt also introduces more granular controls to the realm of remote execution,
allowing for commands to be executed in parallel and for systems to be targeted
based on more than just hostname, but by system properties.
Salt takes advantage of a number of technologies and techniques. The networking
layer is built with the excellent zeromq networking library, so salt itself
contains a viable, and transparent, amq broker inside the daemon. Salt uses
public keys for authentication with the master daemon, then uses faster aes
encryption for payload communication, this means that authentication and
encryption are also built into Salt. Salt takes advantage of communication via
python pickles, enabling fast and light network traffic.
In order to allow for simple expansion, Salt execution routines can be written
as plain python modules, and the data collected from salt executions can be sent
back to the master server, or to any arbitrary program. Salt can be called from
a simple python api, or from the command line, so that salt can be used to
execute one-off commands as well as operate as an integral part of a larger
application.
The result is a system that can execute commands across groups of varying size,
from very few to very many servers at considerably high speed. A system that is
very fast, easy to set up and amazingly malleable, able to suit the needs of any
number of servers working within the same system. Salt’s unique architecture
brings together the best of the remote execution world, amplifies its
capabilities and expands its range, resulting in this system that is as
versatile as it is practical, able to suit any network.
Salt is developed under the Apache 2.0 licence, and can be used for open and
proprietary projects. Please submit your expansions to back so that we can all
benefit together as Salt grows. So, please feel free to sprinkle some of this
around your systems and let the deliciousness come forth.

15
doc/man/salt-cp.1
View File

@ -24,23 +24,28 @@ The timeout in seconds to wait for replies from the salt minions.
.TP
-E, --pcre
The target expresion will be interpereted as a pcre regular expression rather than a shell glob.
The target expression will be interpreted as a pcre regular expression rather than a shell glob.
.TP
-L, --list
The target expression will be interpereted as a comma delimited list, example: server1.foo.bar,server2.foo.bar,example7.quo.qux
The target expression will be interpreted as a comma delimited list,
example: server1.foo.bar,server2.foo.bar,example7.quo.qux
.TP
-G, --grain
The target expression matches values returned by the salt grains system on the minions. The target expresion is in the format of '<grain value>:<pcre regular expresion>'; example: 'os:Arch.*'
The target expression matches values returned by the salt grains system on the
minions. The target expression is in the format of '<grain value>:<pcre regular expression>';
example: 'os:Arch.*'
.TP
-Q, --query
Execute a salt command query, this can be used to find the results os a previous function call: -Q test.echo')
Execute a salt command query, this can be used to find the results os a previous
function call: -Q test.echo')
.TP
-c CONFIG, --config=CONFIG
The location of the salt master configuration file, the salt master settings are required to know where the connections are; default=/etc/salt/master
The location of the salt master configuration file, the salt master settings are
required to know where the connections are; default=/etc/salt/master
.SH AUTHORS
Thomas S. Hatch <thatch@gmail.com>

3
doc/man/salt-key.1
View File

@ -32,7 +32,8 @@ Accepts all pending public keys.
.TP
-c CONFIG, --config=CONFIG
The master configuration file needs to be read to determine where the salt keys are stored via the pki_dir configuration value; default=/etc/salt/master
The master configuration file needs to be read to determine where the salt keys
are stored via the pki_dir configuration value; default=/etc/salt/master
.SH AUTHORS

5
doc/man/salt-minion.1
View File

@ -1,13 +1,14 @@
.TH salt-minion 1 "May 2011" "salt-minion 0.8.7" "salt-minion Manual"
.SH NAME
salt-minion \- The salt minion daemon, recieves commands from a remote salt master.
salt-minion \- The salt minion daemon, receives commands from a remote salt master.
.SH SYNOPSIS
.B salt-minion [ options ]
.SH DESCRIPTION
The salt minion recieves commands from the central salt master and replies with the results of said commands.
The salt minion receives commands from the central salt master and replies with
the results of said commands.
.SH OPTIONS
.TP

19
doc/man/salt.1
View File

@ -13,7 +13,8 @@ salt \- Parallel remote execution system
.B salt -Q test.ping
.SH DESCRIPTION
Salt allows for commands to be executed across a swath of remote systems in parallel. This means that remote systems can be both controleed and querried with ease.
Salt allows for commands to be executed across a swath of remote systems in parallel.
This means that remote systems can be both controlled and queried with ease.
.SH OPTIONS
.TP
@ -26,23 +27,29 @@ The timeout in seconds to wait for replies from the salt minions.
.TP
-E, --pcre
The target expresion will be interpereted as a pcre regular expression rather than a shell glob.
The target expression will be interpreted as a pcre regular expression rather than
a shell glob.
.TP
-L, --list
The target expression will be interpereted as a comma delimited list, example: server1.foo.bar,server2.foo.bar,example7.quo.qux
The target expression will be interpreted as a comma delimited list,
example: server1.foo.bar,server2.foo.bar,example7.quo.qux
.TP
-G, --grain
The target expression matches values returned by the salt grains system on the minions. The target expresion is in the format of '<grain value>:<pcre regular expresion>'; example: 'os:Arch.*'
The target expression matches values returned by the salt grains system on the
minions. The target expression is in the format of '<grain value>:<pcre regular expression>';
example: 'os:Arch.*'
.TP
-Q, --query
Execute a salt command query, this can be used to find the results os a previous function call: -Q test.echo')
Execute a salt command query, this can be used to find the results os a previous
function call: -Q test.echo')
.TP
-c CONFIG, --config=CONFIG
The location of the salt master configuration file, the salt master settings are required to know where the connections are; default=/etc/salt/master
The location of the salt master configuration file, the salt master settings are
required to know where the connections are; default=/etc/salt/master
.SH AUTHORS
Thomas S. Hatch <thatch@gmail.com>

25
doc/man/salt.7
View File

@ -11,13 +11,17 @@ salt \- The salt distributed system controller
.B salt [options]
.SH DESCRIPTION
Salt is a powerful remote execution manager that can be used to administer servers in a fast and efficient way.
Salt is a powerful remote execution manager that can be used to administer
servers in a fast and efficient way.
.SH SETTING UP SALT
The Salt system setup is amazingly simple, as this is one of the central design goals of Salt. Setting up Salt only requires that the salt master be running and the salt minions point to the salt master.
The Salt system setup is amazingly simple, as this is one of the central design
goals of Salt. Setting up Salt only requires that the salt master be running and
the salt minions point to the salt master.
.SS INSTALATION
The salt install is simple, for many distributions pakages should be available for download, or salt can be installed via the python packaging tools.
The salt install is simple, for many distributions packages should be available
for download, or salt can be installed via the python packaging tools.
.P
Installing salt from source is a simple task, download the latest source tarball from
.URL "https://github.com/thatch45/salt/downloads"
@ -31,15 +35,16 @@ Next untar the download and run the python setup:
# python2 setup.py install
.SS SALT DEPENDENCIES
This is a basic python setup, nothing fancy. Salt does require a number of dependencies though, all of which should be available in your distribution's packages.
This is a basic python setup, nothing fancy. Salt does require a number of
dependencies though, all of which should be available in your distribution's packages.
.TP
\(bu
.TP
\(bu
python2.7 - python2.6 should work, but it is untested
.TP
.TP
\(bu
pyzmq - zeromq python bindings
.TP
.TP
\(bu
python-m2crypto - Crypto backend
.TP
@ -50,7 +55,9 @@ python-yaml - Reading configuration files
pycrypto - Another crypto backend
.SS RUNNING SALT
To run Salt you need to ensure that a master and a minion are running and referencing each other. Starting the master and minion daemons is done with the respective commands.
To run Salt you need to ensure that a master and a minion are running and
referencing each other. Starting the master and minion daemons is done with the
respective commands.
On the salt master server:

2
salt/__init__.py
View File

@ -10,7 +10,7 @@ import salt.config
def verify_env(dirs):
'''
Verify that the named direcotries are in place and that the environment
Verify that the named directories are in place and that the environment
can shake the salt
'''
for dir_ in dirs:

16
salt/cli/__init__.py
View File

@ -25,7 +25,7 @@ class SaltCMD(object):
'''
def __init__(self):
'''
Cretae a SaltCMD object
Create a SaltCMD object
'''
self.opts = self.__parse()
@ -34,7 +34,7 @@ class SaltCMD(object):
Parse the command line
'''
parser = optparse.OptionParser()
parser.add_option('-t',
'--timeout',
default=5,
@ -63,7 +63,7 @@ class SaltCMD(object):
help='Instead of using shell globs to evaluate the target'\
+ ' use a grain value to identify targets, the syntax'\
+ ' for the target is the grain key followed by a pcre'\
+ ' regular expresion:\n"os:Arch.*"')
+ ' regular expression:\n"os:Arch.*"')
parser.add_option('-X',
'--exsel',
default=False,
@ -97,7 +97,7 @@ class SaltCMD(object):
action='store_true',
dest='raw_out',
help='Print the output from the salt command in raw python'\
+ ' form, this is suitible for re-reading the output into'\
+ ' form, this is suitable for re-reading the output into'\
+ ' an executing python script with eval.')
if JSON:
parser.add_option('--json-out',
@ -179,7 +179,7 @@ class SaltCMD(object):
args.append('exsel')
else:
args.append('glob')
if self.opts['return']:
args.append(self.opts['return'])
ret = local.cmd(*args)
@ -224,7 +224,7 @@ class SaltCP(object):
Parse the command line
'''
parser = optparse.OptionParser()
parser.add_option('-t',
'--timeout',
default=5,
@ -253,7 +253,7 @@ class SaltCP(object):
help='Instead of using shell globs to evaluate the target'\
+ ' use a grain value to identify targets, the syntax'\
+ ' for the target is the grains key followed by a pcre'\
+ ' regular expresion:\n"os:Arch.*"')
+ ' regular expression:\n"os:Arch.*"')
parser.add_option('-c',
'--config',
default='/etc/salt/master',
@ -315,7 +315,7 @@ class SaltKey(object):
default=False,
action='store_true',
help='List all public keys')
parser.add_option('-a',
'--accept',
dest='accept',

2
salt/cli/caller.py
View File

@ -1,5 +1,5 @@
'''
The caller module is used as a frontend to manage direct calls to the salt
The caller module is used as a front-end to manage direct calls to the salt
minion modules.
'''
# Import python modules

4
salt/cli/cp.py
View File

@ -1,7 +1,7 @@
'''
The cp module is used to execute the logic used by the salt-cp command
line application, salt-cp is NOT intended to broadcast large files, it is
intened to handle text files.
intended to handle text files.
Salt-cp can be used to distribute configuration files
'''
# Import python modules
@ -73,7 +73,7 @@ class SaltCP(object):
args.append('list')
elif self.opts['grain']:
args.append('grain')
ret = local.cmd(*args)
print yaml.dump(ret)

23
salt/client.py
View File

@ -1,6 +1,6 @@
'''
The client module is used to create a client connection to the publisher
The data structurte needs to be:
The data structure needs to be:
{'enc': 'clear',
'load': {'fun': '<mod.callable>',
'arg':, ('arg1', 'arg2', ...),
@ -10,17 +10,17 @@ The data structurte needs to be:
# The components here are simple, and they need to be and stay simple, we
# want a client to have 3 external concerns, and maybe a forth configurable
# option.
# The concers are
# The concerns are
# 1. Who executes the command?
# 2. what is the function being run?
# 3. What arguments need to be passed to the function?
# 4. How long do we wait for all of the replies?
#
# Next there are a number of tasks, first we need some kind of authentication
# This Client initially will be the master root client, which will run as the
# This Client initially will be the master root client, which will run as the
# root user on the master server.
# BUT we also want a client to be able to work over the network, so that
# controllers can exist within disperate applicaitons.
# controllers can exist within disparate applications.
# The problem is that this is a security nightmare, so I am going to start
# small, and only start with the ability to execute salt commands locally.
# This means that the primary client to build is, the LocalClient
@ -90,7 +90,7 @@ class LocalClient(object):
def _check_pcre_minions(self, expr):
'''
Return the minions found by looking via regular expresions
Return the minions found by looking via regular expressions
'''
ret = set()
cwd = os.getcwd()
@ -184,10 +184,10 @@ class LocalClient(object):
'grain': self._check_grain_minions,
'exsel': self._check_grain_minions,
}[expr_form](expr)
def pub(self, tgt, fun, arg=(), expr_form='glob', ret=''):
'''
Take the required arguemnts and publish the given command.
Take the required arguments and publish the given command.
Arguments:
tgt:
The tgt is a regex or a glob used to match up the ids on
@ -195,11 +195,11 @@ class LocalClient(object):
all of the minions and then the minions determine if the
command is for them based on the tgt value.
fun:
The function nane to be called on the remote host(s), this must
The function name to be called on the remote host(s), this must
be a string in the format "<modulename>.<function name>"
arg:
The arg option needs to be a tuple of arguments to pass to the
calling function, if left blank
calling function, if left blank
Returns:
jid:
A string, as returned by the publisher, which is the job id,
@ -207,7 +207,7 @@ class LocalClient(object):
minions:
A set, the targets that the tgt passed should match.
'''
# Run a check_minions, if no minons match return False
# Run a check_minions, if no minions match return False
# format the payload - make a function that does this in the payload
# module
# make the zmq client
@ -229,8 +229,7 @@ class LocalClient(object):
# Prep zmq
context = zmq.Context()
socket = context.socket(zmq.REQ)
socket.connect('tcp://' + self.opts['interface'] + ':'\
+ str(self.opts['ret_port']))
socket.connect('tcp://%(interface)s:%(ret_port)s' % self.opts)
socket.send(package)
payload = salt.payload.unpackage(socket.recv())
return {'jid': payload['load']['jid'],

4
salt/config.py
View File

@ -45,7 +45,7 @@ def minion_config(path):
opts['master_uri'] = 'tcp://' + opts['master'] + ':'\
+ str(opts['master_port'])
# Enableing open mode requires that the value be set to True, and nothing
# Enabling open mode requires that the value be set to True, and nothing
# else!
if opts['open_mode']:
if opts['open_mode'] == True:
@ -91,7 +91,7 @@ def master_config(path):
opts['aes'] = salt.crypt.Crypticle.generate_key_string()
# Enableing open mode requires that the value be set to True, and nothing
# Enabling open mode requires that the value be set to True, and nothing
# else!
if opts['open_mode']:
if opts['open_mode'] == True:

14
salt/crypt.py
View File

@ -48,7 +48,7 @@ class MasterKeys(dict):
def __get_priv_key(self):
'''
Retruns a private key object for the master
Returns a private key object for the master
'''
key = None
try:
@ -89,7 +89,7 @@ class Auth(object):
def get_priv_key(self):
'''
Retruns a private key object for the minion
Returns a private key object for the minion
'''
key = None
try:
@ -107,7 +107,7 @@ class Auth(object):
def minion_sign_in_payload(self):
'''
Generates the payload used to autnenticate with the master server. This
Generates the payload used to authenticate with the master server. This
payload consists of the passed in id_ and the ssh public key to encrypt
the AES key sent back form the master.
'''
@ -129,7 +129,7 @@ class Auth(object):
Pass in the encrypted aes key.
Returns the decrypted aes seed key, a string
'''
log.info('Decypting the current master AES key')
log.info('Decrypting the current master AES key')
key = self.get_priv_key()
return key.private_decrypt(aes, 4)
@ -137,7 +137,7 @@ class Auth(object):
'''
Takes the master pubkey and compares it to the saved master pubkey,
the token is encrypted with the master private key and must be
decrypted sucessfully to verify that the master has been connected to.
decrypted successfully to verify that the master has been connected to.
The token must decrypt with the public key, and it must say:
'salty bacon'
returns a bool
@ -274,7 +274,7 @@ class Crypticle(object):
def loads(self, data, pickler=pickle):
'''
decrypt and unpickle a python object
decrypt and un-pickle a python object
'''
data = self.decrypt(data)
# simple integrity check to verify that we got meaningful data
@ -294,7 +294,7 @@ class SAuth(object):
def __authenticate(self):
'''
Authenticate with the master, this method breaks the functional
pardigmn, it will update the master information from a fresh sign in,
paradigm, it will update the master information from a fresh sign in,
signing in can occur as often as needed to keep up with the revolving
master aes key.
'''

2
salt/grains/core.py
View File

@ -127,7 +127,7 @@ def os_data():
grains['os'] = kernel
# Load the virtual machine info
grains.update(_virtual(grains))
grains.update(_ps(grains))
return grains

6
salt/loader.py
View File

@ -81,9 +81,9 @@ def call(fun, args=[], dirs=[]):
class Loader(object):
'''
Used to load in arbitrairy modules from a directory, the Loader can also be
Used to load in arbitrary modules from a directory, the Loader can also be
used to only load specific functions from a directory, or to call modules
in an arbitrairy directory directly.
in an arbitrary directory directly.
'''
def __init__(self, module_dirs, opts={}):
self.module_dirs = module_dirs
@ -255,7 +255,7 @@ class Loader(object):
def gen_grains(self):
'''
Read the grains directory and execute all of the public callable
members. then verify that the returns are python dicts and return a
members. then verify that the returns are python dict's and return a
dict containing all of the returned values.
'''
grains = {}

2
salt/log.py
View File

@ -7,7 +7,7 @@
:copyright: © 2011 :email:`Pedro Algarvio (pedro@algarvio.me)`
:license: Apache Version 2.0, see LICENSE for more details.
:license: Apache 2.0, see LICENSE for more details.
"""
import logging

31
salt/master.py
View File

@ -19,7 +19,7 @@ import salt.utils
import salt.crypt
import salt.payload
import salt.client
# Import cryptogrogphy modules
# Import cryptography modules
from M2Crypto import RSA
log = logging.getLogger(__name__)
@ -61,7 +61,7 @@ class Master(object):
class Publisher(multiprocessing.Process):
'''
The publihing interface, a simple zeromq publisher that sends out the
The publishing interface, a simple zeromq publisher that sends out the
commands.
'''
def __init__(self, opts):
@ -115,7 +115,7 @@ class ReqServer():
A key needs to be placed in the filesystem with permissions 0400 so
clients are required to run as root.
'''
log.info('Preparing the root key for local comunication')
log.info('Preparing the root key for local communication')
keyfile = os.path.join(self.opts['cachedir'], '.root_key')
key = salt.crypt.Crypticle.generate_key_string()
if os.path.isfile(keyfile):
@ -139,7 +139,7 @@ class ReqServer():
m_worker.start()
out_socket = self.context.socket(zmq.REQ)
out_socket.connect('tcp://127.0.0.1:' + str(work_port))
out_socket.connect('tcp://127.0.0.1:%s' % work_port)
while True:
package = in_socket.recv()
@ -188,7 +188,7 @@ class MWorker(multiprocessing.Process):
'''
context = zmq.Context(1)
socket = context.socket(zmq.REP)
socket.bind('tcp://127.0.0.1:' + self.port)
socket.bind('tcp://127.0.0.1:%s' % self.port)
while True:
package = socket.recv()
@ -202,8 +202,9 @@ class MWorker(multiprocessing.Process):
job id directory.
'''
jid_root = os.path.join(self.opts['cachedir'], 'jobs')
jid = datetime.datetime.strftime(datetime.datetime.now(),
'%Y%m%d%H%M%S%f')
jid = datetime.datetime.strftime(
datetime.datetime.now(), '%Y%m%d%H%M%S%f'
)
jid_dir = os.path.join(jid_root, jid)
if not os.path.isdir(jid_dir):
os.makedirs(jid_dir)
@ -225,21 +226,21 @@ class MWorker(multiprocessing.Process):
'''
Take care of a cleartext command
'''
log.info('Clear payload recieved with command %(cmd)s', load)
log.info('Clear payload received with command %(cmd)s', load)
return getattr(self, load['cmd'])(load)
def _handle_pub(self, load):
'''
Handle a command sent via a public key pair
'''
log.info('Pubkey payload recieved with command %(cmd)s', load)
log.info('Pubkey payload received with command %(cmd)s', load)
def _handle_aes(self, load):
'''
Handle a command sent via an aes key
'''
data = self.crypticle.loads(load)
log.info('AES payload recieved with command %(cmd)s', load)
log.info('AES payload received with command %(cmd)s', load)
return getattr(self, data['cmd'])(data)
def _auth(self, load):
@ -247,7 +248,7 @@ class MWorker(multiprocessing.Process):
Authenticate the client, use the sent public key to encrypt the aes key
which was generated at start up
'''
# 1. Verify that the key we are recieving matches the stored key
# 1. Verify that the key we are receiving matches the stored key
# 2. Store the key if it is not there
# 3. make an rsa key with the pub key
# 4. encrypt the aes key as an encrypted pickle
@ -326,7 +327,7 @@ class MWorker(multiprocessing.Process):
def _serve_file(self, load):
'''
Return a chunk from a file based on the data recieved
Return a chunk from a file based on the data received
'''
if not load.has_key('path') or not load.has_key('loc'):
return False
@ -368,7 +369,7 @@ class MWorker(multiprocessing.Process):
jid_dir = os.path.join(self.opts['cachedir'], 'jobs', load['jid'])
if not os.path.isdir(jid_dir):
log.error(
'An inconsistency occured, a job was recieved with a job id '
'An inconsistency occurred, a job was received with a job id '
'that is not present on the master: %(jid)s', load
)
return False
@ -380,7 +381,7 @@ class MWorker(multiprocessing.Process):
def _send_cluster(self):
'''
Send the cluser data out
Send the cluster data out
'''
log.debug('Sending out cluster data')
ret = self.local.cmd(self.opts['cluster_masters'],
@ -430,7 +431,7 @@ class MWorker(multiprocessing.Process):
payload['load'] = self.crypticle.dumps(load)
context = zmq.Context(1)
pub_sock = context.socket(zmq.PUSH)
pub_sock.connect('tcp://127.0.0.1:' + self.opts['publish_pull_port'])
pub_sock.connect('tcp://127.0.0.1:%(publish_pull_port)s' % self.opts)
pub_sock.send(salt.payload.package(payload))
return {'enc': 'clear',
'load': {'jid': jid}}

32
salt/minion.py
View File

@ -36,7 +36,7 @@ log = logging.getLogger(__name__)
class Minion(object):
'''
This class instanciates a minion, runs connections for a minion, and loads
This class instantiates a minion, runs connections for a minion, and loads
all of the functions into the minion
'''
def __init__(self, opts):
@ -85,7 +85,7 @@ class Minion(object):
def _handle_aes(self, load):
'''
Takes the aes encrypted load, decypts is and runs the encapsulated
Takes the aes encrypted load, decrypts is and runs the encapsulated
instructions
'''
data = None
@ -117,24 +117,32 @@ class Minion(object):
def _handle_clear(self, load):
'''
Handle unencrypted transmisions
Handle un-encrypted transmissions
'''
pass
def _handle_decoded_payload(self, data):
'''
Override this method if you wish to handle the decoded data diferently.
Override this method if you wish to handle the decoded data differently.
'''
if self.opts['multiprocessing']:
if type(data['fun']) == type(list()):
multiprocessing.Process(target=lambda: self._thread_multi_return(data)).start()
multiprocessing.Process(
target=lambda: self._thread_multi_return(data)
).start()
else:
multiprocessing.Process(target=lambda: self._thread_return(data)).start()
multiprocessing.Process(
target=lambda: self._thread_return(data)
).start()
else:
if type(data['fun']) == type(list()):
threading.Thread(target=lambda: self._thread_multi_return(data)).start()
threading.Thread(
target=lambda: self._thread_multi_return(data)
).start()
else:
threading.Thread(target=lambda: self._thread_return(data)).start()
threading.Thread(
target=lambda: self._thread_return(data)
).start()
def _glob_match(self, tgt):
'''
@ -163,7 +171,7 @@ class Minion(object):
def _grain_match(self, tgt):
'''
Reads in the grains regular expresion match
Reads in the grains regular expression match
'''
comps = tgt.split(':')
return bool(re.match(comps[1], self.opts['grains'][comps[0]]))
@ -206,7 +214,7 @@ class Minion(object):
def _thread_multi_return(self, data):
'''
This methos should be used as a threading target, start the actual
This method should be used as a threading target, start the actual
minion side execution.
'''
ret = {'return': {}}
@ -262,7 +270,7 @@ class Minion(object):
def authenticate(self):
'''
Authenticate with the master, this method breaks the functional
pardigmn, it will update the master information from a fresh sign in,
paradigm, it will update the master information from a fresh sign in,
signing in can occur as often as needed to keep up with the revolving
master aes key.
'''
@ -271,7 +279,7 @@ class Minion(object):
while True:
creds = auth.sign_in()
if creds != 'retry':
log.info('Authentication with master sucessful!')
log.info('Authentication with master successful!')
break
log.info('Waiting for minion key to be accepted by the master.')
time.sleep(10)

4
salt/modules/butterkvm.py
View File

@ -1,5 +1,5 @@
'''
Specilized routines used by the butter cloud component
Specialized routines used by the butter cloud component
'''
# Import salt modules
import virt
@ -76,7 +76,7 @@ def _gen_pin_drives(pins):
def _apply_overlay(vda, instance):
'''
Use libguestfs to apply the overlay inder the specified instance to the
Use libguestfs to apply the overlay under the specified instance to the
specified vda
'''
overlay = os.path.join(instance, 'overlay')

5
salt/modules/cp.py
View File

@ -3,6 +3,7 @@ Minion side functions for salt-cp
'''
# Import python libs
import os
import hashlib
# Import salt libs
import salt.crypt
@ -14,7 +15,7 @@ def recv(files, dest):
'''
Used with salt-cp, pass the files dict, and the destination.
This function recieves small fast copy files from the master via salt-cp
This function receives small fast copy files from the master via salt-cp
'''
ret = {}
for path, data in files.items():
@ -67,7 +68,7 @@ def get_file(path, dest):
def cache_files(paths):
'''
Used to gather many files from the master, the gathered files will be
saved in the minion cachedir reflective to the paths retrived from the
saved in the minion cachedir reflective to the paths retrieved from the
master.
'''
auth = salt.crypt.SAuth(__opts__)

4
salt/modules/cytest.pyx
View File

@ -1,5 +1,5 @@
'''
Module for running arbitrairy tests
Module for running arbitrary tests
'''
import time
@ -26,7 +26,7 @@ def ping():
def fib(long num):
'''
Return a fibonachi sequence up to the passed number, and the time it took
Return a Fibonacci sequence up to the passed number, and the time it took
to compute in seconds. Used for performance tests
CLI Example:

3
salt/modules/file.py
View File

@ -10,7 +10,6 @@ data
import os
import grp
import pwd
import hashlib
def gid_to_group(gid):
'''
@ -135,7 +134,7 @@ def set_mode(path, mode):
def chown(path, user, group):
'''
Chown a file, pass the filem the desired user and group
Chown a file, pass the file the desired user and group
CLI Example:
salt '*' file.chown /etc/passwd root root

14
salt/modules/network.py
View File

@ -6,12 +6,14 @@ import socket
from string import ascii_letters, digits
def _sanitize_host(host):
return "".join([c for c in host[0:255] if c in (ascii_letters + digits + '.')])
return "".join([
c for c in host[0:255] if c in (ascii_letters + digits + '.')
])
def ping(host):
'''
Performs a ping to a host
CLI Example:
salt '*' network.ping archlinux.org -c 4
'''
@ -25,7 +27,7 @@ def ping(host):
def netstat():
'''
Return information on open ports and states
CLI Example:
salt '*' network.netstat
'''
@ -66,7 +68,7 @@ def netstat():
def traceroute(host):
'''
Performs a traceroute to a 3rd party host
CLI Example:
salt '*' network.traceroute archlinux.org
'''
@ -98,7 +100,7 @@ def traceroute(host):
def dig(host):
'''
Performs a DNS lookup with dig
CLI Example:
salt '*' network.dig archlinux.org
'''
@ -112,7 +114,7 @@ def dig(host):
def isportopen(host, port):
'''
Return status of a port
CLI Example:
salt '*' network.isportopen 127.0.0.1 22
'''

6
salt/modules/pacman.py
View File

@ -153,13 +153,13 @@ def upgrade():
pkgs[npkg] = {'old': '',
'new': new[npkg]}
return pkgs
def remove(name):
'''
Remove a single package with pacman -R
Return a list containing the removed packages:
CLI Example:
salt '*' pkg.remove <package name>
'''
@ -175,7 +175,7 @@ def purge(name):
with it, this will call a pacman -Rsc
Return a list containing the removed packages:
CLI Example:
salt '*' pkg.purge <package name>

2
salt/modules/service.py
View File

@ -38,7 +38,7 @@ def stop(name):
def status(name, sig=None):
'''
Return the status for a service, returns the PID or an empty string if the
service is running or not, pass a signiture to use to find the service via
service is running or not, pass a signature to use to find the service via
ps
CLI Example:

6
salt/modules/state.py
View File

@ -1,8 +1,6 @@
'''
Control the state system on the minion
'''
# Import Python modules
import os
# Import salt modules
import salt.state
@ -41,7 +39,7 @@ def template(tem):
'''
st_ = salt.state.State(__opts__)
return st_.call_template(tem)
def template_str(tem):
'''
Execute the information stored in a template file on the minion
@ -51,4 +49,4 @@ def template_str(tem):
'''
st_ = salt.state.State(__opts__)
return st_.call_template_str(tem)

45
salt/modules/status.py
View File

@ -7,7 +7,7 @@ import subprocess
__opts__ = {}
def custom():
'''
'''
Return a custom composite of status data and info for this minon,
based on the minion config file. An example config like might be:
@ -19,7 +19,7 @@ def custom():
This function is meant to replace all_status(), which returns
anything and everything, which we probably don't want.
By default, nothing is returned. Warning: Depending on what you
include, there can be a LOT here!
@ -38,7 +38,7 @@ def custom():
for item in __opts__[opt]:
ret[item] = vals[item]
return ret
return ret
def uptime():
'''
@ -59,7 +59,7 @@ def loadavg():
'''
comps = open('/proc/loadavg', 'r').read().strip()
load_avg = comps.split()
return {
return {
'1-min': load_avg[1],
'5-min': load_avg[2],
'15-min': load_avg[3],
@ -105,7 +105,7 @@ def cpustats():
def meminfo():
'''
Return the CPU stats for this minon
Return the CPU stats for this minion
CLI Example:
salt '*' status.meminfo
@ -125,8 +125,8 @@ def meminfo():
return ret
def cpuinfo():
'''
Return the CPU info for this minon
'''
Return the CPU info for this minion
CLI Example:
salt '*' status.cpuinfo
@ -142,11 +142,11 @@ def cpuinfo():
ret[comps[0]] = comps[1].split()
else:
ret[comps[0]] = comps[1].strip()
return ret
return ret
def diskstats():
'''
Return the disk stats for this minon
Return the disk stats for this minion
CLI Example:
salt '*' status.diskstats
@ -177,7 +177,7 @@ def diskstats():
def vmstats():
'''
Return the virtual memory stats for this minon
Return the virtual memory stats for this minion
CLI Example:
salt '*' status.vmstats
@ -192,8 +192,8 @@ def vmstats():
return ret
def netstats():
'''
Return the network stats for this minon
'''
Return the network stats for this minion
CLI Example:
salt '*' status.netstats
@ -206,22 +206,22 @@ def netstats():
continue
comps = line.split()
if comps[0] == headers[0]:
index = len(headers) - 1
index = len(headers) - 1
row = {}
for field in range(index):
if field < 1:
continue
else:
row[headers[field]] = comps[field]
rowname = headers[0].replace(':', '')
ret[rowname] = row
rowname = headers[0].replace(':', '')
ret[rowname] = row
else:
headers = comps
return ret
return ret
def netdev():
'''
Return the network device stats for this minon
Return the network device stats for this minion
CLI Example:
salt '*' status.netdev
@ -256,8 +256,8 @@ def netdev():
return ret
def w():
'''
Return a list of logged in users for this minon, using the w command
'''
Return a list of logged in users for this minion, using the w command
CLI Example:
salt '*' status.w
@ -270,7 +270,7 @@ def w():
if not row.count(' '):
continue
comps = row.split()
rec = {
rec = {
'user': comps[0],
'tty': comps[1],
'login': comps[2],
@ -278,13 +278,14 @@ def w():
'jcpu': comps[4],
'pcpu': comps[5],
'what': ' '.join(comps[6:]),
}
}
user_list.append( rec )
return user_list
def all_status():
'''
Return a composite of all status data and info for this minon. Warning: There is a LOT here!
Return a composite of all status data and info for this minoon.
Warning: There is a LOT here!
CLI Example:
salt '*' status.all_status

2
salt/modules/sysctl.py
View File

@ -1,5 +1,5 @@
'''
Module for viewing and modifying sysctl paramters
Module for viewing and modifying sysctl parameters
'''
import subprocess

4
salt/modules/test.py
View File

@ -1,5 +1,5 @@
'''
Module for running arbitrairy tests
Module for running arbitrary tests
'''
import time
@ -50,7 +50,7 @@ def get_opts():
def fib(num):
'''
Return a fibonachi sequence up to the passed number, and the time it took
Return a Fibonacci sequence up to the passed number, and the time it took
to compute in seconds. Used for performance tests
CLI Example:

17
salt/modules/virt.py
View File

@ -1,5 +1,5 @@
'''
Work with vitual machines managed by libvirt
Work with virtual machines managed by libvirt
'''
# Special Thanks to Michael Dehann, many of the concepts, and a few structures
# of his in the virt func module have been used
@ -39,7 +39,7 @@ def __get_conn():
def _get_dom(vm_):
'''
Return a domain object for the named vm
Return a domain object for the named vm
'''
conn = __get_conn()
if not list_vms().count(vm_):
@ -63,7 +63,7 @@ def _libvirt_creds():
def list_vms():
'''
Return a list of virtual machine names on the minion
CLI Example:
salt '*' virt.list_vms
'''
@ -109,7 +109,6 @@ def node_info():
salt '*' virt.node_info
'''
conn = __get_conn()
info = {}
raw = conn.getInfo()
info = {
'cpumodel' : str(raw[0]),
@ -343,7 +342,7 @@ def migrate(vm_, target):
def seed_non_shared_migrate(disks, force=False):
'''
Non shared migration reqiuires that the disks be present on the migration
Non shared migration requires that the disks be present on the migration
destination, pass the disks information via this function, to the
migration destination before executing the migration.
@ -355,7 +354,7 @@ def seed_non_shared_migrate(disks, force=False):
form = data['file format']
size = data['virtual size'].split()[1][1:]
if os.path.isfile(fn_) and not force:
# the target exists, check to see if is is compatable
# the target exists, check to see if is is compatible
pre = yaml.load(subprocess.Popen('qemu-img info arch',
shell=True,
stdout=subprocess.PIPE).communicate()[0])
@ -375,7 +374,7 @@ def seed_non_shared_migrate(disks, force=False):
def destroy(vm_):
'''
Hard power down the virtual machine, this is equivelent to pulling the
Hard power down the virtual machine, this is equivalent to pulling the
power
CLI Example:
@ -405,7 +404,7 @@ def undefine(vm_):
def purge(vm_, dirs=False):
'''
Recursively destroy and delete a virtual machine, pass True for dirs to
Recursively destroy and delete a virtual machine, pass True for dir's to
also delete the directories containing the virtual machine disk images -
USE WITH EXTREAME CAUTION!
@ -426,7 +425,7 @@ def purge(vm_, dirs=False):
def virt_type():
'''
Returns the virtual machine type as a string
CLI Example:
salt '*' virt.virt_type
'''

8
salt/modules/yum.py
View File

@ -14,7 +14,7 @@ def __virtual__():
def _list_removed(old, new):
'''
List the pachages which have been removed between the two package objects
List the packages which have been removed between the two package objects
'''
pkgs = []
for pkg in old:
@ -118,13 +118,13 @@ def upgrade():
pkgs[npkg] = {'old': '',
'new': new[npkg]}
return pkgs
def remove(pkg):
'''
Remove a single package with yum remove
Return a list containing the removed packages:
CLI Example:
salt '*' pkg.remove <package name>
'''
@ -139,7 +139,7 @@ def purge(pkg):
Yum does not have a purge, this function calls remove
Return a list containing the removed packages:
CLI Example:
salt '*' pkg.purge <package name>

6
salt/returners/redis_return.py
View File

@ -1,7 +1,7 @@
'''
Return data to a redis server
This is a VERY simple example for pushing data to a redis server and is not
nessisarily intended as a usable interface.
necessarily intended as a usable interface.
'''
import redis
@ -21,7 +21,7 @@ def returner(ret):
host=__opts__['redis.host'],
port=__opts__['redis.port'],
db=__opts__['redis.db'])
serv.sadd(ret['id'] + ':' + 'jobs', ret['jid'])
serv.set(ret['jid'] + ':' + ret['id'], json.dumps(ret['return']))
serv.sadd("%(id)s:jobs" % ret, ret['jid'])
serv.set("%(jid)s:%(id)s" % ret, json.dumps(ret['return']))
serv.sadd('jobs', ret['jid'])
serv.sadd(ret['jid'], ret['id'])

20
salt/state.py
View File

@ -1,6 +1,6 @@
'''
The module used to execute states in salt. A state is unlike a module execution
in that instead of just executing a command it ensure that a certian state is
in that instead of just executing a command it ensure that a certain state is
present on the system.
The data sent to the state calls is as follows:
@ -15,6 +15,7 @@ import sys
import os
import copy
import inspect
import tempfile
# Import Salt modules
import salt.loader
@ -71,7 +72,7 @@ class State(object):
def format_call(self, data):
'''
Formats low data into a list of dicts used to acctually call the state,
Formats low data into a list of dict's used to actually call the state,
returns:
{
'full': 'module.function',
@ -110,7 +111,7 @@ class State(object):
def compile_high_data(self, high):
'''
"Compile" the high data as it is retireved from the cli or yaml into
"Compile" the high data as it is retrieved from the cli or yaml into
the individual state executor structures
'''
chunks = []
@ -148,8 +149,8 @@ class State(object):
def compile_template(self, template):
'''
Take the path to a template and return the high data structure derived from the
template.
Take the path to a template and return the high data structure derived
from the template.
'''
if not os.path.isfile(template):
return {}
@ -157,8 +158,8 @@ class State(object):
def compile_template_str(self, template):
'''
Take the path to a template and return the high data structure derived from the
template.
Take the path to a template and return the high data structure derived
from the template.
'''
fn_ = tempfile.mkstemp()[1]
open(fn_, 'w+').write(template)
@ -168,7 +169,8 @@ class State(object):
def call(self, data):
'''
Call a state directly with the low data structure, verify data before processing
Call a state directly with the low data structure, verify data before
processing.
'''
ret = {'changes': None,
'result': False,
@ -178,7 +180,7 @@ class State(object):
def call_chunks(self, chunks):
'''
Itterate over a list of chunks and call them, checking for requires.
Iterate over a list of chunks and call them, checking for requires.
'''
running = {}
for low in chunks:

2
salt/states/cmd.py
View File

@ -37,7 +37,7 @@ def run(name,
return ret
if unless:
if __salt__['cmd.retcode'](unless) == 0:
ret['comment'] = 'unless executed sucsessfully'
ret['comment'] = 'unless executed successfully'
ret['result'] = True
return ret
if not os.path.isdir(cwd):

3
salt/states/file.py
View File

@ -3,7 +3,6 @@ Manage file states
'''
import os
import hashlib
import shutil
import tempfile
import difflib
@ -170,7 +169,7 @@ def managed(name,
'changes': changes,
'result': False,
'comment': 'Source file ' + source + ' not found'}
# Handle any template management that is neded
# Handle any template management that is needed
if template:
t_key = '_' + template
if locals().has_key(t_key):

40
salt/utils.py
View File

@ -30,34 +30,34 @@ def daemonize():
'''
Daemonize a process
'''
try:
pid = os.fork()
try:
pid = os.fork()
if pid > 0:
# exit first parent
sys.exit(0)
except OSError, e:
sys.exit(0)
except OSError, e:
print >> sys.stderr, "fork #1 failed: %d (%s)" % (e.errno, e.strerror)
sys.exit(1)
# decouple from parent environment
os.chdir("/")
os.setsid()
os.umask(022)
os.chdir("/")
os.setsid()
os.umask(022)
# do second fork
try:
pid = os.fork()
try:
pid = os.fork()
if pid > 0:
# print "Daemon PID %d" % pid
sys.exit(0)
except OSError, e:
# print "Daemon PID %d" % pid
sys.exit(0)
except OSError, e:
print >> sys.stderr, "fork #2 failed: %d (%s)" % (e.errno, e.strerror)
sys.exit(1)
sys.exit(1)
dev_null = open('/dev/null','rw')
os.dup2(dev_null.fileno(), sys.stdin.fileno())
os.dup2(dev_null.fileno(), sys.stdout.fileno())
os.dup2(dev_null.fileno(), sys.stderr.fileno())
dev_null = open('/dev/null','rw')
os.dup2(dev_null.fileno(), sys.stdin.fileno())
os.dup2(dev_null.fileno(), sys.stdout.fileno())
os.dup2(dev_null.fileno(), sys.stderr.fileno())
def check_root():
'''
@ -65,9 +65,9 @@ def check_root():
verify that root is the user before the application discovers it.
'''
if os.getuid():
print 'Sorry, the salt must run as root, it needs to opperate'\
+ ' in a privileged environment to do what it does.\n' \
+ 'http://xkcd.com/838/'
print ('Sorry, the salt must run as root, it needs to operate '
'in a privileged environment to do what it does.\n'
'http://xkcd.com/838/')
sys.exit(1)

4
scripts/salt
View File

@ -2,8 +2,8 @@
'''
Publish commands to the salt system from the command line on the master.
'''
# Eventually we need to be able to interlink salt masters and form quorums,
# and create the ability for externel systesm to talk to salt. This interface
# Eventually we need to be able to interlink salt masters and form quorum's,
# and create the ability for external systems to talk to salt. This interface
# is just a cli interface so that we can get this kicked off and working
import salt.cli

4
scripts/salt-cp
View File

@ -2,8 +2,8 @@
'''
Publish commands to the salt system from the command line on the master.
'''
# Eventually we need to be able to interlink salt masters and form quorums,
# and create the ability for externel systesm to talk to salt. This interface
# Eventually we need to be able to interlink salt masters and form quorum's,
# and create the ability for external systems to talk to salt. This interface
# is just a cli interface so that we can get this kicked off and working
import salt.cli