diff --git a/README b/README index 6a8e30d34e..2286f66f30 100644 --- a/README +++ b/README @@ -1,12 +1,50 @@ What is Salt? We’re not just talking about NaCl. -Salt is a distributed remote execution system used to execute commands and query data. It was developed in order to bring the best solutions found in the world of remote execution together and make them better, faster and more malleable. Salt accomplishes this via its ability to handle larger loads of information, and not just dozens, but hundreds or even thousands of individual servers, handle them quickly and through a simple and manageable interface. -Versatility between massive scale deployments and smaller systems may seem daunting, but Salt is very simple to set up and maintain, regardless of the size of the project. The architecture of Salt is designed to work with any number of servers, from a handful of local network systems to international deployments across disparate datacenters. The topology is a simple server/client model with the needed functionality built into a single set of daemons. While the default configuration will work with little to no modification, salt can be fine tuned to meet specific needs. -The core function of salt is to enable remote commands to be called in parallel rather than in serial, to use a secure and encrypted protocol, the smallest and fastest network payloads possible and with a simple programmer interface. Salt also introduces more granular controls to the realm of remote execution, allowing for commands to be executed in parallel and for systems to be targeted based on more than just hostname, but by system properties. -Salt takes advantage of a number of technologies and techniques. The networking layer is built with the excellent zeromq networking library, so salt itself contains a viable, and transparent, amq broker inside the daemon. Salt uses public keys for authentication with the master daemon, then uses faster aes encryption for payload communication, this means that authentication and encryption are also built into Salt. Salt takes advantage of communication via python pickles, enabling fast and light network traffic. -In order to allow for simple expansion, Salt execution routines can be written as plain python modules, and the data collected from salt executions can be sent back to the master server, or to any arbitrary program. Salt can be called from a simple python api, or from the command line, so that salt can be used to execute one-off commands as well as operate as an integral part of a larger application. -The result is a system that can execute commands across groups of varying size, from very few to very many servers at considerably high speed. A system that is very fast, easy to set up and amazingly malleable, able to suit the needs of any number of servers working within the same system. Salt’s unique architecture brings together the best of the remote execution world, amplifies its capabilities and expands its range, resulting in this system that is as versatile as it is practical, able to suit any network. -Salt is developed under the Apache 2.0 licence, and can be used for open and proprietary projects. Please submit your expansions to back so that we can all benefit together as Salt grows. So, please feel free to sprinkle some of this around your systems and let the deliciousness come forth. +Salt is a distributed remote execution system used to execute commands and query +data. It was developed in order to bring the best solutions found in the world +of remote execution together and make them better, faster and more malleable. +Salt accomplishes this via its ability to handle larger loads of information, +and not just dozens, but hundreds or even thousands of individual servers, +handle them quickly and through a simple and manageable interface. +Versatility between massive scale deployments and smaller systems may seem +daunting, but Salt is very simple to set up and maintain, regardless of the +size of the project. The architecture of Salt is designed to work with any +number of servers, from a handful of local network systems to international +deployments across disparate data-centers. The topology is a simple server/client +model with the needed functionality built into a single set of daemons. +While the default configuration will work with little to no modification, salt +can be fine tuned to meet specific needs. +The core function of salt is to enable remote commands to be called in parallel +rather than in serial, to use a secure and encrypted protocol, the smallest and +fastest network payloads possible and with a simple programmer interface. +Salt also introduces more granular controls to the realm of remote execution, +allowing for commands to be executed in parallel and for systems to be targeted +based on more than just hostname, but by system properties. +Salt takes advantage of a number of technologies and techniques. The networking +layer is built with the excellent zeromq networking library, so salt itself +contains a viable, and transparent, amq broker inside the daemon. Salt uses +public keys for authentication with the master daemon, then uses faster aes +encryption for payload communication, this means that authentication and +encryption are also built into Salt. Salt takes advantage of communication via +python pickles, enabling fast and light network traffic. +In order to allow for simple expansion, Salt execution routines can be written +as plain python modules, and the data collected from salt executions can be sent +back to the master server, or to any arbitrary program. Salt can be called from +a simple python api, or from the command line, so that salt can be used to +execute one-off commands as well as operate as an integral part of a larger +application. +The result is a system that can execute commands across groups of varying size, +from very few to very many servers at considerably high speed. A system that is +very fast, easy to set up and amazingly malleable, able to suit the needs of any +number of servers working within the same system. Salt’s unique architecture +brings together the best of the remote execution world, amplifies its +capabilities and expands its range, resulting in this system that is as +versatile as it is practical, able to suit any network. + +Salt is developed under the Apache 2.0 licence, and can be used for open and +proprietary projects. Please submit your expansions to back so that we can all +benefit together as Salt grows. So, please feel free to sprinkle some of this +around your systems and let the deliciousness come forth. diff --git a/doc/man/salt-cp.1 b/doc/man/salt-cp.1 index a45db14a47..ccc406d468 100644 --- a/doc/man/salt-cp.1 +++ b/doc/man/salt-cp.1 @@ -24,23 +24,28 @@ The timeout in seconds to wait for replies from the salt minions. .TP -E, --pcre -The target expresion will be interpereted as a pcre regular expression rather than a shell glob. +The target expression will be interpreted as a pcre regular expression rather than a shell glob. .TP -L, --list -The target expression will be interpereted as a comma delimited list, example: server1.foo.bar,server2.foo.bar,example7.quo.qux +The target expression will be interpreted as a comma delimited list, +example: server1.foo.bar,server2.foo.bar,example7.quo.qux .TP -G, --grain -The target expression matches values returned by the salt grains system on the minions. The target expresion is in the format of ':'; example: 'os:Arch.*' +The target expression matches values returned by the salt grains system on the +minions. The target expression is in the format of ':'; +example: 'os:Arch.*' .TP -Q, --query -Execute a salt command query, this can be used to find the results os a previous function call: -Q test.echo') +Execute a salt command query, this can be used to find the results os a previous +function call: -Q test.echo') .TP -c CONFIG, --config=CONFIG -The location of the salt master configuration file, the salt master settings are required to know where the connections are; default=/etc/salt/master +The location of the salt master configuration file, the salt master settings are +required to know where the connections are; default=/etc/salt/master .SH AUTHORS Thomas S. Hatch diff --git a/doc/man/salt-key.1 b/doc/man/salt-key.1 index 4d4d8f9975..41843796a4 100644 --- a/doc/man/salt-key.1 +++ b/doc/man/salt-key.1 @@ -32,7 +32,8 @@ Accepts all pending public keys. .TP -c CONFIG, --config=CONFIG -The master configuration file needs to be read to determine where the salt keys are stored via the pki_dir configuration value; default=/etc/salt/master +The master configuration file needs to be read to determine where the salt keys +are stored via the pki_dir configuration value; default=/etc/salt/master .SH AUTHORS diff --git a/doc/man/salt-minion.1 b/doc/man/salt-minion.1 index a522ca14a3..544ca110a1 100644 --- a/doc/man/salt-minion.1 +++ b/doc/man/salt-minion.1 @@ -1,13 +1,14 @@ .TH salt-minion 1 "May 2011" "salt-minion 0.8.7" "salt-minion Manual" .SH NAME -salt-minion \- The salt minion daemon, recieves commands from a remote salt master. +salt-minion \- The salt minion daemon, receives commands from a remote salt master. .SH SYNOPSIS .B salt-minion [ options ] .SH DESCRIPTION -The salt minion recieves commands from the central salt master and replies with the results of said commands. +The salt minion receives commands from the central salt master and replies with +the results of said commands. .SH OPTIONS .TP diff --git a/doc/man/salt.1 b/doc/man/salt.1 index 82a185c13b..b7c52aa474 100644 --- a/doc/man/salt.1 +++ b/doc/man/salt.1 @@ -13,7 +13,8 @@ salt \- Parallel remote execution system .B salt -Q test.ping .SH DESCRIPTION -Salt allows for commands to be executed across a swath of remote systems in parallel. This means that remote systems can be both controleed and querried with ease. +Salt allows for commands to be executed across a swath of remote systems in parallel. +This means that remote systems can be both controlled and queried with ease. .SH OPTIONS .TP @@ -26,23 +27,29 @@ The timeout in seconds to wait for replies from the salt minions. .TP -E, --pcre -The target expresion will be interpereted as a pcre regular expression rather than a shell glob. +The target expression will be interpreted as a pcre regular expression rather than +a shell glob. .TP -L, --list -The target expression will be interpereted as a comma delimited list, example: server1.foo.bar,server2.foo.bar,example7.quo.qux +The target expression will be interpreted as a comma delimited list, +example: server1.foo.bar,server2.foo.bar,example7.quo.qux .TP -G, --grain -The target expression matches values returned by the salt grains system on the minions. The target expresion is in the format of ':'; example: 'os:Arch.*' +The target expression matches values returned by the salt grains system on the +minions. The target expression is in the format of ':'; +example: 'os:Arch.*' .TP -Q, --query -Execute a salt command query, this can be used to find the results os a previous function call: -Q test.echo') +Execute a salt command query, this can be used to find the results os a previous +function call: -Q test.echo') .TP -c CONFIG, --config=CONFIG -The location of the salt master configuration file, the salt master settings are required to know where the connections are; default=/etc/salt/master +The location of the salt master configuration file, the salt master settings are +required to know where the connections are; default=/etc/salt/master .SH AUTHORS Thomas S. Hatch diff --git a/doc/man/salt.7 b/doc/man/salt.7 index 487c65d78a..611828702e 100644 --- a/doc/man/salt.7 +++ b/doc/man/salt.7 @@ -11,13 +11,17 @@ salt \- The salt distributed system controller .B salt [options] .SH DESCRIPTION -Salt is a powerful remote execution manager that can be used to administer servers in a fast and efficient way. +Salt is a powerful remote execution manager that can be used to administer +servers in a fast and efficient way. .SH SETTING UP SALT -The Salt system setup is amazingly simple, as this is one of the central design goals of Salt. Setting up Salt only requires that the salt master be running and the salt minions point to the salt master. +The Salt system setup is amazingly simple, as this is one of the central design +goals of Salt. Setting up Salt only requires that the salt master be running and +the salt minions point to the salt master. .SS INSTALATION -The salt install is simple, for many distributions pakages should be available for download, or salt can be installed via the python packaging tools. +The salt install is simple, for many distributions packages should be available +for download, or salt can be installed via the python packaging tools. .P Installing salt from source is a simple task, download the latest source tarball from .URL "https://github.com/thatch45/salt/downloads" @@ -31,15 +35,16 @@ Next untar the download and run the python setup: # python2 setup.py install .SS SALT DEPENDENCIES -This is a basic python setup, nothing fancy. Salt does require a number of dependencies though, all of which should be available in your distribution's packages. +This is a basic python setup, nothing fancy. Salt does require a number of +dependencies though, all of which should be available in your distribution's packages. -.TP -\(bu +.TP +\(bu python2.7 - python2.6 should work, but it is untested -.TP +.TP \(bu pyzmq - zeromq python bindings -.TP +.TP \(bu python-m2crypto - Crypto backend .TP @@ -50,7 +55,9 @@ python-yaml - Reading configuration files pycrypto - Another crypto backend .SS RUNNING SALT -To run Salt you need to ensure that a master and a minion are running and referencing each other. Starting the master and minion daemons is done with the respective commands. +To run Salt you need to ensure that a master and a minion are running and +referencing each other. Starting the master and minion daemons is done with the +respective commands. On the salt master server: diff --git a/salt/__init__.py b/salt/__init__.py index cc9a4e52ab..8f1753a0c0 100644 --- a/salt/__init__.py +++ b/salt/__init__.py @@ -10,7 +10,7 @@ import salt.config def verify_env(dirs): ''' - Verify that the named direcotries are in place and that the environment + Verify that the named directories are in place and that the environment can shake the salt ''' for dir_ in dirs: diff --git a/salt/cli/__init__.py b/salt/cli/__init__.py index 1d9f47246b..2e126bc661 100644 --- a/salt/cli/__init__.py +++ b/salt/cli/__init__.py @@ -25,7 +25,7 @@ class SaltCMD(object): ''' def __init__(self): ''' - Cretae a SaltCMD object + Create a SaltCMD object ''' self.opts = self.__parse() @@ -34,7 +34,7 @@ class SaltCMD(object): Parse the command line ''' parser = optparse.OptionParser() - + parser.add_option('-t', '--timeout', default=5, @@ -63,7 +63,7 @@ class SaltCMD(object): help='Instead of using shell globs to evaluate the target'\ + ' use a grain value to identify targets, the syntax'\ + ' for the target is the grain key followed by a pcre'\ - + ' regular expresion:\n"os:Arch.*"') + + ' regular expression:\n"os:Arch.*"') parser.add_option('-X', '--exsel', default=False, @@ -97,7 +97,7 @@ class SaltCMD(object): action='store_true', dest='raw_out', help='Print the output from the salt command in raw python'\ - + ' form, this is suitible for re-reading the output into'\ + + ' form, this is suitable for re-reading the output into'\ + ' an executing python script with eval.') if JSON: parser.add_option('--json-out', @@ -179,7 +179,7 @@ class SaltCMD(object): args.append('exsel') else: args.append('glob') - + if self.opts['return']: args.append(self.opts['return']) ret = local.cmd(*args) @@ -224,7 +224,7 @@ class SaltCP(object): Parse the command line ''' parser = optparse.OptionParser() - + parser.add_option('-t', '--timeout', default=5, @@ -253,7 +253,7 @@ class SaltCP(object): help='Instead of using shell globs to evaluate the target'\ + ' use a grain value to identify targets, the syntax'\ + ' for the target is the grains key followed by a pcre'\ - + ' regular expresion:\n"os:Arch.*"') + + ' regular expression:\n"os:Arch.*"') parser.add_option('-c', '--config', default='/etc/salt/master', @@ -315,7 +315,7 @@ class SaltKey(object): default=False, action='store_true', help='List all public keys') - + parser.add_option('-a', '--accept', dest='accept', diff --git a/salt/cli/caller.py b/salt/cli/caller.py index aed20a2608..5920200c20 100644 --- a/salt/cli/caller.py +++ b/salt/cli/caller.py @@ -1,5 +1,5 @@ ''' -The caller module is used as a frontend to manage direct calls to the salt +The caller module is used as a front-end to manage direct calls to the salt minion modules. ''' # Import python modules diff --git a/salt/cli/cp.py b/salt/cli/cp.py index fae2a24562..ed2dcd9668 100644 --- a/salt/cli/cp.py +++ b/salt/cli/cp.py @@ -1,7 +1,7 @@ ''' The cp module is used to execute the logic used by the salt-cp command line application, salt-cp is NOT intended to broadcast large files, it is -intened to handle text files. +intended to handle text files. Salt-cp can be used to distribute configuration files ''' # Import python modules @@ -73,7 +73,7 @@ class SaltCP(object): args.append('list') elif self.opts['grain']: args.append('grain') - + ret = local.cmd(*args) print yaml.dump(ret) diff --git a/salt/client.py b/salt/client.py index 214fb7cadc..74cd77e825 100644 --- a/salt/client.py +++ b/salt/client.py @@ -1,6 +1,6 @@ ''' The client module is used to create a client connection to the publisher -The data structurte needs to be: +The data structure needs to be: {'enc': 'clear', 'load': {'fun': '', 'arg':, ('arg1', 'arg2', ...), @@ -10,17 +10,17 @@ The data structurte needs to be: # The components here are simple, and they need to be and stay simple, we # want a client to have 3 external concerns, and maybe a forth configurable # option. -# The concers are +# The concerns are # 1. Who executes the command? # 2. what is the function being run? # 3. What arguments need to be passed to the function? # 4. How long do we wait for all of the replies? # # Next there are a number of tasks, first we need some kind of authentication -# This Client initially will be the master root client, which will run as the +# This Client initially will be the master root client, which will run as the # root user on the master server. # BUT we also want a client to be able to work over the network, so that -# controllers can exist within disperate applicaitons. +# controllers can exist within disparate applications. # The problem is that this is a security nightmare, so I am going to start # small, and only start with the ability to execute salt commands locally. # This means that the primary client to build is, the LocalClient @@ -90,7 +90,7 @@ class LocalClient(object): def _check_pcre_minions(self, expr): ''' - Return the minions found by looking via regular expresions + Return the minions found by looking via regular expressions ''' ret = set() cwd = os.getcwd() @@ -184,10 +184,10 @@ class LocalClient(object): 'grain': self._check_grain_minions, 'exsel': self._check_grain_minions, }[expr_form](expr) - + def pub(self, tgt, fun, arg=(), expr_form='glob', ret=''): ''' - Take the required arguemnts and publish the given command. + Take the required arguments and publish the given command. Arguments: tgt: The tgt is a regex or a glob used to match up the ids on @@ -195,11 +195,11 @@ class LocalClient(object): all of the minions and then the minions determine if the command is for them based on the tgt value. fun: - The function nane to be called on the remote host(s), this must + The function name to be called on the remote host(s), this must be a string in the format "." arg: The arg option needs to be a tuple of arguments to pass to the - calling function, if left blank + calling function, if left blank Returns: jid: A string, as returned by the publisher, which is the job id, @@ -207,7 +207,7 @@ class LocalClient(object): minions: A set, the targets that the tgt passed should match. ''' - # Run a check_minions, if no minons match return False + # Run a check_minions, if no minions match return False # format the payload - make a function that does this in the payload # module # make the zmq client @@ -229,8 +229,7 @@ class LocalClient(object): # Prep zmq context = zmq.Context() socket = context.socket(zmq.REQ) - socket.connect('tcp://' + self.opts['interface'] + ':'\ - + str(self.opts['ret_port'])) + socket.connect('tcp://%(interface)s:%(ret_port)s' % self.opts) socket.send(package) payload = salt.payload.unpackage(socket.recv()) return {'jid': payload['load']['jid'], diff --git a/salt/config.py b/salt/config.py index ec57cff49e..e85747717c 100644 --- a/salt/config.py +++ b/salt/config.py @@ -45,7 +45,7 @@ def minion_config(path): opts['master_uri'] = 'tcp://' + opts['master'] + ':'\ + str(opts['master_port']) - # Enableing open mode requires that the value be set to True, and nothing + # Enabling open mode requires that the value be set to True, and nothing # else! if opts['open_mode']: if opts['open_mode'] == True: @@ -91,7 +91,7 @@ def master_config(path): opts['aes'] = salt.crypt.Crypticle.generate_key_string() - # Enableing open mode requires that the value be set to True, and nothing + # Enabling open mode requires that the value be set to True, and nothing # else! if opts['open_mode']: if opts['open_mode'] == True: diff --git a/salt/crypt.py b/salt/crypt.py index e294d1abbd..1144737aac 100644 --- a/salt/crypt.py +++ b/salt/crypt.py @@ -48,7 +48,7 @@ class MasterKeys(dict): def __get_priv_key(self): ''' - Retruns a private key object for the master + Returns a private key object for the master ''' key = None try: @@ -89,7 +89,7 @@ class Auth(object): def get_priv_key(self): ''' - Retruns a private key object for the minion + Returns a private key object for the minion ''' key = None try: @@ -107,7 +107,7 @@ class Auth(object): def minion_sign_in_payload(self): ''' - Generates the payload used to autnenticate with the master server. This + Generates the payload used to authenticate with the master server. This payload consists of the passed in id_ and the ssh public key to encrypt the AES key sent back form the master. ''' @@ -129,7 +129,7 @@ class Auth(object): Pass in the encrypted aes key. Returns the decrypted aes seed key, a string ''' - log.info('Decypting the current master AES key') + log.info('Decrypting the current master AES key') key = self.get_priv_key() return key.private_decrypt(aes, 4) @@ -137,7 +137,7 @@ class Auth(object): ''' Takes the master pubkey and compares it to the saved master pubkey, the token is encrypted with the master private key and must be - decrypted sucessfully to verify that the master has been connected to. + decrypted successfully to verify that the master has been connected to. The token must decrypt with the public key, and it must say: 'salty bacon' returns a bool @@ -274,7 +274,7 @@ class Crypticle(object): def loads(self, data, pickler=pickle): ''' - decrypt and unpickle a python object + decrypt and un-pickle a python object ''' data = self.decrypt(data) # simple integrity check to verify that we got meaningful data @@ -294,7 +294,7 @@ class SAuth(object): def __authenticate(self): ''' Authenticate with the master, this method breaks the functional - pardigmn, it will update the master information from a fresh sign in, + paradigm, it will update the master information from a fresh sign in, signing in can occur as often as needed to keep up with the revolving master aes key. ''' diff --git a/salt/grains/core.py b/salt/grains/core.py index 19419d62ef..4b971facd8 100644 --- a/salt/grains/core.py +++ b/salt/grains/core.py @@ -127,7 +127,7 @@ def os_data(): grains['os'] = kernel # Load the virtual machine info - + grains.update(_virtual(grains)) grains.update(_ps(grains)) return grains diff --git a/salt/loader.py b/salt/loader.py index 899c5a1d0d..8a206a06e6 100644 --- a/salt/loader.py +++ b/salt/loader.py @@ -81,9 +81,9 @@ def call(fun, args=[], dirs=[]): class Loader(object): ''' - Used to load in arbitrairy modules from a directory, the Loader can also be + Used to load in arbitrary modules from a directory, the Loader can also be used to only load specific functions from a directory, or to call modules - in an arbitrairy directory directly. + in an arbitrary directory directly. ''' def __init__(self, module_dirs, opts={}): self.module_dirs = module_dirs @@ -255,7 +255,7 @@ class Loader(object): def gen_grains(self): ''' Read the grains directory and execute all of the public callable - members. then verify that the returns are python dicts and return a + members. then verify that the returns are python dict's and return a dict containing all of the returned values. ''' grains = {} diff --git a/salt/log.py b/salt/log.py index 53aeea1b6f..5c719fba16 100644 --- a/salt/log.py +++ b/salt/log.py @@ -7,7 +7,7 @@ :copyright: © 2011 :email:`Pedro Algarvio (pedro@algarvio.me)` - :license: Apache Version 2.0, see LICENSE for more details. + :license: Apache 2.0, see LICENSE for more details. """ import logging diff --git a/salt/master.py b/salt/master.py index 9c5c775639..7a6322c53e 100644 --- a/salt/master.py +++ b/salt/master.py @@ -19,7 +19,7 @@ import salt.utils import salt.crypt import salt.payload import salt.client -# Import cryptogrogphy modules +# Import cryptography modules from M2Crypto import RSA log = logging.getLogger(__name__) @@ -61,7 +61,7 @@ class Master(object): class Publisher(multiprocessing.Process): ''' - The publihing interface, a simple zeromq publisher that sends out the + The publishing interface, a simple zeromq publisher that sends out the commands. ''' def __init__(self, opts): @@ -115,7 +115,7 @@ class ReqServer(): A key needs to be placed in the filesystem with permissions 0400 so clients are required to run as root. ''' - log.info('Preparing the root key for local comunication') + log.info('Preparing the root key for local communication') keyfile = os.path.join(self.opts['cachedir'], '.root_key') key = salt.crypt.Crypticle.generate_key_string() if os.path.isfile(keyfile): @@ -139,7 +139,7 @@ class ReqServer(): m_worker.start() out_socket = self.context.socket(zmq.REQ) - out_socket.connect('tcp://127.0.0.1:' + str(work_port)) + out_socket.connect('tcp://127.0.0.1:%s' % work_port) while True: package = in_socket.recv() @@ -188,7 +188,7 @@ class MWorker(multiprocessing.Process): ''' context = zmq.Context(1) socket = context.socket(zmq.REP) - socket.bind('tcp://127.0.0.1:' + self.port) + socket.bind('tcp://127.0.0.1:%s' % self.port) while True: package = socket.recv() @@ -202,8 +202,9 @@ class MWorker(multiprocessing.Process): job id directory. ''' jid_root = os.path.join(self.opts['cachedir'], 'jobs') - jid = datetime.datetime.strftime(datetime.datetime.now(), - '%Y%m%d%H%M%S%f') + jid = datetime.datetime.strftime( + datetime.datetime.now(), '%Y%m%d%H%M%S%f' + ) jid_dir = os.path.join(jid_root, jid) if not os.path.isdir(jid_dir): os.makedirs(jid_dir) @@ -225,21 +226,21 @@ class MWorker(multiprocessing.Process): ''' Take care of a cleartext command ''' - log.info('Clear payload recieved with command %(cmd)s', load) + log.info('Clear payload received with command %(cmd)s', load) return getattr(self, load['cmd'])(load) def _handle_pub(self, load): ''' Handle a command sent via a public key pair ''' - log.info('Pubkey payload recieved with command %(cmd)s', load) + log.info('Pubkey payload received with command %(cmd)s', load) def _handle_aes(self, load): ''' Handle a command sent via an aes key ''' data = self.crypticle.loads(load) - log.info('AES payload recieved with command %(cmd)s', load) + log.info('AES payload received with command %(cmd)s', load) return getattr(self, data['cmd'])(data) def _auth(self, load): @@ -247,7 +248,7 @@ class MWorker(multiprocessing.Process): Authenticate the client, use the sent public key to encrypt the aes key which was generated at start up ''' - # 1. Verify that the key we are recieving matches the stored key + # 1. Verify that the key we are receiving matches the stored key # 2. Store the key if it is not there # 3. make an rsa key with the pub key # 4. encrypt the aes key as an encrypted pickle @@ -326,7 +327,7 @@ class MWorker(multiprocessing.Process): def _serve_file(self, load): ''' - Return a chunk from a file based on the data recieved + Return a chunk from a file based on the data received ''' if not load.has_key('path') or not load.has_key('loc'): return False @@ -368,7 +369,7 @@ class MWorker(multiprocessing.Process): jid_dir = os.path.join(self.opts['cachedir'], 'jobs', load['jid']) if not os.path.isdir(jid_dir): log.error( - 'An inconsistency occured, a job was recieved with a job id ' + 'An inconsistency occurred, a job was received with a job id ' 'that is not present on the master: %(jid)s', load ) return False @@ -380,7 +381,7 @@ class MWorker(multiprocessing.Process): def _send_cluster(self): ''' - Send the cluser data out + Send the cluster data out ''' log.debug('Sending out cluster data') ret = self.local.cmd(self.opts['cluster_masters'], @@ -430,7 +431,7 @@ class MWorker(multiprocessing.Process): payload['load'] = self.crypticle.dumps(load) context = zmq.Context(1) pub_sock = context.socket(zmq.PUSH) - pub_sock.connect('tcp://127.0.0.1:' + self.opts['publish_pull_port']) + pub_sock.connect('tcp://127.0.0.1:%(publish_pull_port)s' % self.opts) pub_sock.send(salt.payload.package(payload)) return {'enc': 'clear', 'load': {'jid': jid}} diff --git a/salt/minion.py b/salt/minion.py index b7c85a2c9b..c6c65dad4d 100644 --- a/salt/minion.py +++ b/salt/minion.py @@ -36,7 +36,7 @@ log = logging.getLogger(__name__) class Minion(object): ''' - This class instanciates a minion, runs connections for a minion, and loads + This class instantiates a minion, runs connections for a minion, and loads all of the functions into the minion ''' def __init__(self, opts): @@ -85,7 +85,7 @@ class Minion(object): def _handle_aes(self, load): ''' - Takes the aes encrypted load, decypts is and runs the encapsulated + Takes the aes encrypted load, decrypts is and runs the encapsulated instructions ''' data = None @@ -117,24 +117,32 @@ class Minion(object): def _handle_clear(self, load): ''' - Handle unencrypted transmisions + Handle un-encrypted transmissions ''' pass def _handle_decoded_payload(self, data): ''' - Override this method if you wish to handle the decoded data diferently. + Override this method if you wish to handle the decoded data differently. ''' if self.opts['multiprocessing']: if type(data['fun']) == type(list()): - multiprocessing.Process(target=lambda: self._thread_multi_return(data)).start() + multiprocessing.Process( + target=lambda: self._thread_multi_return(data) + ).start() else: - multiprocessing.Process(target=lambda: self._thread_return(data)).start() + multiprocessing.Process( + target=lambda: self._thread_return(data) + ).start() else: if type(data['fun']) == type(list()): - threading.Thread(target=lambda: self._thread_multi_return(data)).start() + threading.Thread( + target=lambda: self._thread_multi_return(data) + ).start() else: - threading.Thread(target=lambda: self._thread_return(data)).start() + threading.Thread( + target=lambda: self._thread_return(data) + ).start() def _glob_match(self, tgt): ''' @@ -163,7 +171,7 @@ class Minion(object): def _grain_match(self, tgt): ''' - Reads in the grains regular expresion match + Reads in the grains regular expression match ''' comps = tgt.split(':') return bool(re.match(comps[1], self.opts['grains'][comps[0]])) @@ -206,7 +214,7 @@ class Minion(object): def _thread_multi_return(self, data): ''' - This methos should be used as a threading target, start the actual + This method should be used as a threading target, start the actual minion side execution. ''' ret = {'return': {}} @@ -262,7 +270,7 @@ class Minion(object): def authenticate(self): ''' Authenticate with the master, this method breaks the functional - pardigmn, it will update the master information from a fresh sign in, + paradigm, it will update the master information from a fresh sign in, signing in can occur as often as needed to keep up with the revolving master aes key. ''' @@ -271,7 +279,7 @@ class Minion(object): while True: creds = auth.sign_in() if creds != 'retry': - log.info('Authentication with master sucessful!') + log.info('Authentication with master successful!') break log.info('Waiting for minion key to be accepted by the master.') time.sleep(10) diff --git a/salt/modules/butterkvm.py b/salt/modules/butterkvm.py index 9c02db14ca..a9b9cef34c 100644 --- a/salt/modules/butterkvm.py +++ b/salt/modules/butterkvm.py @@ -1,5 +1,5 @@ ''' -Specilized routines used by the butter cloud component +Specialized routines used by the butter cloud component ''' # Import salt modules import virt @@ -76,7 +76,7 @@ def _gen_pin_drives(pins): def _apply_overlay(vda, instance): ''' - Use libguestfs to apply the overlay inder the specified instance to the + Use libguestfs to apply the overlay under the specified instance to the specified vda ''' overlay = os.path.join(instance, 'overlay') diff --git a/salt/modules/cp.py b/salt/modules/cp.py index 15e544d003..bf35ef4b6f 100644 --- a/salt/modules/cp.py +++ b/salt/modules/cp.py @@ -3,6 +3,7 @@ Minion side functions for salt-cp ''' # Import python libs import os +import hashlib # Import salt libs import salt.crypt @@ -14,7 +15,7 @@ def recv(files, dest): ''' Used with salt-cp, pass the files dict, and the destination. - This function recieves small fast copy files from the master via salt-cp + This function receives small fast copy files from the master via salt-cp ''' ret = {} for path, data in files.items(): @@ -67,7 +68,7 @@ def get_file(path, dest): def cache_files(paths): ''' Used to gather many files from the master, the gathered files will be - saved in the minion cachedir reflective to the paths retrived from the + saved in the minion cachedir reflective to the paths retrieved from the master. ''' auth = salt.crypt.SAuth(__opts__) diff --git a/salt/modules/cytest.pyx b/salt/modules/cytest.pyx index a6eafe9d8a..68e4d3e856 100644 --- a/salt/modules/cytest.pyx +++ b/salt/modules/cytest.pyx @@ -1,5 +1,5 @@ ''' -Module for running arbitrairy tests +Module for running arbitrary tests ''' import time @@ -26,7 +26,7 @@ def ping(): def fib(long num): ''' - Return a fibonachi sequence up to the passed number, and the time it took + Return a Fibonacci sequence up to the passed number, and the time it took to compute in seconds. Used for performance tests CLI Example: diff --git a/salt/modules/file.py b/salt/modules/file.py index 0c5e520e19..45bd9f241b 100644 --- a/salt/modules/file.py +++ b/salt/modules/file.py @@ -10,7 +10,6 @@ data import os import grp import pwd -import hashlib def gid_to_group(gid): ''' @@ -135,7 +134,7 @@ def set_mode(path, mode): def chown(path, user, group): ''' - Chown a file, pass the filem the desired user and group + Chown a file, pass the file the desired user and group CLI Example: salt '*' file.chown /etc/passwd root root diff --git a/salt/modules/network.py b/salt/modules/network.py index d375ccce83..b5f0e03e8d 100644 --- a/salt/modules/network.py +++ b/salt/modules/network.py @@ -6,12 +6,14 @@ import socket from string import ascii_letters, digits def _sanitize_host(host): - return "".join([c for c in host[0:255] if c in (ascii_letters + digits + '.')]) + return "".join([ + c for c in host[0:255] if c in (ascii_letters + digits + '.') + ]) def ping(host): ''' Performs a ping to a host - + CLI Example: salt '*' network.ping archlinux.org -c 4 ''' @@ -25,7 +27,7 @@ def ping(host): def netstat(): ''' Return information on open ports and states - + CLI Example: salt '*' network.netstat ''' @@ -66,7 +68,7 @@ def netstat(): def traceroute(host): ''' Performs a traceroute to a 3rd party host - + CLI Example: salt '*' network.traceroute archlinux.org ''' @@ -98,7 +100,7 @@ def traceroute(host): def dig(host): ''' Performs a DNS lookup with dig - + CLI Example: salt '*' network.dig archlinux.org ''' @@ -112,7 +114,7 @@ def dig(host): def isportopen(host, port): ''' Return status of a port - + CLI Example: salt '*' network.isportopen 127.0.0.1 22 ''' diff --git a/salt/modules/pacman.py b/salt/modules/pacman.py index 88ac090995..08421fc1cb 100644 --- a/salt/modules/pacman.py +++ b/salt/modules/pacman.py @@ -153,13 +153,13 @@ def upgrade(): pkgs[npkg] = {'old': '', 'new': new[npkg]} return pkgs - + def remove(name): ''' Remove a single package with pacman -R Return a list containing the removed packages: - + CLI Example: salt '*' pkg.remove ''' @@ -175,7 +175,7 @@ def purge(name): with it, this will call a pacman -Rsc Return a list containing the removed packages: - + CLI Example: salt '*' pkg.purge diff --git a/salt/modules/service.py b/salt/modules/service.py index 5ed62e77ca..4a90a7b968 100644 --- a/salt/modules/service.py +++ b/salt/modules/service.py @@ -38,7 +38,7 @@ def stop(name): def status(name, sig=None): ''' Return the status for a service, returns the PID or an empty string if the - service is running or not, pass a signiture to use to find the service via + service is running or not, pass a signature to use to find the service via ps CLI Example: diff --git a/salt/modules/state.py b/salt/modules/state.py index 93a05a9a44..c115f1c26e 100644 --- a/salt/modules/state.py +++ b/salt/modules/state.py @@ -1,8 +1,6 @@ ''' Control the state system on the minion ''' -# Import Python modules -import os # Import salt modules import salt.state @@ -41,7 +39,7 @@ def template(tem): ''' st_ = salt.state.State(__opts__) return st_.call_template(tem) - + def template_str(tem): ''' Execute the information stored in a template file on the minion @@ -51,4 +49,4 @@ def template_str(tem): ''' st_ = salt.state.State(__opts__) return st_.call_template_str(tem) - + diff --git a/salt/modules/status.py b/salt/modules/status.py index e424484d13..a72978d911 100644 --- a/salt/modules/status.py +++ b/salt/modules/status.py @@ -7,7 +7,7 @@ import subprocess __opts__ = {} def custom(): - ''' + ''' Return a custom composite of status data and info for this minon, based on the minion config file. An example config like might be: @@ -19,7 +19,7 @@ def custom(): This function is meant to replace all_status(), which returns anything and everything, which we probably don't want. - + By default, nothing is returned. Warning: Depending on what you include, there can be a LOT here! @@ -38,7 +38,7 @@ def custom(): for item in __opts__[opt]: ret[item] = vals[item] - return ret + return ret def uptime(): ''' @@ -59,7 +59,7 @@ def loadavg(): ''' comps = open('/proc/loadavg', 'r').read().strip() load_avg = comps.split() - return { + return { '1-min': load_avg[1], '5-min': load_avg[2], '15-min': load_avg[3], @@ -105,7 +105,7 @@ def cpustats(): def meminfo(): ''' - Return the CPU stats for this minon + Return the CPU stats for this minion CLI Example: salt '*' status.meminfo @@ -125,8 +125,8 @@ def meminfo(): return ret def cpuinfo(): - ''' - Return the CPU info for this minon + ''' + Return the CPU info for this minion CLI Example: salt '*' status.cpuinfo @@ -142,11 +142,11 @@ def cpuinfo(): ret[comps[0]] = comps[1].split() else: ret[comps[0]] = comps[1].strip() - return ret + return ret def diskstats(): ''' - Return the disk stats for this minon + Return the disk stats for this minion CLI Example: salt '*' status.diskstats @@ -177,7 +177,7 @@ def diskstats(): def vmstats(): ''' - Return the virtual memory stats for this minon + Return the virtual memory stats for this minion CLI Example: salt '*' status.vmstats @@ -192,8 +192,8 @@ def vmstats(): return ret def netstats(): - ''' - Return the network stats for this minon + ''' + Return the network stats for this minion CLI Example: salt '*' status.netstats @@ -206,22 +206,22 @@ def netstats(): continue comps = line.split() if comps[0] == headers[0]: - index = len(headers) - 1 + index = len(headers) - 1 row = {} for field in range(index): if field < 1: continue else: row[headers[field]] = comps[field] - rowname = headers[0].replace(':', '') - ret[rowname] = row + rowname = headers[0].replace(':', '') + ret[rowname] = row else: headers = comps - return ret + return ret def netdev(): ''' - Return the network device stats for this minon + Return the network device stats for this minion CLI Example: salt '*' status.netdev @@ -256,8 +256,8 @@ def netdev(): return ret def w(): - ''' - Return a list of logged in users for this minon, using the w command + ''' + Return a list of logged in users for this minion, using the w command CLI Example: salt '*' status.w @@ -270,7 +270,7 @@ def w(): if not row.count(' '): continue comps = row.split() - rec = { + rec = { 'user': comps[0], 'tty': comps[1], 'login': comps[2], @@ -278,13 +278,14 @@ def w(): 'jcpu': comps[4], 'pcpu': comps[5], 'what': ' '.join(comps[6:]), - } + } user_list.append( rec ) return user_list def all_status(): ''' - Return a composite of all status data and info for this minon. Warning: There is a LOT here! + Return a composite of all status data and info for this minoon. + Warning: There is a LOT here! CLI Example: salt '*' status.all_status diff --git a/salt/modules/sysctl.py b/salt/modules/sysctl.py index db9c564ae4..4d2b8e245d 100644 --- a/salt/modules/sysctl.py +++ b/salt/modules/sysctl.py @@ -1,5 +1,5 @@ ''' -Module for viewing and modifying sysctl paramters +Module for viewing and modifying sysctl parameters ''' import subprocess diff --git a/salt/modules/test.py b/salt/modules/test.py index 34b51d4e83..7ad748c89e 100644 --- a/salt/modules/test.py +++ b/salt/modules/test.py @@ -1,5 +1,5 @@ ''' -Module for running arbitrairy tests +Module for running arbitrary tests ''' import time @@ -50,7 +50,7 @@ def get_opts(): def fib(num): ''' - Return a fibonachi sequence up to the passed number, and the time it took + Return a Fibonacci sequence up to the passed number, and the time it took to compute in seconds. Used for performance tests CLI Example: diff --git a/salt/modules/virt.py b/salt/modules/virt.py index 623feb7c0e..ce89bb7748 100644 --- a/salt/modules/virt.py +++ b/salt/modules/virt.py @@ -1,5 +1,5 @@ ''' -Work with vitual machines managed by libvirt +Work with virtual machines managed by libvirt ''' # Special Thanks to Michael Dehann, many of the concepts, and a few structures # of his in the virt func module have been used @@ -39,7 +39,7 @@ def __get_conn(): def _get_dom(vm_): ''' - Return a domain object for the named vm + Return a domain object for the named vm ''' conn = __get_conn() if not list_vms().count(vm_): @@ -63,7 +63,7 @@ def _libvirt_creds(): def list_vms(): ''' Return a list of virtual machine names on the minion - + CLI Example: salt '*' virt.list_vms ''' @@ -109,7 +109,6 @@ def node_info(): salt '*' virt.node_info ''' conn = __get_conn() - info = {} raw = conn.getInfo() info = { 'cpumodel' : str(raw[0]), @@ -343,7 +342,7 @@ def migrate(vm_, target): def seed_non_shared_migrate(disks, force=False): ''' - Non shared migration reqiuires that the disks be present on the migration + Non shared migration requires that the disks be present on the migration destination, pass the disks information via this function, to the migration destination before executing the migration. @@ -355,7 +354,7 @@ def seed_non_shared_migrate(disks, force=False): form = data['file format'] size = data['virtual size'].split()[1][1:] if os.path.isfile(fn_) and not force: - # the target exists, check to see if is is compatable + # the target exists, check to see if is is compatible pre = yaml.load(subprocess.Popen('qemu-img info arch', shell=True, stdout=subprocess.PIPE).communicate()[0]) @@ -375,7 +374,7 @@ def seed_non_shared_migrate(disks, force=False): def destroy(vm_): ''' - Hard power down the virtual machine, this is equivelent to pulling the + Hard power down the virtual machine, this is equivalent to pulling the power CLI Example: @@ -405,7 +404,7 @@ def undefine(vm_): def purge(vm_, dirs=False): ''' - Recursively destroy and delete a virtual machine, pass True for dirs to + Recursively destroy and delete a virtual machine, pass True for dir's to also delete the directories containing the virtual machine disk images - USE WITH EXTREAME CAUTION! @@ -426,7 +425,7 @@ def purge(vm_, dirs=False): def virt_type(): ''' Returns the virtual machine type as a string - + CLI Example: salt '*' virt.virt_type ''' diff --git a/salt/modules/yum.py b/salt/modules/yum.py index 4a24726c59..84884b6f7f 100644 --- a/salt/modules/yum.py +++ b/salt/modules/yum.py @@ -14,7 +14,7 @@ def __virtual__(): def _list_removed(old, new): ''' - List the pachages which have been removed between the two package objects + List the packages which have been removed between the two package objects ''' pkgs = [] for pkg in old: @@ -118,13 +118,13 @@ def upgrade(): pkgs[npkg] = {'old': '', 'new': new[npkg]} return pkgs - + def remove(pkg): ''' Remove a single package with yum remove Return a list containing the removed packages: - + CLI Example: salt '*' pkg.remove ''' @@ -139,7 +139,7 @@ def purge(pkg): Yum does not have a purge, this function calls remove Return a list containing the removed packages: - + CLI Example: salt '*' pkg.purge diff --git a/salt/returners/redis_return.py b/salt/returners/redis_return.py index 275b32d291..17181216a8 100644 --- a/salt/returners/redis_return.py +++ b/salt/returners/redis_return.py @@ -1,7 +1,7 @@ ''' Return data to a redis server This is a VERY simple example for pushing data to a redis server and is not -nessisarily intended as a usable interface. +necessarily intended as a usable interface. ''' import redis @@ -21,7 +21,7 @@ def returner(ret): host=__opts__['redis.host'], port=__opts__['redis.port'], db=__opts__['redis.db']) - serv.sadd(ret['id'] + ':' + 'jobs', ret['jid']) - serv.set(ret['jid'] + ':' + ret['id'], json.dumps(ret['return'])) + serv.sadd("%(id)s:jobs" % ret, ret['jid']) + serv.set("%(jid)s:%(id)s" % ret, json.dumps(ret['return'])) serv.sadd('jobs', ret['jid']) serv.sadd(ret['jid'], ret['id']) diff --git a/salt/state.py b/salt/state.py index d8131ac019..9bf9ddabf4 100644 --- a/salt/state.py +++ b/salt/state.py @@ -1,6 +1,6 @@ ''' The module used to execute states in salt. A state is unlike a module execution -in that instead of just executing a command it ensure that a certian state is +in that instead of just executing a command it ensure that a certain state is present on the system. The data sent to the state calls is as follows: @@ -15,6 +15,7 @@ import sys import os import copy import inspect +import tempfile # Import Salt modules import salt.loader @@ -71,7 +72,7 @@ class State(object): def format_call(self, data): ''' - Formats low data into a list of dicts used to acctually call the state, + Formats low data into a list of dict's used to actually call the state, returns: { 'full': 'module.function', @@ -110,7 +111,7 @@ class State(object): def compile_high_data(self, high): ''' - "Compile" the high data as it is retireved from the cli or yaml into + "Compile" the high data as it is retrieved from the cli or yaml into the individual state executor structures ''' chunks = [] @@ -148,8 +149,8 @@ class State(object): def compile_template(self, template): ''' - Take the path to a template and return the high data structure derived from the - template. + Take the path to a template and return the high data structure derived + from the template. ''' if not os.path.isfile(template): return {} @@ -157,8 +158,8 @@ class State(object): def compile_template_str(self, template): ''' - Take the path to a template and return the high data structure derived from the - template. + Take the path to a template and return the high data structure derived + from the template. ''' fn_ = tempfile.mkstemp()[1] open(fn_, 'w+').write(template) @@ -168,7 +169,8 @@ class State(object): def call(self, data): ''' - Call a state directly with the low data structure, verify data before processing + Call a state directly with the low data structure, verify data before + processing. ''' ret = {'changes': None, 'result': False, @@ -178,7 +180,7 @@ class State(object): def call_chunks(self, chunks): ''' - Itterate over a list of chunks and call them, checking for requires. + Iterate over a list of chunks and call them, checking for requires. ''' running = {} for low in chunks: diff --git a/salt/states/cmd.py b/salt/states/cmd.py index 786d7e71c5..b610134413 100644 --- a/salt/states/cmd.py +++ b/salt/states/cmd.py @@ -37,7 +37,7 @@ def run(name, return ret if unless: if __salt__['cmd.retcode'](unless) == 0: - ret['comment'] = 'unless executed sucsessfully' + ret['comment'] = 'unless executed successfully' ret['result'] = True return ret if not os.path.isdir(cwd): diff --git a/salt/states/file.py b/salt/states/file.py index 244c7bb11b..ef7e645862 100644 --- a/salt/states/file.py +++ b/salt/states/file.py @@ -3,7 +3,6 @@ Manage file states ''' import os -import hashlib import shutil import tempfile import difflib @@ -170,7 +169,7 @@ def managed(name, 'changes': changes, 'result': False, 'comment': 'Source file ' + source + ' not found'} - # Handle any template management that is neded + # Handle any template management that is needed if template: t_key = '_' + template if locals().has_key(t_key): diff --git a/salt/utils.py b/salt/utils.py index 96bb78816e..fd648fdec0 100644 --- a/salt/utils.py +++ b/salt/utils.py @@ -30,34 +30,34 @@ def daemonize(): ''' Daemonize a process ''' - try: - pid = os.fork() + try: + pid = os.fork() if pid > 0: # exit first parent - sys.exit(0) - except OSError, e: + sys.exit(0) + except OSError, e: print >> sys.stderr, "fork #1 failed: %d (%s)" % (e.errno, e.strerror) sys.exit(1) # decouple from parent environment - os.chdir("/") - os.setsid() - os.umask(022) + os.chdir("/") + os.setsid() + os.umask(022) # do second fork - try: - pid = os.fork() + try: + pid = os.fork() if pid > 0: - # print "Daemon PID %d" % pid - sys.exit(0) - except OSError, e: + # print "Daemon PID %d" % pid + sys.exit(0) + except OSError, e: print >> sys.stderr, "fork #2 failed: %d (%s)" % (e.errno, e.strerror) - sys.exit(1) + sys.exit(1) - dev_null = open('/dev/null','rw') - os.dup2(dev_null.fileno(), sys.stdin.fileno()) - os.dup2(dev_null.fileno(), sys.stdout.fileno()) - os.dup2(dev_null.fileno(), sys.stderr.fileno()) + dev_null = open('/dev/null','rw') + os.dup2(dev_null.fileno(), sys.stdin.fileno()) + os.dup2(dev_null.fileno(), sys.stdout.fileno()) + os.dup2(dev_null.fileno(), sys.stderr.fileno()) def check_root(): ''' @@ -65,9 +65,9 @@ def check_root(): verify that root is the user before the application discovers it. ''' if os.getuid(): - print 'Sorry, the salt must run as root, it needs to opperate'\ - + ' in a privileged environment to do what it does.\n' \ - + 'http://xkcd.com/838/' + print ('Sorry, the salt must run as root, it needs to operate ' + 'in a privileged environment to do what it does.\n' + 'http://xkcd.com/838/') sys.exit(1) diff --git a/scripts/salt b/scripts/salt index 0ee2864d50..b50d7616e3 100644 --- a/scripts/salt +++ b/scripts/salt @@ -2,8 +2,8 @@ ''' Publish commands to the salt system from the command line on the master. ''' -# Eventually we need to be able to interlink salt masters and form quorums, -# and create the ability for externel systesm to talk to salt. This interface +# Eventually we need to be able to interlink salt masters and form quorum's, +# and create the ability for external systems to talk to salt. This interface # is just a cli interface so that we can get this kicked off and working import salt.cli diff --git a/scripts/salt-cp b/scripts/salt-cp index c2436603dc..c014732de1 100644 --- a/scripts/salt-cp +++ b/scripts/salt-cp @@ -2,8 +2,8 @@ ''' Publish commands to the salt system from the command line on the master. ''' -# Eventually we need to be able to interlink salt masters and form quorums, -# and create the ability for externel systesm to talk to salt. This interface +# Eventually we need to be able to interlink salt masters and form quorum's, +# and create the ability for external systems to talk to salt. This interface # is just a cli interface so that we can get this kicked off and working import salt.cli