Merged salt-api into Salt

Woot!

The conflicts below are intentional because they contained salt-api-isms
that needed to be included into the corresponding Salt files.

Conflicts:
	debian/changelog
	debian/control
	doc/conf.py
	doc/index.rst
	opt_requirements.txt
	salt/config.py
	salt/loader.py
This commit is contained in:
Seth House 2014-06-18 21:27:49 -06:00
commit 53bab586f4
59 changed files with 8323 additions and 4 deletions

22
debian/control vendored
View File

@ -160,6 +160,28 @@ Description: public cloud VM management system
controlled profile and mapping system.
Package: salt-api
Architecture: all
Depends: ${python:Depends},
${misc:Depends},
${shlibs:Depends},
python,
salt-master
Recommends: python-cherrypy3
Description: Generic, modular network access system
a modular interface on top of Salt that can provide a variety of entry points
into a running Salt system. It can start and manage multiple interfaces
allowing a REST API to coexist with XMLRPC or even a Websocket API.
.
The Salt API system is used to expose the fundamental aspects of Salt control
to external sources. salt-api acts as the bridge between Salt itself and
REST, Websockets, etc.
.
Documentation is available on Read the Docs:
.
http://salt-api.readthedocs.org/
Package: salt-doc
Architecture: all
Section: doc

1
debian/salt-api.install vendored Normal file
View File

@ -0,0 +1 @@
scripts/salt-api /usr/bin

2
debian/salt-api.manpages vendored Normal file
View File

@ -0,0 +1,2 @@
doc/man/salt-api.1
doc/man/salt-api.7

View File

@ -24,10 +24,14 @@ class Mock(object):
pass
def __call__(self, *args, **kwargs):
return Mock()
ret = Mock()
# If mocked function is used as a decorator, expose decorated function.
if args and callable(args[0]):
functools.update_wrapper(ret, args[0])
return ret
@classmethod
def __getattr__(self, name):
def __getattr__(cls, name):
if name in ('__file__', '__path__'):
return '/dev/null'
else:
@ -48,7 +52,8 @@ MOCK_MODULES = [
'yaml.nodes',
'yaml.scanner',
'zmq',
# salt.cloud
# third-party libs for cloud modules
'libcloud',
'libcloud.compute',
'libcloud.compute.base',
@ -60,6 +65,27 @@ MOCK_MODULES = [
'libcloud.loadbalancer.providers',
'libcloud.common',
'libcloud.common.google',
# third-party libs for netapi modules
'cherrypy',
'cherrypy.lib',
'cherrypy.process',
'cherrypy.wsgiserver',
'cherrypy.wsgiserver.ssl_builtin',
'tornado',
'tornado.concurrent',
'tornado.gen',
'tornado.httpserver',
'tornado.ioloop',
'tornado.web',
'tornado.websocket',
'ws4py',
'ws4py.server',
'ws4py.server.cherrypyserver',
'ws4py.websocket',
# modules, renderers, states, returners, et al
'django',
'libvirt',
@ -139,6 +165,7 @@ extensions = [
'sphinx.ext.autosummary',
'sphinx.ext.extlinks',
'sphinx.ext.intersphinx',
'sphinxcontrib.httpdomain',
'youtube',
'saltautodoc', # Must be AFTER autodoc
'shorturls',

View File

@ -225,9 +225,13 @@ Salt is many splendid things.
Controlling devices and machines unable to run a salt-minion.
:ref:`Python API interface <python-api>`
Use Salt programmatically from scripts and programs easily and
Use Salt locally from scripts and programs easily and
simply via ``import salt``.
:ref:`External API interfaces <netapi-introduction>`
Expose a Salt API such as REST, XMPP, WebSockets, and more using netapi
modules. See the :ref:`full list of netapi modules <all-netapi-modules>`.
:doc:`Automatic Updates and Frozen Binary Deployments <topics/tutorials/esky>`
Use a frozen install to make deployments easier (Even on Windows!). Or
take advantage of automatic updates to keep minions running the latest

70
doc/man/salt-api.1 Normal file
View File

@ -0,0 +1,70 @@
.\" Man page generated from reStructuredText.
.
.TH "SALT-API" "1" "April 05, 2014" "0.8.3" "salt-api"
.SH NAME
salt-api \- salt-api
.
.nr rst2man-indent-level 0
.
.de1 rstReportMargin
\\$1 \\n[an-margin]
level \\n[rst2man-indent-level]
level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
-
\\n[rst2man-indent0]
\\n[rst2man-indent1]
\\n[rst2man-indent2]
..
.de1 INDENT
.\" .rstReportMargin pre:
. RS \\$1
. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin]
. nr rst2man-indent-level +1
.\" .rstReportMargin post:
..
.de UNINDENT
. RE
.\" indent \\n[an-margin]
.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]]
.nr rst2man-indent-level -1
.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]]
.in \\n[rst2man-indent\\n[rst2man-indent-level]]u
..
.sp
Start interfaces used to remotely connect to the salt master
.SH SYNOPSIS
.INDENT 0.0
.INDENT 3.5
.sp
.nf
.ft C
salt\-api
.ft P
.fi
.UNINDENT
.UNINDENT
.SH DESCRIPTION
.sp
The Salt API system manages network api connectors for the Salt Master
.SH OPTIONS
.INDENT 0.0
.TP
.B \-h, \-\-help
Print a usage message briefly summarizing these command\-line options.
.UNINDENT
.INDENT 0.0
.TP
.B \-C CONFIG, \-\-config=CONFIG
Specify an alternative location for the salt master configuration file.
.UNINDENT
.SH SEE ALSO
.sp
\fIsalt\-api(7)\fP
\fIsalt(7)\fP
\fIsalt\-master(1)\fP
.SH AUTHOR
Thomas S. Hatch <thatch45@gmail.com> and many others, please see the Authors file
.SH COPYRIGHT
2012, Thomas S. Hatch
.\" Generated by docutils manpage writer.
.

2237
doc/man/salt-api.7 Normal file

File diff suppressed because it is too large Load Diff

37
doc/ref/cli/salt-api.rst Normal file
View File

@ -0,0 +1,37 @@
============
``salt-api``
============
Start interfaces used to remotely connect to the salt master
Synopsis
========
::
salt-api
Description
===========
The Salt API system manages network api connectors for the Salt Master
Options
=======
.. program:: salt-api
.. option:: -h, --help
Print a usage message briefly summarizing these command-line options.
.. option:: -C CONFIG, --config=CONFIG
Specify an alternative location for the salt master configuration file.
See also
========
:manpage:`salt-api(7)`
:manpage:`salt(7)`
:manpage:`salt-master(1)`

View File

@ -0,0 +1,12 @@
.. _all-netapi-modules:
===========================
Full list of netapi modules
===========================
.. toctree::
:maxdepth: 2
salt.netapi.rest_cherrypy
salt.netapi.rest_tornado
salt.netapi.rest_wsgi

View File

@ -0,0 +1,77 @@
=============
rest_cherrypy
=============
.. automodule:: salt.netapi.rest_cherrypy.app
.. automodule:: salt.netapi.rest_cherrypy.wsgi
.. ............................................................................
REST URI Reference
==================
.. py:currentmodule:: salt.netapi.rest_cherrypy.app
.. contents::
:local:
``/``
-----
.. autoclass:: LowDataAdapter
:members: GET, POST
``/login``
----------
.. autoclass:: Login
:members: GET, POST
``/logout``
-----------
.. autoclass:: Logout
:members: POST
``/minions``
------------
.. autoclass:: Minions
:members: GET, POST
``/jobs``
---------
.. autoclass:: Jobs
:members: GET
``/run``
--------
.. autoclass:: Run
:members: POST
``/events``
-----------
.. autoclass:: Events
:members: GET
``/ws``
-------
.. autoclass:: WebsocketEndpoint
:members: GET
``/hook``
---------
.. autoclass:: Webhook
:members: POST
``/stats``
----------
.. autoclass:: Stats
:members: GET

View File

@ -0,0 +1,7 @@
=============
rest_tornado
=============
.. automodule:: salt.netapi.rest_tornado.saltnado
.. ............................................................................

View File

@ -0,0 +1,7 @@
=========
rest_wsgi
=========
.. automodule:: salt.netapi.rest_wsgi
.. py:currentmodule:: salt.netapi.rest_wsgi

View File

@ -0,0 +1,36 @@
.. _netapi-introduction:
==============================
Introduction to netapi modules
==============================
netapi modules provide API-centric access to Salt. Usually externally-facing
services such as REST or WebSockets, XMPP, XMLRPC, etc.
In general netapi modules bind to a port and start a service. They are
purposefully open-ended. A single module can be configured to run as well as
multiple modules simultaneously.
netapi modules are enabled by adding configuration to your Salt Master config
file and then starting the :command:`salt-api` daemon. Check the docs for each
module to see external requirements and configuration settings.
Communication with Salt and Salt satellite projects is done using Salt's own
:ref:`Python API <python-api>`. A list of available client interfaces is below.
.. admonition:: salt-api
Prior to Salt's Helium release, netapi modules lived in the separate sister
projected ``salt-api``. That project has been merged into the main Salt
project.
Client interfaces
=================
Salt's client interfaces expose executing functions by crafting a dictionary of
values that are mapped to function arguments. This allows calling functions
simply by creating a data structure. (And this is exactly how much of Salt's
own internals work!)
.. autoclass:: salt.netapi.APIClient
:members: local, local_async, local_batch, runner, wheel

View File

@ -0,0 +1,54 @@
======================
Writing netapi modules
======================
:py:mod:`~salt.netapi` modules, put simply, bind a port and start a service.
They are purposefully open-ended and can be used to present a variety of
external interfaces to Salt, and even present multiple interfaces at once.
.. seealso:: :ref:`The full list of netapi modules <all-netapi-modules>`
Configuration
=============
All :py:mod:`~salt.netapi` configuration is done in the :ref:`Salt master
config <configuration-salt-master>` and takes a form similar to the following:
.. code-block:: yaml
rest_cherrypy:
port: 8000
debug: True
ssl_crt: /etc/pki/tls/certs/localhost.crt
ssl_key: /etc/pki/tls/certs/localhost.key
The ``__virtual__`` function
============================
Like all module types in Salt, :py:mod:`~salt.netapi` modules go through
Salt's loader interface to determine if they should be loaded into memory and
then executed.
The ``__virtual__`` function in the module makes this determination and should
return ``False`` or a string that will serve as the name of the module. If the
module raises an ``ImportError`` or any other errors, it will not be loaded.
The ``start`` function
======================
The ``start()`` function will be called for each :py:mod:`~salt.netapi`
module that is loaded. This function should contain the server loop that
actually starts the service. This is started in a multiprocess.
Inline documentation
====================
As with the rest of Salt, it is a best-practice to include liberal inline
documentation in the form of a module docstring and docstrings on any classes,
methods, and functions in your :py:mod:`~salt.netapi` module.
Loader “magic” methods
======================
The loader makes the ``__opts__`` data structure available to any function in
a :py:mod:`~salt.netapi` module.

View File

@ -0,0 +1,28 @@
==============
salt-api 0.5.0
==============
:program:`salt-api` is gearing up for the initial public release with 0.5.0.
Although this release ships with working basic functionality it is awaiting the
authentication backend that will be introduced in Salt 0.10.4 before it can be
considered ready for testing at large.
REST API
========
This release presents the flagship netapi module which provides a RESTful
interface to a running Salt system. It allows for viewing minions, runners, and
jobs as well as running execution modules and runners of a running Salt system
through a REST API that returns JSON.
Participation
=============
:program:`salt-api` is just getting off the ground so feedback, questions, and
ideas are critical as we solidify how this project fits into the overall Salt
infrastructure management stack. Please get involved by `filing issues`__ on
GitHub, `discussing on the mailing list`__, and chatting in ``#salt`` on
Freenode.
.. __: https://github.com/saltstack/salt-api/issues
.. __: https://groups.google.com/forum/#!forum/salt-users

View File

@ -0,0 +1,50 @@
==============
salt-api 0.6.0
==============
:program:`salt-api` inches closer to prime-time with 0.6.0. This release adds
the beginnings of a universal interface for accessing Salt components via the
tried and true method of passing low-data to functions (a core component of
Salt's remote execution and state management).
Low-data interface
==================
A new view accepts :http:post: requests at the root URL that accepts raw
low-data as :http:post: data and passes that low-data along to a client
interface in Salt. Currently only LocalClient and RunnerClient interfaces have
been implemented in Salt with more coming in the next Salt release.
External authentication
-----------------------
Raw low-data can contain authentication credentials that make use of Salt's new
:conf_master:`external_auth` system.
The following is a proof-of-concept of a working eauth call. (It bears
repeating this is a pre-alpha release and this should not be used by anyone for
anything real.)
.. code-block:: bash
% curl -si localhost:8000 \
-d client=local \
-d tgt='*' \
-d fun='test.ping' \
-d arg \
-d eauth=pam \
-d username=saltdev \
-d password=saltdev
Participation
=============
:program:`salt-api` is just getting off the ground so feedback, questions, and
ideas are critical as we solidify how this project fits into the overall Salt
infrastructure management stack. Please get involved by `filing issues`__ on
GitHub, `discussing on the mailing list`__, and chatting in ``#salt-devel`` on
Freenode.
.. __: https://github.com/saltstack/salt-api/issues
.. __: https://groups.google.com/forum/#!forum/salt-users

View File

@ -0,0 +1,49 @@
==============
salt-api 0.7.0
==============
:program:`salt-api` is ready for alpha-testing in the real world. This release
solidifies how :program:`salt-api` will communicate with the larger Salt
ecosystem. In addition authentication and encryption (via SSL) have been added.
The first netapi module was a proof of concept written in Flask. It was quite
useful to be able to quickly hammer out a URL structure and solidify on an
interface for programmatically calling out to Salt components. As of this
release that module has been deprecated and removed in favor of a netapi module
written in CherryPy. CherryPy affords tremendous flexibility when composing a
REST interface and will present a stable platform for building out a very
adaptable and featureful REST API—also we're using the excellent and fast
CherryPy webserver for securely serving the API.
Low-data interface
==================
The last release introduced a proof-of-concept for how the various Salt
components will communicate with each other. This is done by passing a data
structure to a client interface. This release expands on that. There are
currently three client interfaces in Salt.
.. seealso:: :ref:`netapi-introduction`
Encryption and authentication
=============================
Encryption has been added via SSL. You can supply an existing certificate or
generate a self-signed certificate through Salt's :py:mod:`~salt.modules.tls`
module.
Authentication is performed through Salt's incredibly flexible :ref:`external
auth <acl-eauth>` system and is maintained when accessing the API via session
tokens.
Participation
=============
:program:`salt-api` is just getting off the ground so feedback, questions, and
ideas are critical as we solidify how this project fits into the overall Salt
infrastructure management stack. Please get involved by `filing issues`__ on
GitHub, `discussing on the mailing list`__, and chatting in ``#salt-devel`` on
Freenode.
.. __: https://github.com/saltstack/salt-api/issues
.. __: https://groups.google.com/forum/#!forum/salt-users

View File

@ -0,0 +1,35 @@
==============
salt-api 0.7.5
==============
This release is a mostly a minor release to pave a better path for
:program:`salt-ui` though there are some small feature additions and bugfixes.
Changes
=======
* Convenience URLs ``/minions`` and ``/jobs`` have been added as well as a
async client wrapper. This starts a job and immediately returns the job ID,
allowing you to fetch the result of that job at a later time.
* The return format will now default to JSON if no specific format is
requested.
* A new setting ``static`` has been added that will serve any static media from
the directory specified. In addition if an :file:`index.html` file is found
in that directory and the ``Accept`` header in the request prefer HTML that
file will be served.
* All HTML, including the login form, has been removed from :program:`salt-api`
and moved into the :program:`salt-ui` project.
* Sessions now live as long as the Salt token.
Participation
=============
:program:`salt-api` is just getting off the ground so feedback, questions, and
ideas are critical as we solidify how this project fits into the overall Salt
infrastructure management stack. Please get involved by `filing issues`__ on
GitHub, `discussing on the mailing list`__, and chatting in ``#salt-devel`` on
Freenode.
.. __: https://github.com/saltstack/salt-api/issues
.. __: https://groups.google.com/forum/#!forum/salt-users

View File

@ -0,0 +1,166 @@
==============
salt-api 0.8.0
==============
We are happy to announce the release of :program:`salt-api` 0.8.0.
This release encompasses bugfixes and new features for the
:py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>` netapi module that
provides a RESTful interface for a running Salt system.
.. note::
Requires Salt 0.13
Changes
=======
In addition to the usual documentation improvements and bug fixes this release
introduces the following changes and additions.
Please note the backward incompatible change detailed below.
RPM packaging
-------------
Thanks to Andrew Niemantsvedriet (`@kaptk2`_) :program:`salt-api` is now
available in Fedora package repositories as well as RHEL compatible systems via
EPEL.
* http://dl.fedoraproject.org/pub/epel/5/i386/repoview/salt-api.html
* http://dl.fedoraproject.org/pub/epel/5/x86_64/repoview/salt-api.html
* http://dl.fedoraproject.org/pub/epel/6/i386/repoview/salt-api.html
* http://dl.fedoraproject.org/pub/epel/6/x86_64/repoview/salt-api.html
Thanks also to Clint Savage (`@herlo`_) and Thomas Spura (`@tomspur`_) for
helping with that process.
.. _`@kaptk2`: https://github.com/kaptk2
.. _`@herlo`: https://github.com/herlo
.. _`@tomspur`: https://github.com/tomspur
Ubuntu PPA packaging
--------------------
Thanks to Sean Channel (`@seanchannel`_, pentabular) :program:`salt-api` is
available as a PPA on the SaltStack LaunchPad team.
https://launchpad.net/~saltstack/+archive/salt
.. _`@seanchannel`: https://github.com/seanchannel
Authentication information on login
-----------------------------------
.. warning:: Backward incompatible change
The :py:class:`/login <salt.netapi.rest_cherrypy.app.Login>` URL no
longer responds with a 302 redirect for success.
Although this is behavior is common in the browser world it is not useful
from an API so we have changed it to return a 200 response in this release.
We take backward compatibility very seriously and we apologize for the
inconvenience. In this case we felt the previous behavior was limiting.
Changes such as this will be rare.
New in this release is displaying information about the current session and the
current user. For example::
% curl -sS localhost:8000/login \
-H 'Accept: application/x-yaml'
-d username='saltdev'
-d password='saltdev'
-d eauth='pam'
return:
- eauth: pam
expire: 1365508324.359403
perms:
- '@wheel'
- grains.*
- state.*
- status.*
- sys.*
- test.*
start: 1365465124.359402
token: caa7aa2b9dbc4a8adb6d2e19c3e52be68995ef4b
user: saltdev
Bypass session handling
-----------------------
A convenience URL has been added
(:py:class:`/run <salt.netapi.rest_cherrypy.app.Run>`) to bypass the normal
session-handling process.
The REST interface uses the concept of "lowstate" data to specify what function
should be executed in Salt (plus where that function is and any arguments to
the function). This is a thin wrapper around Salt's various "client"
interfaces, for example Salt's :ref:`LocalClient() <python-api>` which can
accept authentication credentials directly.
Authentication with the REST API typically goes through the login URL and a
session is generated that is tied to a Salt external_auth token. That token is
then automatically added to the lowstate for subsequent requests that match the
current session.
It is sometimes useful to handle authentication or token management manually
from another program or script. For example::
curl -sS localhost:8000/run \
-d client='local' \
-d tgt='*' \
-d fun='test.ping' \
-d eauth='pam' \
-d username='saltdev' \
-d password='saltdev'
It is a Bad Idea (TM) to do this unless you have a very good reason and a well
thought out security model.
Logout
------
An URL has been added
(:py:class:`/logout <salt.netapi.rest_cherrypy.app.Logout>`) that will cause
the client-side to expire the session cookie and the server-side session to be
invalidated.
Running the REST interface via any WSGI-compliant server
--------------------------------------------------------
The :py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>` netapi module is
a regular WSGI application written using the CherryPy framework. It was written
with the intent of also running from any WSGI-compliant server such as Apache
and mod_wsgi, Gunicorn, uWSGI, Nginx and FastCGI, etc.
The WSGI application entry point has been factored out into a stand-alone file
in this release suitable for calling from an external server.
:program:`salt-api` does not need to be running in this scenario.
For example, an Apache virtual host configuration::
<VirtualHost *:80>
ServerName example.com
ServerAlias *.example.com
ServerAdmin webmaster@example.com
LogLevel warn
ErrorLog /var/www/example.com/logs/error.log
CustomLog /var/www/example.com/logs/access.log combined
DocumentRoot /var/www/example.com/htdocs
WSGIScriptAlias / /path/to/salt/netapi/rest_cherrypy/wsgi.py
</VirtualHost>
Participation
=============
Please get involved by `filing issues`__ on GitHub, `discussing on the mailing
list`__, and chatting in ``#salt-devel`` on Freenode.
.. __: https://github.com/saltstack/salt-api/issues
.. __: https://groups.google.com/forum/#!forum/salt-users

View File

@ -0,0 +1,25 @@
==============
salt-api 0.8.2
==============
:program:`salt-api` 0.8.2 is largely a bugfix release that fixes a
compatibility issue with changes in Salt 0.15.9.
.. note::
Requires Salt 0.15.9 or greater
The following changes have been made to the :py:mod:`rest_cherrypy
<salt.netapi.rest_cherrypy.app>` netapi module that provides a RESTful
interface for a running Salt system:
* Fixed issue #87 which caused the Salt master's PID file to be overwritten.
* Fixed an inconsistency with the return format for the ``/minions``
convenience URL.
.. warning::
This is a backward incompatible change.
* Added a dedicated URL for serving an HTML app
* Added dedicated URL for serving static media

View File

@ -0,0 +1,44 @@
==============
salt-api 0.8.3
==============
:program:`salt-api` 0.8.3 is a small release largely concerning changes and
fixes to the :py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>` netapi
module.
This release will likely be the final salt-api release as a separate project.
The Salt team has begun the process of merging this project directly in to the
main Salt project. What this means for end users is only that there will be one
fewer package to install. Salt itself will ship with the current ``netapi``
modules and the API and configuration will remain otherwise unchanged.
The reasoning behind merging the two projects is simply to lower the barrier to
entry. Having a separate project was useful for experimentation and exploration
but there was no technical reason for the separation -- salt-api uses the same
flexible module system that Salt uses and those modules will simply be moved
into Salt.
Going forward, Salt will ship with the same REST interface that salt-api
currently provides. This will have the side benefit of not having to coordinate
incompatible Salt and salt-api releases.
:py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>` changes
==================================================================
An HTTP stream of Salt's event bus has been added. This stream conforms to the
SSE (Server Sent Events) spec and is easily consumed via JavaScript clients.
This HTTP stream allows a real-time window into a running Salt system. A client
watching the stream can see as soon as individual minions return data for a
job, authentication events, and any other events that go through the Salt
master.
A new configuration option to only allow access to whitelisted IP addresses. Of
course, IP addresses can be easily spoofed so this feature should be thought of
as a usability addition and not used for security purposes.
An option to disable SSL has been added. Previously SSL could only be disabled
while running the HTTP server with debugging options enabled. Now each item can
be enabled or disabled independently of the other.
In addition, there has been several bug fixes, packaging fixes, and minor code
simplification.

View File

@ -0,0 +1,102 @@
==============
salt-api 0.8.4
==============
:program:`salt-api` 0.8.4 sees a number of new features and feature
enhancements in the :py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>`
netapi module.
Work to merge :program:`salt-api` into the main Salt distribution continues and
it is likely to be included in Salt's Helium release.
:py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>` changes
==================================================================
Web hooks
---------
This release adds a :py:class:`new URL /hook
<salt.netapi.rest_cherrypy.app.Webhook>` that allows salt-api to serve as a
generic web hook interface for Salt. POST requests to the URL trigger events on
Salt's event bus.
External services like Amazon SNS, Travis CI, GitHub, etc can easily send
signals through Salt's Reactor.
The following HTTP call will trigger the following Salt event.
.. code-block:: bash
% curl -sS http://localhost:8000/hook/some/tag \
-d some='Data!'
Event tag: ``salt/netapi/hook/some/tag``. Event data:
.. code-block:: json
{
"_stamp": "2014-04-04T12:14:54.389614",
"post": {
"some": "Data!"
},
"headers": {
"Content-Type": "application/x-www-form-urlencoded",
"Host": "localhost:8000",
"User-Agent": "curl/7.32.0",
"Accept": "*/*",
"Content-Length": "10",
"Remote-Addr": "127.0.0.1"
}
}
Batch mode
----------
The :py:meth:`~salt.APIClient.local_batch` client exposes Salt's batch mode
for executing commands on incremental subsets of minions.
Tests!
------
We have added the necessary framework for testing the rest_cherrypy module and
this release includes a number of both unit and integration tests. The suite
can be run with the following command:
.. code-block:: bash
python -m unittest discover -v
CherryPy server stats and configuration
---------------------------------------
A number of settings have been added to better configure the performance of the
CherryPy web server. In addition, a :py:class:`new URL /stats
<salt.netapi.rest_cherrypy.app.Stats>` has been added to expose metrics on
the health of the CherryPy web server.
Improvements for running with external WSGI servers
---------------------------------------------------
Running the ``rest_cherrypy`` module via a WSGI-capable server such as Apache
or Nginx can be tricky since the user the server is running as must have
permission to access the running Salt system. This release eases some of those
restrictions by accessing Salt's key interface through the external auth
system. Read access to the Salt configuration is required for the user the
server is running as and everything else should go through external auth.
More information in the jobs URLs
---------------------------------
The output for the :py:class:`/jobs/<jid> URLs
<salt.netapi.rest_cherrypy.app.Jobs>` has been augmented with more
information about the job such as which minions are expected to return for that
job. This same output will be added to the other salt-api URLs in the next
release.
Improvements to the Server Sent Events stream
---------------------------------------------
Event tags have been added to :py:class:`the HTTP event stream
<salt.netapi.rest_cherrypy.app.Event>` as SSE tags which allows JavaScript
or other consumers to more easily match on certain tags without having to
inspect the whole event.

View File

@ -0,0 +1,9 @@
=============
Release notes
=============
.. releasestree::
:maxdepth: 1
:glob:
*

View File

@ -3,3 +3,4 @@ timelib
yappi >= 0.8.2
--allow-unverified python-novaclient > 2.17.0
python-gnupg
cherrypy>=3.2.2

View File

@ -0,0 +1,32 @@
# Maintainer: Christer Edwards <christer.edwards@gmail.com>
pkgname=salt-api
pkgver=0.8.0
pkgrel=1
pkgdesc="Salt API is used to expose the fundamental aspects of Salt control to external sources."
arch=(any)
url="https://github.com/saltstack/salt-api"
license=("APACHE")
depends=('python2'
'salt')
backup=()
makedepends=()
optdepends=()
options=()
conflicts=()
source=("http://pypi.python.org/packages/source/s/${pkgname}/${pkgname}-${pkgver}.tar.gz"
salt-api.service)
md5sums=('e9239a7184ced5d426696735456ee829'
'37f667db44f63fb5dd7b81acf736b0db')
package() {
cd ${srcdir}/${pkgname}-${pkgver}
python2 setup.py install --root=${pkgdir}/ --optimize=1
install -Dm644 ${srcdir}/salt-api.service ${pkgdir}/usr/lib/systemd/system/salt-api.service
}

View File

@ -0,0 +1,59 @@
# Maintainer: Christer Edwards <christer.edwards@gmail.com>
pkgname=salt-api-git
_gitname=salt-api
pkgver=0.0.0
pkgrel=1
pkgdesc="Salt API is used to expose the fundamental aspects of Salt control to external sources."
arch=('i686' 'x86_64')
url="https://github.com/saltstack/salt-api"
license=("APACHE")
depends=('python2'
'salt')
backup=()
makedepends=('git')
optdepends=()
options=()
conflicts=('salt-api')
provides=('salt-api')
# makepkg 4.1 knows about git and will pull main branch
source=("git://github.com/saltstack/salt-api.git")
# makepkg knows it's a git repo because the url starts with 'git'
# it then knows to checkout the branch 'pacman41' upon cloning, expediting versioning.
# branch="develop"
# source=("git://github.com/saltstack/salt-api.git#branch=$branch")
# makepkg also knows about tags
#tags="v0.8.0"
#source=("git://github.com/saltstack/salt-api.git#tag=$tag")
# because the sources are not static, skip checksums
md5sums=('SKIP')
pkgver() {
cd "$srcdir/$_gitname"
echo $(git describe --always | sed 's/-/./g')
# for git, if the repo has no tags, comment out the above and uncomment the next line:
#echo "0.$(git rev-list --count $branch).$(git describe --always)"
# This will give you a count of the total commits and the hash of the commit you are on.
# Useful if you're making a repository with git packages so that they can have sequential
# version numbers. (Else a pacman -Syu may not update the package)
}
#build() {
# cd "${srcdir}/${_gitname}"
# python2 setup.py build
# no need to build setup.py install will do this
#}
package() {
cd "${srcdir}/${_gitname}"
export USE_SETUPTOOLS=true
python2 setup.py install --root=${pkgdir}/ --optimize=1
install -Dm644 ${srcdir}/salt-api/pkg/salt-api.service ${pkgdir}/usr/lib/systemd/system/salt-api.service
# remove vcs leftovers
find "$pkgdir" -type d -name .git -exec rm -r '{}' +
}

153
pkg/rpm/salt-api Normal file
View File

@ -0,0 +1,153 @@
#!/bin/sh
#
# Salt API
###################################
# LSB header
### BEGIN INIT INFO
# Provides: salt-api
# Required-Start: $local_fs $remote_fs $network $named $time
# Should-Start: $time ypbind smtp
# Required-Stop: $local_fs $remote_fs $network $named $time
# Should-Stop: ypbind smtp
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: Salt API control daemon
# Description: This is a daemon that controls the Salt API.
### END INIT INFO
# chkconfig header
# chkconfig: 345 99 99
# description: This is a daemon that controls the Salt API.
#
# processname: /usr/bin/salt-api
if [ -f /etc/default/salt ]; then
. /etc/default/salt
else
SALTAPI=/usr/bin/salt-api
PYTHON=/usr/bin/python
fi
# Sanity checks.
[ -x $SALTAPI ] || exit 0
DEBIAN_VERSION=/etc/debian_version
SUSE_RELEASE=/etc/SuSE-release
# Source function library.
if [ -f $DEBIAN_VERSION ]; then
break
elif [ -f $SUSE_RELEASE -a -r /etc/rc.status ]; then
. /etc/rc.status
else
. /etc/rc.d/init.d/functions
fi
SERVICE=salt-api
PROCESS=salt-api
CONFIG_ARGS="-d"
PID_FILE="/var/run/salt-api.pid"
RETVAL=0
start() {
echo -n $"Starting salt-api daemon: "
if [ -f $SUSE_RELEASE ]; then
startproc -f -p /var/run/$SERVICE.pid $SALTAPI $CONFIG_ARGS
rc_status -v
elif [ -e $DEBIAN_VERSION ]; then
if [ -f $LOCKFILE ]; then
echo -n "already started, lock file found"
RETVAL=1
elif $PYTHON $SALTAPI; then
echo -n "OK"
RETVAL=0
fi
else
if status $PROCESS &> /dev/null; then
failure "Already running."
RETVAL=1
else
daemon --pidfile=$PID_FILE --check $SERVICE $SALTAPI $CONFIG_ARGS
RETVAL=0
fi
fi
RETVAL=$?
echo
return $RETVAL
}
stop() {
echo -n $"Stopping salt-api daemon: "
if [ -f $SUSE_RELEASE ]; then
killproc -TERM $SALTAPI
rc_status -v
elif [ -f $DEBIAN_VERSION ]; then
# Added this since Debian's start-stop-daemon doesn't support spawned processes
if ps -ef | grep "$PYTHON $SALTAPI" | grep -v grep | awk '{print $2}' | xargs kill &> /dev/null; then
echo -n "OK"
RETVAL=0
else
echo -n "Daemon is not started"
RETVAL=1
fi
else
if [ -f $PID_FILE ] && cat $PID_FILE | xargs pkill -P &> /dev/null; then
success
RETVAL=0
rm -f $PID_FILE
else
failure "$PID_FILE does not exist or could not kill."
RETVAL=1
fi
fi
RETVAL=$?
echo
return $RETVAL
}
restart() {
stop
start
}
# See how we were called.
case "$1" in
start|stop|restart)
$1
;;
status)
if [ -f $SUSE_RELEASE ]; then
echo -n "Checking for service salt-api "
checkproc $SALTAPI
rc_status -v
elif [ -f $DEBIAN_VERSION ]; then
if [ -f $LOCKFILE ]; then
RETVAL=0
echo "salt-api is running."
else
RETVAL=1
echo "salt-api is stopped."
fi
else
status $PROCESS
RETVAL=$?
fi
;;
condrestart)
[ -f $LOCKFILE ] && restart || :
;;
reload)
echo "can't reload configuration, you have to restart it"
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|reload}"
exit 1
;;
esac
exit $RETVAL

167
pkg/rpm/salt-api.spec Normal file
View File

@ -0,0 +1,167 @@
%if ! (0%{?rhel} >= 6 || 0%{?fedora} > 12)
%global with_python26 1
%define pybasever 2.6
%define __python_ver 26
%define __python %{_bindir}/python%{?pybasever}
%endif
%define namespace saltapi
%define eggspace salt_api
%{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")}
%{!?python_sitearch: %global python_sitearch %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib(1))")}
Name: salt-api
Version: 0.8.3
Release: 0%{?dist}
Summary: A web api for to access salt the parallel remote execution system
Group: System Environment/Daemons
License: ASL 2.0
URL: http://github.com/saltstack/salt-api
Source0: http://pypi.python.org/packages/source/s/%{name}/%{name}-%{version}.tar.gz
Source1: %{name}.service
Source2: %{name}
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildArch: noarch
BuildRequires: python2-devel
Requires: salt
Requires: python-cherrypy
%if ! (0%{?rhel} >= 7 || 0%{?fedora} >= 15)
Requires(post): chkconfig
Requires(preun): chkconfig
Requires(preun): initscripts
Requires(postun): initscripts
%else
%if 0%{?systemd_preun:1}
Requires(post): systemd-units
Requires(preun): systemd-units
Requires(postun): systemd-units
%endif
BuildRequires: systemd-units
%endif
%description
salt-api is a modular interface on top of Salt that can provide a variety of
entry points into a running Salt system. It can start and manage multiple
interfaces allowing a REST API to coexist with XMLRPC or even a Websocket API.
%prep
%setup -q
%build
%install
rm -rf $RPM_BUILD_ROOT
%{__python} setup.py install -O1 --root $RPM_BUILD_ROOT
%if ! (0%{?rhel} >= 7 || 0%{?fedora} >= 15)
mkdir -p $RPM_BUILD_ROOT%{_initrddir}
install -p %{SOURCE2} $RPM_BUILD_ROOT%{_initrddir}/
%else
mkdir -p $RPM_BUILD_ROOT%{_unitdir}
install -p -m 0644 %{SOURCE1} $RPM_BUILD_ROOT%{_unitdir}/
%endif
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc LICENSE
%{_bindir}/%{name}
%{python_sitelib}/%{namespace}/*
%{python_sitelib}/%{eggspace}-%{version}-py?.?.egg-info
%doc %{_mandir}/man1/%{name}.1*
%doc %{_mandir}/man7/%{name}.7*
%if ! (0%{?rhel} >= 7 || 0%{?fedora} >= 15)
%attr(0755, root, root) %{_initrddir}/%{name}
%else
%{_unitdir}/%{name}.service
%endif
# less than RHEL 8 / Fedora 16
# not sure if RHEL 7 will use systemd yet
%if ! (0%{?rhel} >= 7 || 0%{?fedora} >= 15)
%preun
if [ $1 -eq 0 ] ; then
/sbin/service %{name} stop >/dev/null 2>&1
/sbin/chkconfig --del %{name}
fi
%post
/sbin/chkconfig --add %{name}
%postun
if [ "$1" -ge "1" ] ; then
/sbin/service %{name} condrestart >/dev/null 2>&1 || :
fi
%else
%preun
%if 0%{?systemd_preun:1}
%systemd_preun %{name}.service
%else
if [ $1 -eq 0 ] ; then
# Package removal, not upgrade
/bin/systemctl --no-reload disable %{name}.service > /dev/null 2>&1 || :
/bin/systemctl stop %{name}.service > /dev/null 2>&1 || :
fi
%endif
%post
%if 0%{?systemd_post:1}
%systemd_post %{name}.service
%else
/bin/systemctl daemon-reload &>/dev/null || :
%endif
%postun
%if 0%{?systemd_post:1}
%systemd_postun %{name}.service
%else
/bin/systemctl daemon-reload &>/dev/null
[ $1 -gt 0 ] && /bin/systemctl try-restart %{name}.service &>/dev/null || :
%endif
%endif
%changelog
* Wed Jul 17 2013 Andrew Niemantsverdriet <andrewniemants@gmail.com> - 0.8.2-0
- Bugfix release that fixes a compatibility issue with changes in Salt 0.15.9.
- Fixed an inconsistency with the return format for the /minions convenience URL.
- Added a dedicated URL for serving an HTML app and static media
* Tue Apr 16 2013 Andrew Niemantsverdriet <andrewniemants@gmail.com> - 0.8.1-0
- Minor bugfix version released
* Tue Apr 16 2013 Andrew Niemantsverdriet <andrewniemants@gmail.com> - 0.8.0-0
- New version released
* Tue Feb 25 2013 Andrew Niemantsverdriet <andrewniemants@gmail.com> - 0.7.5-3
- Added a more detailed decription
- Removed trailing whitespace on description.
- Added BR of python-devel
* Tue Feb 25 2013 Andrew Niemantsverdriet <andrewniemants@gmail.com> - 0.7.5-2
- Fixes as suggested by https://bugzilla.redhat.com/show_bug.cgi?id=913296#
* Tue Feb 12 2013 Andrew Niemantsverdriet <andrewniemants@gmail.com> - 0.7.5-1
- Initial package

10
pkg/salt-api.service Normal file
View File

@ -0,0 +1,10 @@
[Unit]
Description=The Salt API
After=syslog.target network.target
[Service]
Type=simple
ExecStart=/usr/bin/salt-api
[Install]
WantedBy=multi-user.target

10
pkg/salt-api.upstart Normal file
View File

@ -0,0 +1,10 @@
description "Salt API"
start on (net-device-up
and local-filesystems
and runlevel [2345])
stop on runlevel [!2345]
script
exec salt-api
end script

1
pkg/suse/salt-api Symbolic link
View File

@ -0,0 +1 @@
../rpm/salt-api

95
pkg/suse/salt-api.changes Normal file
View File

@ -0,0 +1,95 @@
-------------------------------------------------------------------
Tue Oct 29 22:38:07 UTC 2013 - aboe76@gmail.com
- Salt-api updated to 0.8.3
- this will likely be the last salt-api solo release,
project is merging into main Salt project.
- fixed proper logging
- better ssl options
- improved python rest_wsgi and cherrypy support
-------------------------------------------------------------------
Fri Oct 18 11:44:15 UTC 2013 - p.drouand@gmail.com
- Don't support sysvinit and systemd for the same system; add conditionnal
macros to use systemd only on systems which support it and sysvinit
on other systems
-------------------------------------------------------------------
Fri Aug 9 20:24:28 UTC 2013 - aboe76@gmail.com
- Updated salt-api init file:
Same file as the salt-api package for Rhel/Fedora
-------------------------------------------------------------------
Thu Jul 18 04:46:39 UTC 2013 - aboe76@gmail.com
- Update package to 0.8.2
- Backward incompatible needs salt 0.15.9 or greater
- Changes to rest_cherrypy:
- Fixed issue #87 which caused the Salt master's PID file to be overwritten.
- Fixed an inconsistency with the return format for the /minions convenience URL.
- Added a dedicated URL for serving an HTML app
- Added dedicated URL for serving static media
-------------------------------------------------------------------
Sun May 12 20:18:57 UTC 2013 - aboe76@gmail.com
- Updated package spec, for systemd unit files
according to how systemd files needs to be packaged
- fixed rpmlint about reload missing with init files
-------------------------------------------------------------------
Tue Apr 23 19:20:42 UTC 2013 - aboe76@gmail.com
- updated init file:
removed probe/reload/force-reload they are not supported
-------------------------------------------------------------------
Tue Apr 23 18:10:38 UTC 2013 - aboe76@gmail.com
- Update to salt-api 0.8.1
- Cherrypy module fixes:
* Fixes helpful error messages when loading the module if
dependencies are missing or incorrect.
* Fixes the /login view to return a 401 instead of a 500 when
authentication fails.
* This release also includes a new plain-WSGI (no deps) REST module. This
module requires an external webserver and careful deployment -- be sure
to read the docs in full before using it.
-------------------------------------------------------------------
Mon Apr 15 18:48:31 UTC 2013 - aboe76@gmail.com
- Updated recommends cherrypy instead of requirement
cherrypy only needed as wsgi server if user wants
-------------------------------------------------------------------
Sun Apr 14 14:52:34 UTC 2013 - aboe76@gmail.com
- Updated salt-api init file
-------------------------------------------------------------------
Tue Apr 9 18:56:15 UTC 2013 - aboe76@gmail.com
- Updated to 0.8.0
- New authentication login
- salt-api can now run on WSGI application
- added service file for > opensuse 12.1
- added init file for the rest
-------------------------------------------------------------------
Wed Jan 30 21:00:43 UTC 2013 - aboe76@gmail.com
- Updated spec file with Suse Copyright
-------------------------------------------------------------------
Sat Jan 26 09:19:19 UTC 2013 - aboe76@gmail.com
- updated spec file depencies fixed include python-cherrypy for salt-ui
-------------------------------------------------------------------
Tue Jan 22 20:28:52 UTC 2013 - aboe76@gmail.com
- initial upload 0.7.5

1
pkg/suse/salt-api.service Symbolic link
View File

@ -0,0 +1 @@
../salt-api.service

118
pkg/suse/salt-api.spec Normal file
View File

@ -0,0 +1,118 @@
#
# spec file for package salt-api
#
# Copyright (c) 2012 SUSE LINUX Products GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
# upon. The license for this file, and modifications and additions to the
# file, is the same license as for the pristine package itself (unless the
# license for the pristine package is not an Open Source License, in which
# case the license is the MIT License). An "Open Source License" is a
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.
# Please submit bugfixes or comments via http://bugs.opensuse.org/
#
Name: salt-api
Version: 0.8.3
Release: 0
License: Apache-2.0
Summary: The api for Salt a parallel remote execution system
Url: http://saltstack.org/
Group: System/Monitoring
Source0: http://pypi.python.org/packages/source/s/%{name}/%{name}-%{version}.tar.gz
Source1: salt-api
Source2: salt-api.service
BuildRoot: %{_tmppath}/%{name}-%{version}-build
%if 0%{?suse_version} && 0%{?suse_version} <= 1110
%{!?python_sitelib: %global python_sitelib %(python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")}
%else
BuildArch: noarch
%endif
BuildRequires: fdupes
BuildRequires: python-devel
BuildRequires: salt >= 0.15.9
BuildRequires: salt-master
Requires: salt
Requires: salt-master
Recommends: python-CherryPy
%if 0%{?suse_version} >= 1210
BuildRequires: systemd
%{?systemd_requires}
%else
Requires(pre): %insserv_prereq
Requires(pre): %fillup_prereq
%endif
%description
salt-api is a modular interface on top of Salt that can provide a variety of entry points into a running Salt system.
%prep
%setup -q
%build
python setup.py build
%install
python setup.py install --prefix=%{_prefix} --root=%{buildroot}
%fdupes %{buildroot}%{_prefix}
#
##missing directories
%if 0%{?suse_version} < 1210
mkdir -p %{buildroot}%{_sysconfdir}/init.d
mkdir -p %{buildroot}/%{_sbindir}
%endif
mkdir -p %{buildroot}%{_localstatedir}/log/salt
#
##init scripts
%if 0%{?suse_version} < 1210
install -Dpm 0755 %{SOURCE1} %{buildroot}%{_sysconfdir}/init.d/salt-api
ln -sf /etc/init.d/salt-api %{buildroot}%{_sbindir}/rcsalt-api
%else
install -Dpm 644 %{SOURCE2} %{buildroot}%_unitdir/salt-api.service
%endif
%preun
%if 0%{?_unitdir:1}
%service_del_preun salt-api.service
%else
%stop_on_removal
%endif
%post
%if 0%{?_unitdir:1}
%service_add_post salt-api.service
%else
%fillup_and_insserv
%endif
%postun
%if 0%{?_unitdir:1}
%service_del_postun salt-api.service
%else
%insserv_cleanup
%restart_on_update
%endif
%files
%defattr(-,root,root)
%doc LICENSE
%if 0%{?_unitdir:1}
%_unitdir
%else
%{_sysconfdir}/init.d/salt-api
%{_sbindir}/rcsalt-api
%endif
%{_mandir}/man1/salt-api.1.*
%{_mandir}/man7/salt-api.7.*
%{_bindir}/salt-api
%{python_sitelib}/*
%changelog

View File

@ -5,6 +5,7 @@ The management of salt command line utilities are stored in here
# Import python libs
from __future__ import print_function
import logging
import os
import sys
@ -27,6 +28,7 @@ from salt.exceptions import (
EauthAuthenticationError,
)
log = logging.getLogger(__name__)
class SaltCMD(parsers.SaltCMDOptionParser):
'''
@ -440,3 +442,47 @@ class SaltSSH(parsers.SaltSSHOptionParser):
ssh = salt.client.ssh.SSH(self.config)
ssh.run()
class SaltAPI(parsers.OptionParser, parsers.ConfigDirMixIn,
parsers.LogLevelMixIn, parsers.PidfileMixin, parsers.DaemonMixIn,
parsers.MergeConfigMixIn):
'''
The cli parser object used to fire up the salt api system.
'''
__metaclass__ = parsers.OptionParserMeta
VERSION = salt.version.__version__
# ConfigDirMixIn config filename attribute
_config_filename_ = 'master'
# LogLevelMixIn attributes
_default_logging_logfile_ = '/var/log/salt/api'
def setup_config(self):
return salt.config.api_config(self.get_config_file_path())
def run(self):
'''
Run the api
'''
self.parse_args()
try:
if self.config['verify_env']:
logfile = self.config['log_file']
if logfile is not None and not logfile.startswith('tcp://') \
and not logfile.startswith('udp://') \
and not logfile.startswith('file://'):
# Logfile is not using Syslog, verify
salt.utils.verify.verify_files(
[logfile], self.config['user']
)
except OSError as err:
log.error(err)
sys.exit(err.errno)
self.setup_logfile_logger()
client = salt.client.netapi.NetapiClient(self.config)
self.daemonize_if_required()
self.set_pidfile()
client.run()

27
salt/client/netapi.py Normal file
View File

@ -0,0 +1,27 @@
'''
The main entry point for salt-api
'''
# Import python libs
import logging
import multiprocessing
# Import salt-api libs
import salt.loader
logger = logging.getLogger(__name__)
class NetapiClient(object):
'''
'''
def __init__(self, opts):
self.opts = opts
def run(self):
'''
Load and start all available api modules
'''
netapi = salt.loader.netapi(self.opts)
for fun in netapi:
if fun.endswith('.start'):
logger.info("Starting '{0}' api module".format(fun))
multiprocessing.Process(target=netapi[fun]).start()

View File

@ -539,6 +539,13 @@ CLOUD_CONFIG_DEFAULTS = {
'log_granular_levels': {},
}
DEFAULT_API_OPTS = {
# ----- Salt master settings overridden by Salt-API --------------------->
'pidfile': '/var/run/salt-api.pid',
'logfile': '/var/log/salt/api',
# <---- Salt master settings overridden by Salt-API ----------------------
}
VM_CONFIG_DEFAULTS = {
'default_include': 'cloud.profiles.d/*.conf',
}
@ -2122,3 +2129,16 @@ def client_config(path, env_var='SALT_CLIENT_CONFIG', defaults=None):
# Return the client options
_validate_opts(opts)
return opts
def api_config(path):
'''
Read in the salt master config file and add additional configs that
need to be stubbed out for salt-api
'''
# Let's grab a copy of salt's master default opts
defaults = DEFAULT_MASTER_OPTS
# Let's override them with salt-api's required defaults
defaults.update(DEFAULT_API_OPTS)
return master_config(path, defaults=defaults)

View File

@ -451,6 +451,19 @@ def clouds(opts):
return functions
def netapi(opts):
'''
Return the network api functions
'''
load = salt.loader._create_loader(
opts,
'netapi',
'netapi',
base_path=os.path.join(SALT_BASE_PATH, 'netapi'),
)
return load.gen_functions()
def _generate_module(name):
if name in sys.modules:
return

102
salt/netapi/__init__.py Normal file
View File

@ -0,0 +1,102 @@
'''
Make api awesomeness
'''
# Import Python libs
import inspect
# Import Salt libs
import salt.log # pylint: disable=W0611
import salt.client
import salt.runner
import salt.wheel
import salt.utils
from salt.exceptions import SaltException, EauthAuthenticationError
class NetapiClient(object):
'''
Provide a uniform method of accessing the various client interfaces in Salt
in the form of low-data data structures. For example:
>>> client = NetapiClient(__opts__)
>>> lowstate = {'client': 'local', 'tgt': '*', 'fun': 'test.ping', 'arg': ''}
>>> client.run(lowstate)
'''
def __init__(self, opts):
self.opts = opts
def run(self, low):
'''
Execute the specified function in the specified client by passing the
lowstate
'''
if not 'client' in low:
raise SaltException('No client specified')
if not ('token' in low or 'eauth' in low):
raise EauthAuthenticationError(
'No authentication credentials given')
l_fun = getattr(self, low['client'])
f_call = salt.utils.format_call(l_fun, low)
ret = l_fun(*f_call.get('args', ()), **f_call.get('kwargs', {}))
return ret
def local_async(self, *args, **kwargs):
'''
Run :ref:`execution modules <all-salt.modules>` asyncronously
Wraps :py:meth:`salt.client.LocalClient.run_job`.
:return: job ID
'''
local = salt.client.get_local_client(self.opts['conf_file'])
return local.run_job(*args, **kwargs)
def local(self, *args, **kwargs):
'''
Run :ref:`execution modules <all-salt.modules>` syncronously
Wraps :py:meth:`salt.client.LocalClient.cmd`.
:return: Returns the result from the execution module
'''
local = salt.client.get_local_client(self.opts['conf_file'])
return local.cmd(*args, **kwargs)
def local_batch(self, *args, **kwargs):
'''
Run :ref:`execution modules <all-salt.modules>` against batches of minions
.. versionadded:: 0.8.4
Wraps :py:meth:`salt.client.LocalClient.cmd_batch`
:return: Returns the result from the exeuction module for each batch of
returns
'''
local = salt.client.get_local_client(self.opts['conf_file'])
return local.cmd_batch(*args, **kwargs)
def runner(self, fun, **kwargs):
'''
Run `runner modules <all-salt.runners>`
Wraps :py:meth:`salt.runner.RunnerClient.low`.
:return: Returns the result from the runner module
'''
runner = salt.runner.RunnerClient(self.opts)
return runner.low(fun, kwargs)
def wheel(self, fun, **kwargs):
'''
Run :ref:`wheel modules <all-salt.wheel>`
Wraps :py:meth:`salt.wheel.WheelClient.master_call`.
:return: Returns the result from the wheel module
'''
kwargs['fun'] = fun
wheel = salt.wheel.Wheel(self.opts)
return wheel.master_call(**kwargs)

View File

@ -0,0 +1,98 @@
'''
A script to start the CherryPy WSGI server
This is run by ``salt-api`` and started in a multiprocess.
'''
# pylint: disable=C0103
# Import Python libs
import logging
import os
import signal
import sys
# Import CherryPy without traceback so we can provide an intelligent log
# message in the __virtual__ function
try:
import cherrypy
cpy_error = None
except ImportError as exc:
cpy_error = exc
logger = logging.getLogger(__name__)
cpy_min = '3.2.2'
__virtualname__ = 'rest'
def __virtual__():
short_name = __name__.rsplit('.')[-1]
mod_opts = __opts__.get(short_name, {})
if mod_opts:
# User has a rest_cherrypy section in config; assume the user wants to
# run the module and increase logging severity to be helpful
# Everything looks good; return the module name
if not cpy_error and 'port' in mod_opts:
return True
# CherryPy wasn't imported; explain why
if cpy_error:
from distutils.version import LooseVersion as V
if 'cherrypy' in globals() and V(cherrypy.__version__) < V(cpy_min):
error_msg = ("Required version of CherryPy is {0} or "
"greater.".format(cpy_min))
else:
error_msg = cpy_error
logger.error("Not loading '%s'. Error loading CherryPy: %s",
__name__, error_msg)
# Missing port config
if not 'port' in mod_opts:
logger.error("Not loading '%s'. 'port' not specified in config",
__name__)
return False
def verify_certs(*args):
'''
Sanity checking for the specified SSL certificates
'''
msg = ("Could not find a certificate: {0}\n"
"If you want to quickly generate a self-signed certificate, "
"use the tls.create_self_signed_cert function in Salt")
for arg in args:
if not os.path.exists(arg):
raise Exception(msg.format(arg))
def start():
'''
Start the server loop
'''
from . import app
root, apiopts, conf = app.get_app(__opts__)
if not apiopts.get('disable_ssl', False):
if not 'ssl_crt' in apiopts or not 'ssl_key' in apiopts:
logger.error("Not starting '%s'. Options 'ssl_crt' and "
"'ssl_key' are required if SSL is not disabled.",
__name__)
return None
verify_certs(apiopts['ssl_crt'], apiopts['ssl_key'])
cherrypy.server.ssl_module = 'builtin'
cherrypy.server.ssl_certificate = apiopts['ssl_crt']
cherrypy.server.ssl_private_key = apiopts['ssl_key']
def signal_handler(*args):
cherrypy.engine.exit()
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
cherrypy.quickstart(root, apiopts.get('root_prefix', '/'), conf)

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,206 @@
import json
import logging
logger = logging.getLogger(__name__)
class SaltInfo:
'''
Class to handle processing and publishing of "real time" Salt upates.
'''
def __init__(self, handler):
'''
handler is expected to be the server side end of a websocket
connection.
'''
self.handler = handler
'''
These represent a "real time" view into Salt's jobs.
'''
self.jobs = {}
'''
This represents a "real time" view of minions connected to
Salt.
'''
self.minions = {}
def publish_minions(self):
'''
Publishes minions as a list of dicts.
'''
minions = []
for minion, minion_info in self.minions.iteritems():
curr_minion = {}
curr_minion.update(minion_info)
curr_minion.update({'id': minion})
minions.append(curr_minion)
ret = {'minions': minions}
self.handler.send(json.dumps(ret), False)
def publish(self, key, data):
'''
Publishes the data to the event stream.
'''
publish_data = {key: data}
self.handler.send(json.dumps(publish_data), False)
def process_minion_update(self, event_data):
'''
Associate grains data with a minion and publish minion update
'''
tag = event_data['tag']
event_info = event_data['data']
_, _, _, _, mid = tag.split('/')
if not self.minions.get(mid, None):
self.minions[mid] = {}
minion = self.minions[mid]
minion.update({'grains': event_info['return']})
self.publish_minions()
def process_ret_job_event(self, event_data):
'''
Process a /ret event returned by Salt for a particular minion.
These events contain the returned results from a particular execution.
'''
tag = event_data['tag']
event_info = event_data['data']
_, _, jid, _, mid = tag.split('/')
job = self.jobs.setdefault(jid, {})
minion = job.setdefault('minions', {}).setdefault(mid, {})
minion.update({'return': event_info['return']})
minion.update({'retcode': event_info['retcode']})
minion.update({'success': event_info['success']})
job_complete = all([minion['success'] for mid, minion
in job['minions'].iteritems()])
if job_complete:
job['state'] = 'complete'
self.publish('jobs', self.jobs)
def process_new_job_event(self, event_data):
'''
Creates a new job with properties from the event data
like jid, function, args, timestamp.
Also sets the initial state to started.
Minions that are participating in this job are also noted.
'''
job = None
tag = event_data['tag']
event_info = event_data['data']
minions = {}
for mid in event_info['minions']:
minions[mid] = {'success': False}
job = {
'jid': event_info['jid'],
'start_time': event_info['_stamp'],
'minions': minions, # is a dictionary keyed by mids
'fun': event_info['fun'],
'tgt': event_info['tgt'],
'tgt_type': event_info['tgt_type'],
'state': 'running',
}
self.jobs[event_info['jid']] = job
self.publish('jobs', self.jobs)
def process_key_event(self, event_data):
tag = event_data['tag']
event_info = event_data['data']
'''
Tag: salt/key
Data:
{'_stamp': '2014-05-20T22:45:04.345583',
'act': 'delete',
'id': 'compute.home',
'result': True}
'''
if event_info['act'] == 'delete':
self.minions.pop(event_info['id'], None)
elif event_info['act'] == 'accept':
self.minions.setdefault(event_info['id'], {})
self.publish_minions()
def process_presense_events(salt_data, token, opts):
'''
Check if any minions have connected or dropped.
Send a message to the client if they have.
'''
tag = event_data['tag']
event_info = event_data['data']
minions_detected = event_info['present']
curr_minions = self.minions.keys()
changed = False
# check if any connections were dropped
dropped_minions = set(curr_minions) - set(minions_detected)
for minion in dropped_minions:
changed = True
self.minions.pop(minion, None)
# check if any new connections were made
new_minions = set(minions_detected) - set(curr_minions)
tgt = ','.join(new_minions)
if tgt:
changed = True
client = salt.netapi.NetapiClient(opts)
client.run(
{
'fun': 'grains.items',
'tgt': tgt,
'expr_type': 'list',
'mode': 'client',
'client': 'local',
'async': 'local_async',
'token': token,
})
if changed:
self.publish_minions()
def process(self, salt_data, token, opts):
'''
Process events and publish data
'''
parts = salt_data['tag'].split('/')
if len(parts) < 2:
return
# TBD: Simplify these conditional expressions
if parts[1] == 'job':
if parts[3] == 'new':
self.process_new_job_event(salt_data)
if salt_data['data']['fun'] == 'grains.items':
self.minions = {}
elif parts[3] == 'ret':
self.process_ret_job_event(salt_data)
if salt_data['data']['fun'] == 'grains.items':
self.process_minion_update(salt_data)
if parts[1] == 'key':
self.process_key_event(salt_data)
if parts[1] == 'presense':
self.process_presense_events(salt_data, token, opts)

View File

@ -0,0 +1,3 @@
from .rate_limit import RateLimitTool
__all__ = ('RateLimitTool',)

View File

@ -0,0 +1,60 @@
import cherrypy
from ws4py.server.cherrypyserver import WebSocketPlugin, WebSocketTool
from ws4py.websocket import WebSocket
from multiprocessing import Lock, Pipe
cherrypy.tools.websocket = WebSocketTool()
WebSocketPlugin(cherrypy.engine).subscribe()
class SynchronizingWebsocket(WebSocket):
'''
Class to handle requests sent to this websocket connection.
Each instance of this class represents a Salt websocket connection.
Waits to receive a ``ready`` message fom the client.
Calls send on it's end of the pipe to signal to the sender on receipt
of ``ready``.
This class also kicks off initial information probing jobs when clients
initially connect. These jobs help gather information about minions, jobs,
and documentation.
'''
def __init__(self, *args, **kwargs):
super(SynchronizingWebsocket, self).__init__(*args, **kwargs)
'''
This pipe needs to represent the parent end of a pipe.
Clients need to ensure that the pipe assigned to ``self.pipe`` is
the ``parent end`` of a
`pipe <https://docs.python.org/2/library/multiprocessing.html#exchanging-objects-between-processes>`_.
'''
self.pipe = None
'''
The token that we can use to make API calls.
There are times when we would like to kick off jobs,
examples include trying to obtain minions connected.
'''
self.token = None
'''
Options represent ``salt`` options defined in the configs.
'''
self.opts = None
def received_message(self, message):
'''
Checks if the client has sent a ready message.
A ready message causes ``send()`` to be called on the
``parent end`` of the pipe.
Clients need to ensure that the pipe assigned to ``self.pipe`` is
the ``parent end`` of a pipe.
This ensures completion of the underlying websocket connection
and can be used to synchronize parallel senders.
'''
if message.data == 'websocket client ready':
self.pipe.send(message)
self.send('server received message', False)

View File

@ -0,0 +1,80 @@
#!/usr/bin/env python
'''
Deployment
==========
The ``rest_cherrypy`` netapi module is a standard Python WSGI app. It can be
deployed one of two ways.
:program:`salt-api` using the CherryPy server
---------------------------------------------
The default configuration is to run this module using :program:`salt-api` to
start the Python-based CherryPy server. This server is lightweight,
multi-threaded, encrypted with SSL, and should be considered production-ready.
Using a WSGI-compliant web server
---------------------------------
This module may be deplayed on any WSGI-compliant server such as Apache with
mod_wsgi or Nginx with FastCGI, to name just two (there are many).
Note, external WSGI servers handle URLs, paths, and SSL certs directly. The
``rest_cherrypy`` configuration options are ignored and the ``salt-api`` daemon
does not need to be running at all. Remember Salt authentication credentials
are sent in the clear unless SSL is being enforced!
An example Apache virtual host configuration::
<VirtualHost *:80>
ServerName example.com
ServerAlias *.example.com
ServerAdmin webmaster@example.com
LogLevel warn
ErrorLog /var/www/example.com/logs/error.log
CustomLog /var/www/example.com/logs/access.log combined
DocumentRoot /var/www/example.com/htdocs
WSGIScriptAlias / /path/to/salt/netapi/rest_cherrypy/wsgi.py
</VirtualHost>
'''
# pylint: disable=C0103
import os
import cherrypy
def bootstrap_app():
'''
Grab the opts dict of the master config by trying to import Salt
'''
from salt.netapi.rest_cherrypy import app
import salt.config
__opts__ = salt.config.client_config(
os.environ.get('SALT_MASTER_CONFIG', '/etc/salt/master'))
return app.get_app(__opts__)
def get_application(*args):
'''
Returns a WSGI application function. If you supply the WSGI app and config
it will use that, otherwise it will try to obtain them from a local Salt
installation
'''
opts_tuple = args
def wsgi_app(environ, start_response):
root, _, conf = opts_tuple or bootstrap_app()
cherrypy.config.update({'environment': 'embedded'})
cherrypy.tree.mount(root, '/', conf)
return cherrypy.tree(environ, start_response)
return wsgi_app
application = get_application()

View File

@ -0,0 +1,102 @@
import hashlib
import logging
__virtualname__ = 'rest_tornado'
logger = logging.getLogger(__virtualname__)
try:
import tornado.httpserver
import tornado.ioloop
import tornado.web
import tornado.gen
has_tornado = True
except ImportError as err:
has_tornado = False
logger.info('ImportError! {}'.format(str(err)))
import salt.auth
def __virtual__():
mod_opts = __opts__.get(__virtualname__, {})
if has_tornado and 'port' in mod_opts:
return __virtualname__
return False
def start():
'''
Start the saltnado!
'''
from . import saltnado
mod_opts = __opts__.get(__virtualname__, {})
if 'num_processes' not in mod_opts:
mod_opts['num_processes'] = 1
token_pattern = r"([0-9A-Fa-f]{%s})" % len(getattr(hashlib, __opts__.get('hash_type', 'md5'))().hexdigest())
all_events_pattern = r"/all_events/{}".format(token_pattern)
formatted_events_pattern = r"/formatted_events/{}".format(token_pattern)
logger.debug("All events URL pattern is {}".format(all_events_pattern))
application = tornado.web.Application([
(r"/", saltnado.SaltAPIHandler),
(r"/login", saltnado.SaltAuthHandler),
(r"/minions/(.*)", saltnado.MinionSaltAPIHandler),
(r"/minions", saltnado.MinionSaltAPIHandler),
(r"/jobs/(.*)", saltnado.JobsSaltAPIHandler),
(r"/jobs", saltnado.JobsSaltAPIHandler),
(r"/run", saltnado.RunSaltAPIHandler),
(r"/events", saltnado.EventsSaltAPIHandler),
(r"/hook(/.*)?", saltnado.WebhookSaltAPIHandler),
# Matches /all_events/[0-9A-Fa-f]{n}
# Where n is the length of hexdigest
# for the current hashing algorithm.
# This algorithm is specified in the
# salt master config file.
(all_events_pattern, saltnado.AllEventsHandler),
(formatted_events_pattern, saltnado.FormattedEventsHandler),
], debug=mod_opts.get('debug', False))
application.opts = __opts__
application.mod_opts = mod_opts
application.auth = salt.auth.LoadAuth(__opts__)
application.event_listener = saltnado.EventListener(mod_opts, __opts__)
# the kwargs for the HTTPServer
kwargs = {}
if not mod_opts.get('disable_ssl', False):
if 'ssl_crt' not in mod_opts:
logger.error("Not starting '%s'. Options 'ssl_crt' and "
"'ssl_key' are required if SSL is not disabled.",
__name__)
return None
# cert is required, key may be optional
# https://docs.python.org/2/library/ssl.html#ssl.wrap_socket
ssl_opts = {'certfile': mod_opts['ssl_crt']}
if mod_opts.get('ssl_key', False):
ssl_opts.update({'keyfile': mod_opts['ssl_key']})
kwargs['ssl_options'] = ssl_opts
http_server = tornado.httpserver.HTTPServer(application, **kwargs)
try:
http_server.bind(mod_opts['port'])
http_server.start(mod_opts['num_processes'])
except:
print 'Rest_tornado unable to bind to port {0}'.format(mod_opts['port'])
raise SystemExit(1)
tornado.ioloop.IOLoop.instance().add_callback(application.event_listener.iter_events)
try:
tornado.ioloop.IOLoop.instance().start()
except KeyboardInterrupt:
raise SystemExit(0)

View File

@ -0,0 +1,234 @@
import json
import logging
import salt.netapi
logger = logging.getLogger(__name__)
class SaltInfo:
'''
Class to handle processing and publishing of "real time" Salt upates.
'''
def __init__(self, handler):
'''
handler is expected to be the server side end of a websocket
connection.
'''
self.handler = handler
'''
These represent a "real time" view into Salt's jobs.
'''
self.jobs = {}
'''
This represents a "real time" view of minions connected to
Salt.
'''
self.minions = {}
def publish_minions(self):
'''
Publishes minions as a list of dicts.
'''
logger.debug('in publish minions')
minions = {}
logger.debug('starting loop')
for minion, minion_info in self.minions.iteritems():
logger.debug(minion)
# logger.debug(minion_info)
curr_minion = {}
curr_minion.update(minion_info)
curr_minion.update({'id': minion})
minions[minion] = curr_minion
logger.debug('ended loop')
ret = {'minions': minions}
self.handler.write_message(u'{}\n\n'.format(json.dumps(ret)))
def publish(self, key, data):
'''
Publishes the data to the event stream.
'''
publish_data = {key: data}
pub = u'{}\n\n'.format(json.dumps(publish_data))
self.handler.write_message(pub)
def process_minion_update(self, event_data):
'''
Associate grains data with a minion and publish minion update
'''
tag = event_data['tag']
event_info = event_data['data']
_, _, _, _, mid = tag.split('/')
if not self.minions.get(mid, None):
self.minions[mid] = {}
minion = self.minions[mid]
minion.update({'grains': event_info['return']})
logger.debug("In process minion grains update with minions={}".format(self.minions.keys()))
self.publish_minions()
def process_ret_job_event(self, event_data):
'''
Process a /ret event returned by Salt for a particular minion.
These events contain the returned results from a particular execution.
'''
tag = event_data['tag']
event_info = event_data['data']
_, _, jid, _, mid = tag.split('/')
job = self.jobs.setdefault(jid, {})
minion = job.setdefault('minions', {}).setdefault(mid, {})
minion.update({'return': event_info['return']})
minion.update({'retcode': event_info['retcode']})
minion.update({'success': event_info['success']})
job_complete = all([minion['success'] for mid, minion
in job['minions'].iteritems()])
if job_complete:
job['state'] = 'complete'
self.publish('jobs', self.jobs)
def process_new_job_event(self, event_data):
'''
Creates a new job with properties from the event data
like jid, function, args, timestamp.
Also sets the initial state to started.
Minions that are participating in this job are also noted.
'''
job = None
tag = event_data['tag']
event_info = event_data['data']
minions = {}
for mid in event_info['minions']:
minions[mid] = {'success': False}
job = {
'jid': event_info['jid'],
'start_time': event_info['_stamp'],
'minions': minions, # is a dictionary keyed by mids
'fun': event_info['fun'],
'tgt': event_info['tgt'],
'tgt_type': event_info['tgt_type'],
'state': 'running',
}
self.jobs[event_info['jid']] = job
self.publish('jobs', self.jobs)
def process_key_event(self, event_data):
tag = event_data['tag']
event_info = event_data['data']
'''
Tag: salt/key
Data:
{'_stamp': '2014-05-20T22:45:04.345583',
'act': 'delete',
'id': 'compute.home',
'result': True}
'''
if event_info['act'] == 'delete':
self.minions.pop(event_info['id'], None)
elif event_info['act'] == 'accept':
self.minions.setdefault(event_info['id'], {})
self.publish_minions()
def process_presence_events(self, salt_data, token, opts):
'''
Check if any minions have connected or dropped.
Send a message to the client if they have.
'''
logger.debug('In presence')
changed = False
# check if any connections were dropped
if set(salt_data['data'].get('lost', [])):
dropped_minions = set(salt_data['data'].get('lost', []))
else:
dropped_minions = set(self.minions.keys()) - set(salt_data['data'].get('present', []))
for minion in dropped_minions:
changed = True
logger.debug('Popping {}'.format(minion))
self.minions.pop(minion, None)
# check if any new connections were made
if set(salt_data['data'].get('new', [])):
logger.debug('got new minions')
new_minions = set(salt_data['data'].get('new', []))
changed = True
elif set(salt_data['data'].get('present', [])) - set(self.minions.keys()):
logger.debug('detected new minions')
new_minions = set(salt_data['data'].get('present', [])) - set(self.minions.keys())
changed = True
else:
new_minions = []
tgt = ','.join(new_minions)
for mid in new_minions:
logger.debug('Adding minion')
self.minions[mid] = {}
if tgt:
changed = True
client = salt.netapi.NetapiClient(opts)
client.run(
{
'fun': 'grains.items',
'tgt': tgt,
'expr_type': 'list',
'mode': 'client',
'client': 'local',
'async': 'local_async',
'token': token,
})
if changed:
self.publish_minions()
def process(self, salt_data, token, opts):
'''
Process events and publish data
'''
import threading
logger.debug('In process {}'.format(threading.current_thread()))
logger.debug(salt_data['tag'])
logger.debug(salt_data)
parts = salt_data['tag'].split('/')
if len(parts) < 2:
return
# TBD: Simplify these conditional expressions
if parts[1] == 'job':
logger.debug('In job part 1')
if parts[3] == 'new':
logger.debug('In new job')
self.process_new_job_event(salt_data)
# if salt_data['data']['fun'] == 'grains.items':
# self.minions = {}
elif parts[3] == 'ret':
logger.debug('In ret')
self.process_ret_job_event(salt_data)
if salt_data['data']['fun'] == 'grains.items':
self.process_minion_update(salt_data)
elif parts[1] == 'key':
logger.debug('In key')
self.process_key_event(salt_data)
elif parts[1] == 'presence':
self.process_presence_events(salt_data, token, opts)
# logger.debug('In presence')

File diff suppressed because it is too large Load Diff

317
salt/netapi/rest_wsgi.py Normal file
View File

@ -0,0 +1,317 @@
'''
A minimalist REST API for Salt
==============================
This ``rest_wsgi`` module provides a no-frills REST interface for sending
commands to the Salt master. There are no dependencies.
Extra care must be taken when deploying this module into production. Please
read this documentation in entirety.
All authentication is done through Salt's :ref:`external auth <acl-eauth>`
system.
Usage
=====
* All requests must be sent to the root URL (``/``).
* All requests must be sent as a POST request with JSON content in the request
body.
* All responses are in JSON.
.. seealso:: :py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>`
The :py:mod:`rest_cherrypy <salt.netapi.rest_cherrypy.app>` module is
more full-featured, production-ready, and has builtin security features.
Deployment
==========
The ``rest_wsgi`` netapi module is a standard Python WSGI app. It can be
deployed one of two ways.
Using a WSGI-compliant web server
---------------------------------
This module may be run via any WSGI-compliant production server such as Apache
with mod_wsgi or Nginx with FastCGI.
It is strongly recommended that this app be used with a server that supports
HTTPS encryption since raw Salt authentication credentials must be sent with
every request. Any apps that access Salt through this interface will need to
manually manage authentication credentials (either username and password or a
Salt token). Tread carefully.
:program:`salt-api` using a development-only server
---------------------------------------------------
If run directly via the salt-api daemon it uses the `wsgiref.simple_server()`__
that ships in the Python standard library. This is a single-threaded server
that is intended for testing and development. **This server does not use
encryption**; please note that raw Salt authentication credentials must be sent
with every HTTP request.
**Running this module via salt-api is not recommended!**
In order to start this module via the ``salt-api`` daemon the following must be
put into the Salt master config::
rest_wsgi:
port: 8001
.. __: http://docs.python.org/2/library/wsgiref.html#module-wsgiref.simple_server
Usage examples
==============
.. http:post:: /
**Example request** for a basic ``test.ping``::
% curl -sS -i \\
-H 'Content-Type: application/json' \\
-d '[{"eauth":"pam","username":"saltdev","password":"saltdev","client":"local","tgt":"*","fun":"test.ping"}]' localhost:8001
**Example response**:
.. code-block:: http
HTTP/1.0 200 OK
Content-Length: 89
Content-Type: application/json
{"return": [{"ms--4": true, "ms--3": true, "ms--2": true, "ms--1": true, "ms--0": true}]}
**Example request** for an asyncronous ``test.ping``::
% curl -sS -i \\
-H 'Content-Type: application/json' \\
-d '[{"eauth":"pam","username":"saltdev","password":"saltdev","client":"local_async","tgt":"*","fun":"test.ping"}]' localhost:8001
**Example response**:
.. code-block:: http
HTTP/1.0 200 OK
Content-Length: 103
Content-Type: application/json
{"return": [{"jid": "20130412192112593739", "minions": ["ms--4", "ms--3", "ms--2", "ms--1", "ms--0"]}]}
**Example request** for looking up a job ID::
% curl -sS -i \\
-H 'Content-Type: application/json' \\
-d '[{"eauth":"pam","username":"saltdev","password":"saltdev","client":"runner","fun":"jobs.lookup_jid","jid":"20130412192112593739"}]' localhost:8001
**Example response**:
.. code-block:: http
HTTP/1.0 200 OK
Content-Length: 89
Content-Type: application/json
{"return": [{"ms--4": true, "ms--3": true, "ms--2": true, "ms--1": true, "ms--0": true}]}
:form lowstate: A list of :term:`lowstate` data appropriate for the
:ref:`client <client-apis>` interface you are calling.
:status 200: success
:status 401: authentication required
'''
import errno
import json
import logging
import os
# Import salt libs
import salt
import salt.netapi
# HTTP response codes to response headers map
H = {
200: '200 OK',
400: '400 BAD REQUEST',
401: '401 UNAUTHORIZED',
404: '404 NOT FOUND',
405: '405 METHOD NOT ALLOWED',
406: '406 NOT ACCEPTABLE',
500: '500 INTERNAL SERVER ERROR',
}
__virtualname__ = 'rest_wsgi'
logger = logging.getLogger(__virtualname__)
def __virtual__():
mod_opts = __opts__.get(__virtualname__, {})
if 'port' in mod_opts:
return __virtualname__
return False
class HTTPError(Exception):
'''
A custom exception that can take action based on an HTTP error code
'''
def __init__(self, code, message):
self.code = code
Exception.__init__(self, '{0}: {1}'.format(code, message))
def mkdir_p(path):
'''
mkdir -p
http://stackoverflow.com/a/600612/127816
'''
try:
os.makedirs(path)
except OSError as exc: # Python >2.5
if exc.errno == errno.EEXIST and os.path.isdir(path):
pass
else: raise
def read_body(environ):
'''
Pull the body from the request and return it
'''
length = environ.get('CONTENT_LENGTH', '0')
length = 0 if length == '' else int(length)
return environ['wsgi.input'].read(length)
def get_json(environ):
'''
Return the request body as JSON
'''
content_type = environ.get('CONTENT_TYPE', '')
if content_type != 'application/json':
raise HTTPError(406, 'JSON required')
try:
return json.loads(read_body(environ))
except ValueError as exc:
raise HTTPError(400, exc)
def get_headers(data, extra_headers=None):
'''
Takes the response data as well as any additional headers and returns a
tuple of tuples of headers suitable for passing to start_response()
'''
response_headers = {
'Content-Length': str(len(data)),
}
if extra_headers:
response_headers.update(extra_headers)
return response_headers.items()
def run_chunk(environ, lowstate):
'''
Expects a list of lowstate dictionaries that are executed and returned in
order
'''
client = environ['SALT_APIClient']
for chunk in lowstate:
yield client.run(chunk)
def dispatch(environ):
'''
Do any path/method dispatching here and return a JSON-serializable data
structure appropriate for the response
'''
method = environ['REQUEST_METHOD'].upper()
if method == 'GET':
return ("They found me. I don't know how, but they found me. "
"Run for it, Marty!")
elif method == 'POST':
data = get_json(environ)
return run_chunk(environ, data)
else:
raise HTTPError(405, 'Method Not Allowed')
def saltenviron(environ):
'''
Make Salt's opts dict and the APIClient available in the WSGI environ
'''
if not '__opts__' in locals():
import salt.config
__opts__ = salt.config.client_config(
os.environ.get('SALT_MASTER_CONFIG', '/etc/salt/master'))
environ['SALT_OPTS'] = __opts__
environ['SALT_APIClient'] = salt.netapi.NetapiClient(__opts__)
def application(environ, start_response):
'''
Process the request and return a JSON response. Catch errors and return the
appropriate HTTP code.
'''
# Instantiate APIClient once for the whole app
saltenviron(environ)
# Call the dispatcher
try:
resp = list(dispatch(environ))
code = 200
except HTTPError as exc:
code = exc.code
resp = str(exc)
except salt.exceptions.EauthAuthenticationError as exc:
code = 401
resp = str(exc)
except Exception as exc:
code = 500
resp = str(exc)
# Convert the response to JSON
try:
ret = json.dumps({'return': resp})
except TypeError as exc:
code = 500
ret = str(exc)
# Return the response
start_response(H[code], get_headers(ret, {
'Content-Type': 'application/json',
}))
return (ret,)
def get_opts():
'''
Return the Salt master config as __opts__
'''
import salt.config
return salt.config.client_config(
os.environ.get('SALT_MASTER_CONFIG', '/etc/salt/master'))
def start():
'''
Start simple_server()
'''
from wsgiref.simple_server import make_server
# When started outside of salt-api __opts__ will not be injected
if not '__opts__' in globals():
globals()['__opts__'] = get_opts()
if __virtual__() == False:
raise SystemExit(1)
mod_opts = __opts__.get(__virtualname__, {})
# pylint: disable-msg=C0103
httpd = make_server('localhost', mod_opts['port'], application)
try:
httpd.serve_forever()
except KeyboardInterrupt:
raise SystemExit(0)
if __name__ == '__main__':
start()

View File

@ -15,6 +15,7 @@ import logging
import salt
import salt.exceptions
import salt.cli
import salt.cli.saltapi
try:
import salt.cloud.cli
HAS_SALTCLOUD = True
@ -229,6 +230,14 @@ def salt_cloud():
hardcrash, trace=trace)
def salt_api():
'''
The main function for salt-api
'''
sapi = salt.cli.saltapi.SaltAPI()
sapi.run()
def salt_main():
'''
Publish commands to the salt system from the command line on the

11
scripts/salt-api Executable file
View File

@ -0,0 +1,11 @@
#!/usr/bin/env python
# Import salt libs
import salt.cli
def main():
sapi = salt.cli.SaltAPI()
sapi.run()
if __name__ == '__main__':
main()

View File

View File

@ -0,0 +1,101 @@
# coding: utf-8
import cgi
import json
import urllib
import cherrypy
import yaml
from tests.utils import BaseRestCherryPyTest
class TestAuth(BaseRestCherryPyTest):
def test_get_root_noauth(self):
'''
GET requests to the root URL should not require auth
'''
request, response = self.request('/')
self.assertEqual(response.status, '200 OK')
def test_post_root_auth(self):
'''
POST requests to the root URL redirect to login
'''
self.assertRaisesRegexp(cherrypy.InternalRedirect, '\/login',
self.request, '/', method='POST', data={})
def test_login_noauth(self):
'''
GET requests to the login URL should not require auth
'''
request, response = self.request('/login')
self.assertEqual(response.status, '200 OK')
def test_webhook_auth(self):
'''
Requests to the webhook URL require auth by default
'''
self.assertRaisesRegexp(cherrypy.InternalRedirect, '\/login',
self.request, '/hook', method='POST', data={})
class TestLogin(BaseRestCherryPyTest):
auth_creds = (
('username', 'saltdev'),
('password', 'saltdev'),
('eauth', 'auto'))
def test_good_login(self):
'''
Test logging in
'''
# Mock mk_token for a positive return
self.Resolver.return_value.mk_token.return_value = {
'token': '6d1b722e',
'start': 1363805943.776223,
'expire': 1363849143.776224,
'name': 'saltdev',
'eauth': 'auto',
}
body = urllib.urlencode(self.auth_creds)
request, response = self.request('/login', method='POST', body=body,
headers={
'content-type': 'application/x-www-form-urlencoded'
})
self.assertEqual(response.status, '200 OK')
def test_bad_login(self):
'''
Test logging in
'''
# Mock mk_token for a negative return
self.Resolver.return_value.mk_token.return_value = {}
body = urllib.urlencode({'totally': 'invalid_creds'})
request, response = self.request('/login', method='POST', body=body,
headers={
'content-type': 'application/x-www-form-urlencoded'
})
self.assertEqual(response.status, '401 Unauthorized')
class TestWebhookDisableAuth(BaseRestCherryPyTest):
__opts__ = {
'rest_cherrypy': {
'port': 8000,
'debug': True,
'webhook_disable_auth': True,
},
}
def test_webhook_noauth(self):
'''
Auth can be disabled for requests to the webhook URL
'''
# Mock fire_event() since we're only testing auth here.
self.get_event.return_value.fire_event.return_value = True
body = urllib.urlencode({'foo': 'Foo!'})
request, response = self.request('/hook', method='POST', body=body,
headers={
'content-type': 'application/x-www-form-urlencoded'
})
self.assertEqual(response.status, '200 OK')

View File

View File

@ -0,0 +1,81 @@
# coding: utf-8
import json
import urllib
import cherrypy
import yaml
from salt.netapi.rest_cherrypy import app
from tests.utils import BaseRestCherryPyTest, BaseToolsTest
class TestOutFormats(BaseToolsTest):
_cp_config = {
'tools.hypermedia_out.on': True,
}
def test_default_accept(self):
request, response = self.request('/')
self.assertEqual(response.headers['Content-type'], 'application/json')
def test_unsupported_accept(self):
request, response = self.request('/', headers=(
('Accept', 'application/ms-word'),
))
self.assertEqual(response.status, '406 Not Acceptable')
def test_json_out(self):
request, response = self.request('/', headers=(
('Accept', 'application/json'),
))
self.assertEqual(response.headers['Content-type'], 'application/json')
def test_yaml_out(self):
request, response = self.request('/', headers=(
('Accept', 'application/x-yaml'),
))
self.assertEqual(response.headers['Content-type'], 'application/x-yaml')
class TestInFormats(BaseToolsTest):
_cp_config = {
'tools.hypermedia_in.on': True,
}
def test_urlencoded_ctype(self):
data = {'valid': 'stuff'}
request, response = self.request('/', method='POST',
body=urllib.urlencode(data), headers=(
('Content-type', 'application/x-www-form-urlencoded'),
))
self.assertEqual(response.status, '200 OK')
self.assertDictEqual(request.unserialized_data, data)
def test_json_ctype(self):
data = {'valid': 'stuff'}
request, response = self.request('/', method='POST',
body=json.dumps(data), headers=(
('Content-type', 'application/json'),
))
self.assertEqual(response.status, '200 OK')
self.assertDictEqual(request.unserialized_data, data)
def test_json_as_text_out(self):
'''
Some service send JSON as text/plain for compatibility purposes
'''
data = {'valid': 'stuff'}
request, response = self.request('/', method='POST',
body=json.dumps(data), headers=(
('Content-type', 'text/plain'),
))
self.assertEqual(response.status, '200 OK')
self.assertDictEqual(request.unserialized_data, data)
def test_yaml_ctype(self):
data = {'valid': 'stuff'}
request, response = self.request('/', method='POST',
body=yaml.dump(data), headers=(
('Content-type', 'application/x-yaml'),
))
self.assertEqual(response.status, '200 OK')
self.assertDictEqual(request.unserialized_data, data)

93
tests/utils/__init__.py Normal file
View File

@ -0,0 +1,93 @@
# coding: utf-8
import cherrypy
import mock
from salt.netapi.rest_cherrypy import app
from . cptestcase import BaseCherryPyTestCase
class BaseRestCherryPyTest(BaseCherryPyTestCase):
'''
A base TestCase subclass for the rest_cherrypy module
This mocks all interactions with Salt-core and sets up a dummy
(unsubscribed) CherryPy web server.
'''
__opts__ = None
@mock.patch('salt.netapi.NetapiClient', autospec=True)
@mock.patch('salt.auth.Resolver', autospec=True)
@mock.patch('salt.auth.LoadAuth', autospec=True)
@mock.patch('salt.utils.event.get_event', autospec=True)
def setUp(self, get_event, LoadAuth, Resolver, NetapiClient):
app.salt.netapi.NetapiClient = NetapiClient
app.salt.auth.Resolver = Resolver
app.salt.auth.LoadAuth = LoadAuth
app.salt.utils.event.get_event = get_event
# Make local references to mocked objects so individual tests can
# access and modify the mocked interfaces.
self.Resolver = Resolver
self.NetapiClient = NetapiClient
self.get_event = get_event
__opts__ = self.__opts__ or {
'external_auth': {
'auto': {
'saltdev': [
'@wheel',
'@runner',
'.*',
],
}
},
'rest_cherrypy': {
'port': 8000,
'debug': True,
},
}
root, apiopts, conf = app.get_app(__opts__)
cherrypy.tree.mount(root, '/', conf)
cherrypy.server.unsubscribe()
cherrypy.engine.start()
def tearDown(self):
cherrypy.engine.exit()
class Root(object):
'''
The simplest CherryPy app needed to test individual tools
'''
exposed = True
_cp_config = {}
def GET(self):
return {'return': ['Hello world.']}
def POST(self, *args, **kwargs):
return {'return': [{'args': args}, {'kwargs': kwargs}]}
class BaseToolsTest(BaseCherryPyTestCase):
'''
A base class so tests can selectively turn individual tools on for testing
'''
conf = {
'/': {
'request.dispatch': cherrypy.dispatch.MethodDispatcher(),
},
}
def setUp(self):
Root._cp_config = self._cp_config
root = Root()
cherrypy.tree.mount(root, '/', self.conf)
cherrypy.server.unsubscribe()
cherrypy.engine.start()
def tearDown(self):
cherrypy.engine.exit()

118
tests/utils/cptestcase.py Normal file
View File

@ -0,0 +1,118 @@
# -*- coding: utf-8 -*-
# Copyright (c) 2011-2012, Sylvain Hellegouarch
# All rights reserved.
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
# * Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# * Neither the name of Sylvain Hellegouarch nor the names of his contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# Modified from the original. See the Git history of this file for details.
# https://bitbucket.org/Lawouach/cherrypy-recipes/src/50aff88dc4e24206518ec32e1c32af043f2729da/testing/unit/serverless/cptestcase.py
from StringIO import StringIO
import unittest
import urllib
import cherrypy
# Not strictly speaking mandatory but just makes sense
cherrypy.config.update({'environment': "test_suite"})
# This is mandatory so that the HTTP server isn't started
# if you need to actually start (why would you?), simply
# subscribe it back.
cherrypy.server.unsubscribe()
# simulate fake socket address... they are irrelevant in our context
local = cherrypy.lib.httputil.Host('127.0.0.1', 50000, "")
remote = cherrypy.lib.httputil.Host('127.0.0.1', 50001, "")
__all__ = ['BaseCherryPyTestCase']
class BaseCherryPyTestCase(unittest.TestCase):
def request(self, path='/', method='GET', app_path='', scheme='http',
proto='HTTP/1.1', body=None, qs=None, headers=None, **kwargs):
"""
CherryPy does not have a facility for serverless unit testing.
However this recipe demonstrates a way of doing it by
calling its internal API to simulate an incoming request.
This will exercise the whole stack from there.
Remember a couple of things:
* CherryPy is multithreaded. The response you will get
from this method is a thread-data object attached to
the current thread. Unless you use many threads from
within a unit test, you can mostly forget
about the thread data aspect of the response.
* Responses are dispatched to a mounted application's
page handler, if found. This is the reason why you
must indicate which app you are targetting with
this request by specifying its mount point.
You can simulate various request settings by setting
the `headers` parameter to a dictionary of headers,
the request's `scheme` or `protocol`.
.. seealso: http://docs.cherrypy.org/stable/refman/_cprequest.html#cherrypy._cprequest.Response
"""
# This is a required header when running HTTP/1.1
h = {'Host': '127.0.0.1'}
# if we had some data passed as the request entity
# let's make sure we have the content-length set
fd = None
if body is not None:
h['content-length'] = '%d' % len(body)
fd = StringIO(body)
if headers is not None:
h.update(headers)
# Get our application and run the request against it
app = cherrypy.tree.apps.get(app_path)
if not app:
# XXX: perhaps not the best exception to raise?
raise AssertionError("No application mounted at '%s'" % app_path)
# Cleanup any previous returned response
# between calls to this method
app.release_serving()
# Let's fake the local and remote addresses
request, response = app.get_serving(local, remote, scheme, proto)
try:
h = [(k, v) for k, v in h.iteritems()]
response = request.run(method, path, qs, proto, h, fd)
finally:
if fd:
fd.close()
fd = None
if response.output_status.startswith('500'):
print response.body
raise AssertionError("Unexpected error")
# collapse the response into a bytestring
response.collapse_body()
return request, response