Proxy minion documentation

doc/_static/proxy_minions.xml

@@ -0,0 +1 @@
+<mxfile><diagram>7V1tc5u4Fv41mdn9YA2SkEAfk7RpZ3a709vs3Ltzv3iwrdhMMbiAk7S//kq8GSTAmMjG7Vx7mgQBQnrOc3SOzhHqDb7fvn6Ivd3mU7TiwQ2yVq83+N0NQpAiKn7Jku95iWtZecE69lfFRUXB3l/xpFGURlGQ+rtm4TIKQ75MG2VPUdCsbOeteV5gHQoel17Atcv+46/STdG4srWy/CP315vyMZCy/MzCW35dx9E+LJ53g/BD9slPJ+n38hEr/uTtg3SWFSF5Gr+/wfdxFKX5X9vXex5ItEog8qY+dJytmhzzMB10A3LyW569YM/L5iqtFLcIgMXBXbLxdrJwGUR7UcPdy8ZP+ePOW8rCFyHdojIep/y1s0mw6qigBI+2PI2/i0uKGwi0gWtbiDEoftmY5BUU3LCt4vjlIBJYCnBTFwcuCr2CGuvqSQc8xB8FJO3wEPc4OlLSu45OtkJRkNNbBHX6deIhutxEoOxYHQFqAZfVPq4OiG0CD3YcjySNo6/8PgqiOCvANPvU2LN9XcuBAMRCT8Cahzz2AgD3c3k8T3gsMBNX65BW0lAx1bFTsGEt/BAsezsg1NIA2fqhH4VCyAKHIOAx1hDKhgYua4DdOjSy4yVpLGAT6/Cxm1qEWijUpkTIAGco1CCaXIcQdgCt4QMn0yiKptQo2oXwhBqFj2qUjtDJGnVCx69QowaMwufWKJs1NcoZp0LEAByOPgabUqGllxKea9FWUCyd77x0uRE/Qx7MkT3fRXGatGsW60B6Os1y9IF4td8uVvzZX0rnU/i1+NUTvfkt5OlLFH+VEL74osO/a3iKPom23XmBvw6lPyj6nQ0xsru+cKFvixNbf7WS97wNoeIsPl0FrRYVNOEYOgOG7UurIMQQWKJU1bcpVFIfxKdXyUpmV6SSdo9K4our5AkIXZ9K0iuYq6kqyQCua5vjTKaRVGfaFRjJK5zQ6fGQg0bCyxvJk2d+V6SRZEBs6dIzP0JHqaCR4Ak5mwoOCZ44HZBOp2ulLHqmelBD6PTgyfCOX99UzxnAmXOrEMTAcdEBkDJkXLqdFDh17SF6gJZhgHENUbuFUdQGwh3SlPBN/BoQeRqpcLmizQVqd3GUeqkgrbhUWH98F3gLHtxVKYBa1Vb2FZc8CX7XyuvJgYY7RjpE2C0qCwj5HJBukYUjZdGCP8EAEQOY635G4gXpbOsl0vip8A/Q3xNQKBhJEUA1ECw0jpDAMYCHe5pXOlhbT8mo2LRpA4ViHIt+OhYgLVZQ+LMurKaYDjMB0BCrWKiiv81SdpUD9adUtc9R4hf6t4jSNNq2eFhptBOl2e23yS7PDErz4ZUHT/6rtCl3xRPebdJU3HEre4EeUgzWiVTyJViK6tFDdpXo88M3WfdCPOH29q8V+7D88q//3j0/zr4t//mC2R+ULN/Btb+Bfmz9eM/+Smb/zGj47y+3r5+cPx6//fnxIX4PEf7w96Obdoq/LuacTD2unyZIGwECoc1sRJDrYNRiy5EL7Pq4Cw2MvK5u2Q9etBC49ZuPlvNANiLZRC+66zzIyo9Eq9ACDAU0bmVyXHuoFkAHuLQWZyEmtGCAq3yU9bktK3PVqCqpmyDLtR6klXryg6BW/pR9WqyeHyUO8MUYlIBMXIt9sKj8y7fRlbGmm6XbKoRAiy8uZtWOAcRPC6JfZGCGDALHIdcxMDsDLNf/B+Y6mX6GgbkvB2BfamDuQOsqB+YBkeyfb2Dup+u0AzO7woEZo8PsqM5D12pFqg4KMxAFgPgiswjWQYtJenxa7HAsC3CHizJJly9D/BzZq+iyo08TTh9c212K5pALWwZQHgioYnkniL2VHwGPx74XzGdzL0x5GHpZdKcNwJMT1+po0jKgnil+WEppaoS9hbjcW6Zgu3+drbj4OdRWFXD38pW2oGdEJa0B1t+ESh6xx+3aWE4aIGCwdtZFGrtYCz6UAhMOJRtgmS5BsBfvmT9F8TYBiR/yuTwczLBcyr0MQw6AzCXCX4cWZa7F0Hn0tcrF/cx49pKZUMnm5nhoEwBpXzwSyxSERZiLqWu5tBV9G7j1ZLsJcsOyKb+uNBxdGuRqpYF+dWm4ujTo1UrDPi4Nvlrzx+KQB4vo5f2h4C4rECc2Uez/iMLUE4+6W0tfjDfSY9at/LZNmm+zb+2ud34spJRLnHtJmjdItqJfIqLR0T5ellfBomupF695mUYmR41E60KKqjDmgZf6z82WtEmgeMbnyBdtrGVhFSfSUsxN3oPiroMctYqQUhFSK8p7rVWUEaLq+ECODEjunM6R4zzQ2WOOCPmK/QmJQBQiqEt1BhNBqUhb82OSCANmej8bEciAcMlFiaAuUhpNBLUik0Qwkegaa8P9KAH+500USkYt1iXZFusPMeehHnz1so8Z205x72uNLSkCpy3EbWItKLRMTNrOHNTe8iTJkzlG0LcpoJBU6NMm+kxHn8D2RWCNRZKOkTUi0BoQ+StTbGUizN+uRb2BvxA/l4G/m3txKlNi0k+Nttt9KISXiQw9PAot8YJ5Gr3weB6Fc4jcV/EP7MK1AWiJQAH3EFv3WN2WQdE1wWs4YPKsUm7pJ8sIbKMV3yZzL1zNd3J8SED8NM8KdfZawtFmThvfD7xWdOPtKNtlCqDEleiULSM/jcwwMoHrgGnwqdZ7lB0uM3oNf8ye1gxj9TUcOtIMqy8PqPUYtMJwwEx6MoE6HctRLyZQlwFmaVPuKtzKAK6dtZSU52BpO6xRr4OAU69WsSsmhT8gqD2d8I/HYs8qfBtZPcJ3cfOFHiWOMlT2BCqee8cS6nMI3x5gIkeuic5eg+HJHLYscb6X3/7V0dnHgKmcIWU6gxhw68uBWxL57a85iBupCfM5IArixbHQqXFqBHU1KuQ8nR7NlPdunJFGUUmdzs4Yo0DX4uWUe0LUBep2cP5iRlHdQQOO9XKUiqBakUmJXoub0ypROrGlg32WThhnYNXfERhp6mxH4Y0tX86oLWs8n/Cvxc1pEz6b2Mftd3MQbQpfmUMOlr2ryJ5cTPZnyDuMs81Ml30x4k2q+Q25UDX4MtaPJWpFJkU6IEa23MfPxdrkwdK6qhyfqjBjA/qKjZ1pm+F1COY2jr3vtct28oKku8FV7rDZ4M52qZ75kctZ3+Xij7y5oylVgnJWSk3st+nZ3nGUmkF4nZyCqoNx5PLe2g1waoDL+fNzigDIIMPEdrHrVOtySo/edQGq5U7KYOOplHMJwAxim7lE7pRjNXM2AgZgWdq7UsbJOKar3Vw1Wttb2maA6QP867cynU7MdOhehOlY3XFC2XAC2b2PuUquS/00WBtr1nZZptu//piuRaTH+gnMBgRZDrbE5ICQaj+I/sWlhpk861gZ0+3cKF5Bubiuu5dKEIuZdiMGTGDfTLmp1y8Zohy0+2IZNkXAQbVYxrDcyqmU0xjEjvih6nLOI9djFS3jjOvbsoHWNz6zpeW3Pn38kXFFkIAnye8aO028KHxkTY+0mvqOQYOXsbdu2mRi3zNYTr9bsXTqWCbbRG7a5XvBFAjaYrAWPa7Wjjc3diIt76mdETM9+5JvHJTtCKaBcw44BJ9offeFcj1NtT6WAELJ4Q3zls2VGCA1RrK2FXwuYNStHoPQ6a9Mi8PD/xCR6/rhP9bA7/8H</diagram></mxfile>

doc/contents.rst

@@ -34,6 +34,7 @@ Salt Table of Contents
     topics/development/index
     topics/translating
     topics/salt_projects
+    topics/proxyminion/index
 
     ref/configuration/logging/*
     ref/configuration/logging/handlers/*

doc/index.rst

@@ -219,6 +219,9 @@ Salt is many splendid things.
 :doc:`Testing Salt <topics/tests/index>`
     A howto for writing unit tests and integration tests.
 
+:doc:`Salt Proxy Minions <topics/proxyminion/index>`
+    Controlling devices and machines unable to run a salt-minion.
+
 :ref:`Python API interface <python-api>`
     Use Salt programmatically from scripts and programs easily and
     simply via ``import salt``.

doc/topics/proxyminion/index.rst

@@ -1,48 +1,240 @@
+===============================
 Salt Proxy Minion Documentation
 ===============================
 
-.. toctree::
-    :maxdepth: 3
-    :glob:
-    :hidden:
-
-    *
-    install/index
-
 Proxy minions are a Salt feature that enables controlling devices that, for
 whatever reason, cannot run a standard salt-minion.  Examples include network
 gear that has an API but runs a proprietary OS, devices with limited CPU or
 memory, or devices that could run a minion, but for security reasons, will not.
 
-Proxy minions are not, for the most part, an "out of the box" feature.  Because
-there are a myriad of devices outside the typical Linux or Windows worlds, if
-you are trying to control something you will most likely have to write the
+*Proxy minions are not an "out of the box" feature*.  Because
+there are an infinite number of controllable devices,
+you will most likely have to write the
 interface yourself. Fortunately, this is only as difficult as the actual
 interface to the proxied device.  Devices that have an existing Python module
 (PyUSB for example) would be relatively simple to interface.  Code to control
 a device that has an HTML REST-based interface should be easy.  Code to control
 your typical housecat would be excellent source material for a PhD thesis.
 
-Salt-proxy-minions provide the 'plumbing' that allows device enumeration,
-control, status, remote execution, and state management.
+Salt proxy-minions provide the 'plumbing' that allows device enumeration
+and discovery, control, status, remote execution, and state management.
 
 Getting Started
-===============
+---------------
 
-Minion management
------------------
+The following diagram may be helpful in understanding the structure of a Salt
+installation that includes proxy-minions:
 
-Minion Discovery
-----------------
+.. image:: /_static/proxy_minions.png
 
-Essential Components of a Proxy Minion Interface
-------------------------------------------------
+The key thing to remember is the left-most section of the diagram.  Salt's
+nature is to have a minion connect to a master, then the master may control
+the minion.  However, for proxy minions, the target device cannot run a minion,
+and thus must rely on a separate minion to fire up the proxy-minion and make the
+initial and persistent connection.
 
-* Configuration file parameters
-
-* The __proxyenabled__ directive
-
-* test.ping
+After the proxy minion is started and initiates its connection to the 'dumb'
+device, it connects back to the salt-master and ceases to be affiliated in
+any way with the minion that started it.
 
 
+Configuration parameters on the master
+######################################
+
+In ``/etc/salt/master``, add the following key:
+
+.. code-block:: yaml
+
+    enumerate_proxy_minions: True
+
+This will invoke the salt-master routines to look for other configuration parameters
+to drive the proxies.
+
+Salt's Pillar system is ideally suited for configuring proxy-minions.  Proxies can
+either be designated via a pillar file in pillar_roots, or through an external pillar.
+External pillars afford the opportunity for interfacing with a configuration management
+system, database, or other knowledgeable system that that may already contain all the details
+of proxy targets.  To use static files in pillar_roots, pattern your files after the following
+examples, which are based on the diagram above:
+
+``/srv/salt/pillar/top.sls``
+
+.. code-block:: yaml
+
+    base:
+      minioncontroller1:
+        - networkswitches
+      minioncontroller2:
+        - reallydumbdevices
+      minioncontroller3:
+        - smsgateway
+
+
+``/srv/salt/pillar/networkswitches.sls``
+
+.. code-block:: yaml
+
+    proxy:
+      dumbdevice1:
+        proxytype: networkswitch
+        host: 172.23.23.5
+        username: root
+        passwd: letmein
+      dumbdevice2:
+        proxytype: networkswitch
+        host: 172.23.23.6
+        username: root
+        passwd: letmein
+      dumbdevice3:
+        proxytype: networkswitch
+        host: 172.23.23.7
+        username: root
+        passwd: letmein
+
+``/srv/salt/pillar/reallydumbdevices.sls``
+
+.. code-block:: yaml
+
+    proxy:
+      dumbdevice4:
+        proxytype: i2c_lightshow
+        i2c_address: 1
+      dumbdevice5:
+        proxytype: i2c_lightshow
+        i2c_address: 2
+      dumbdevice6:
+        proxytype: 433mhz_wireless
+
+``/srv/salt/pillar/smsgateway.sls``
+
+.. code-block:: yaml
+
+    proxy:
+      minioncontroller3:
+        dumbdevice7:
+          proxytype: sms_serial
+          deventry: /dev/tty04
+
+Note the contents of each minioncontroller key may differ widely based on
+the type of device that the proxy-minion is managing.
+
+In the above example
+
+- dumbdevices 1, 2, and 3 are network switches that have a management
+  interface available at a particular IP address.
+
+- dumbdevices 4 and 5 are very low-level devices controlled over an i2c bus.  In this case
+  the devices are physically connected to machine 'minioncontroller2', and are addressable
+  on the i2c bus at their respective i2c addresses.
+
+- dumbdevice6 is a 433 MHz wireless transmitter, also physically connected to minioncontroller2
+
+- dumbdevice7 is an SMS gateway connected to machine minioncontroller3 via a serial port.
+
+Also, in general, proxy-minions are lightweight, so the machines that run them could
+conceivably control a large number of devices.  The example above is just to illustrate
+that it is possible for the proxy services to be spread across many machines if necessary, or
+intentionally run on machines that need to control devices because of some physical interface
+(e.g. i2c and serial above).  Another reason to divide proxy services might be security.  In
+more secure environments only certain machines may have a network path to certain devices.
+
+Now our salt-minions know if they are supposed to spawn a proxy-minion process to control
+a particular device.  That proxy-minion process will initiate a connection back to the master
+to enable control.
+
+Proxytypes
+##########
+
+A proxytype is a Python file that encapsulates all the code necessary to interface with
+a device.  Proxytypes are located inside the salt.proxy module.
+At a minimum a proxytype must implement the following functions:
+
+``proxytype()``: Returns a string with the name of the proxy type.
+
+``proxyconn(*args, **kwargs)``: Provides the primary way to connect and communicate
+with the device. Some proxyconns instantiate a particular object that opens a
+network connection to a device and leaves the connection open for communication.
+Others simply abstract a serial connection or even implement endpoints to communicate
+via REST over HTTP.
+
+``id(opts)``: Returns a unique, unchanging id for the controlled device.  This is
+the "name" of the device, and is used by the salt-master for targeting and key
+authentication.
+
+It is highly recommended that the ``test.ping`` execution module also be defined
+for a proxytype. The code for ``ping`` should contact the controlled device and make
+sure it is really available.
+
+Here is an example proxytype used to interface to Juniper Networks devices that run
+the Junos operating system.  Note the additional library requirements--most of the
+"hard part" of talking to these devices is handled by the jnpr.junos, jnpr.junos.utils
+and jnpr.junos.cfg modules.
+
+.. code-block:: python
+
+    # Import python libs
+    import logging
+    import os
+
+    import jnpr.junos
+    import jnpr.junos.utils
+    import jnpr.junos.cfg
+    HAS_JUNOS = True
+
+    def proxyconn(user=None, host=None, passwd=None):
+        jdev = jnpr.junos.Device(user=user, host=host, password=passwd)
+        jdev.open()
+        jdev.bind(cu=jnpr.junos.utils.Config)
+        return jdev
+
+    def proxytype():
+        return 'junos'
+
+    def id(opts):
+        return opts['proxyconn'].facts['hostname']
+
+
+The __proxyenabled__ directive
+##############################
+
+Salt states and execution modules, by and large, cannot "automatically" work with
+proxied devices.  Execution modules like ``pkg`` or ``sqlite3`` have no meaning on
+a network switch or a housecat.  For a state/execution module to be available to
+a proxy-minion, the ``__proxyenabled__`` variable must be defined in the module as an
+array containing the names of all the proxytypes that this module can support.  The
+array can contain the special value ``*`` to indicate that the module supports all
+proxies.
+
+If no ``__proxyenabled__`` variable is defined, then by default, the state/execution
+module is unavailable to any proxy.
+
+Here is an excerpt from a module that was modified to support proxy-minions:
+
+.. code-block:: python
+
+    def ping():
+
+        if 'proxytype' in __opts__:
+            fun = 'salt.proxy.{0}.ping'.format(__opts__['proxytype'])
+            if fun in __salt__:
+                return __salt__[fun]()
+            else:
+                return False
+        else:
+            return True
+
+And then in salt.proxy.junos we find
+
+.. code-block:: python
+
+    def ping():
+
+        if 'proxytype' in __opts__:
+            fun = 'salt.proxy.{0}.ping'.format(__opts__['proxytype'])
+            if fun in __salt__:
+                return __salt__[fun]()
+            else:
+                return False
+        else:
+            return True