mirror of
https://github.com/valitydev/salt.git
synced 2024-11-06 08:35:21 +00:00
Improve pillar documentation (#38407)
* rename anchor * Move pillar cache config option to different section, add an anchor * Add links to master and minion config items * Add documentation for pillar.item's "delimiter" argument * Improve pillar documentation This improves RST formatting, adding hyperlinks as well as adding information on the difference between in-minion pillar data and pillar data that is compiled on-demand. * Add section on how pillar environments are handled
This commit is contained in:
Erik Johnson
Nicole Thomas
parent
423b1fddff
commit
27253522c8
@ -2252,7 +2252,7 @@ exposed.
|
||||
- 'mail\d+.mydomain.tld'
|
||||
|
||||
|
||||
.. _pillar-configuration:
|
||||
.. _pillar-configuration-master:
|
||||
|
||||
Pillar Configuration
|
||||
====================
|
||||
|
||||
@ -448,26 +448,6 @@ executed. By default this feature is disabled, to enable set cache_jobs to
|
||||
|
||||
cache_jobs: False
|
||||
|
||||
.. conf_minion:: minion_pillar_cache
|
||||
|
||||
``minion_pillar_cache``
|
||||
-----------------------
|
||||
|
||||
Default: ``False``
|
||||
|
||||
The minion can locally cache rendered pillar data under
|
||||
:conf_minion:`cachedir`/pillar. This allows a temporarily disconnected minion
|
||||
to access previously cached pillar data by invoking salt-call with the --local
|
||||
and --pillar_root=:conf_minion:`cachedir`/pillar options. Before enabling this
|
||||
setting consider that the rendered pillar may contain security sensitive data.
|
||||
Appropriate access restrictions should be in place. By default the saved pillar
|
||||
data will be readable only by the user account running salt. By default this
|
||||
feature is disabled, to enable set minion_pillar_cache to ``True``.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
minion_pillar_cache: False
|
||||
|
||||
.. conf_minion:: grains
|
||||
|
||||
``grains``
|
||||
@ -1442,8 +1422,10 @@ sha512 are also supported.
|
||||
hash_type: md5
|
||||
|
||||
|
||||
Pillar Settings
|
||||
===============
|
||||
.. _pillar-configuration-minion:
|
||||
|
||||
Pillar Configuration
|
||||
====================
|
||||
|
||||
.. conf_minion:: pillar_roots
|
||||
|
||||
@ -1484,6 +1466,28 @@ the environment setting, but for pillar instead of states.
|
||||
|
||||
pillarenv: None
|
||||
|
||||
.. conf_minion:: minion_pillar_cache
|
||||
|
||||
``minion_pillar_cache``
|
||||
-----------------------
|
||||
|
||||
.. versionadded:: 2016.3.0
|
||||
|
||||
Default: ``False``
|
||||
|
||||
The minion can locally cache rendered pillar data under
|
||||
:conf_minion:`cachedir`/pillar. This allows a temporarily disconnected minion
|
||||
to access previously cached pillar data by invoking salt-call with the --local
|
||||
and --pillar_root=:conf_minion:`cachedir`/pillar options. Before enabling this
|
||||
setting consider that the rendered pillar may contain security sensitive data.
|
||||
Appropriate access restrictions should be in place. By default the saved pillar
|
||||
data will be readable only by the user account running salt. By default this
|
||||
feature is disabled, to enable set minion_pillar_cache to ``True``.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
minion_pillar_cache: False
|
||||
|
||||
.. conf_minion:: file_recv_max_size
|
||||
|
||||
``file_recv_max_size``
|
||||
|
||||
@ -8,8 +8,14 @@ Pillars
|
||||
Salt includes a number of built-in external pillars, listed at
|
||||
:ref:`all-salt.pillars`.
|
||||
|
||||
You may also wish to look at the standard pillar documentation, at
|
||||
:ref:`pillar-configuration`
|
||||
The below links contain documentation for the configuration options
|
||||
|
||||
- :ref:`master-side configuration <pillar-configuration-master>`
|
||||
- :ref:`minion-side configuration <pillar-configuration-minion>`
|
||||
|
||||
Note that some of same the configuration options from the master are present in
|
||||
the minion configuration file, these are used in :ref:`masterless
|
||||
<tutorial-standalone-minion>` mode.
|
||||
|
||||
The source for the built-in Salt pillars can be found here:
|
||||
:blob:`salt/pillar`
|
||||
:blob:`salt/pillar`
|
||||
|
||||
@ -12,24 +12,25 @@ Pillar was added to Salt in version 0.9.8
|
||||
|
||||
.. note:: Storing sensitive data
|
||||
|
||||
Unlike state tree, pillar data is only available for the targeted
|
||||
minion specified by the matcher type. This makes it useful for
|
||||
storing sensitive data specific to a particular minion.
|
||||
Pillar data is compiled on the master. Additionally, pillar data for a
|
||||
given minion is only accessible by the minion for which it is targeted in
|
||||
the pillar configuration. This makes pillar useful for storing sensitive
|
||||
data specific to a particular minion.
|
||||
|
||||
|
||||
Declaring the Master Pillar
|
||||
===========================
|
||||
|
||||
The Salt Master server maintains a pillar_roots setup that matches the
|
||||
structure of the file_roots used in the Salt file server. Like the
|
||||
Salt file server the ``pillar_roots`` option in the master config is based
|
||||
on environments mapping to directories. The pillar data is then mapped to
|
||||
minions based on matchers in a top file which is laid out in the same way
|
||||
as the state top file. Salt pillars can use the same matcher types as the
|
||||
standard top file.
|
||||
The Salt Master server maintains a :conf_master:`pillar_roots` setup that
|
||||
matches the structure of the :conf_master:`file_roots` used in the Salt file
|
||||
server. Like :conf_master:`file_roots`, the :conf_master:`pillar_roots` option
|
||||
mapps environments to directories. The pillar data is then mapped to minions
|
||||
based on matchers in a top file which is laid out in the same way as the state
|
||||
top file. Salt pillars can use the same matcher types as the standard :ref:`top
|
||||
file <states-top>`.
|
||||
|
||||
The configuration for the :conf_master:`pillar_roots` in the master config file
|
||||
is identical in behavior and function as :conf_master:`file_roots`:
|
||||
conf_master:`pillar_roots` is configured just like :conf_master:`file_roots`.
|
||||
For example:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -149,10 +150,15 @@ And the actual pillar file at '/srv/pillar/common_pillar.sls':
|
||||
foo: bar
|
||||
boo: baz
|
||||
|
||||
Pillar namespace flattened
|
||||
==========================
|
||||
Pillar Namespace Flattening
|
||||
===========================
|
||||
|
||||
The separate pillar files all share the same namespace. Given a ``top.sls`` of:
|
||||
The separate pillar SLS files all merge down into a single dictionary of
|
||||
key-value pairs. When the same key is defined in multiple SLS files, this can
|
||||
result in unexpected behavior if care is not taken to how the pillar SLS files
|
||||
are laid out.
|
||||
|
||||
For example, given a ``top.sls`` containing the following:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -161,44 +167,49 @@ The separate pillar files all share the same namespace. Given a ``top.sls`` of:
|
||||
- packages
|
||||
- services
|
||||
|
||||
a ``packages.sls`` file of:
|
||||
with ``packages.sls`` containing:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
bind: bind9
|
||||
|
||||
and a ``services.sls`` file of:
|
||||
and ``services.sls`` containing:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
bind: named
|
||||
|
||||
Then a request for the ``bind`` pillar will only return ``named``; the
|
||||
``bind9`` value is not available. It is better to structure your pillar files
|
||||
with more hierarchy. For example your ``package.sls`` file could look like:
|
||||
Then a request for the ``bind`` pillar key will only return ``named``. The
|
||||
``bind9`` value will be lost, because ``services.sls`` was evaluated later.
|
||||
|
||||
.. note::
|
||||
Pillar files are applied in the order they are listed in the top file.
|
||||
Therefore conflicting keys will be overwritten in a 'last one wins' manner!
|
||||
For example, in the above scenario conflicting key values in ``services``
|
||||
will overwrite those in ``packages`` because it's at the bottom of the list.
|
||||
|
||||
It can be better to structure your pillar files with more hierarchy. For
|
||||
example the ``package.sls`` file could be configured like so:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
packages:
|
||||
bind: bind9
|
||||
|
||||
Pillar Namespace Merges
|
||||
=======================
|
||||
This would make the ``packages`` pillar key a nested dictionary containing a
|
||||
``bind`` key.
|
||||
|
||||
With some care, the pillar namespace can merge content from multiple pillar
|
||||
files under a single key, so long as conflicts are avoided as described above.
|
||||
Pillar Dictionary Merging
|
||||
=========================
|
||||
|
||||
For example, if the above example were modified as follows, the values are
|
||||
merged below a single key:
|
||||
If the same pillar key is defined in multiple pillar SLS files, and the keys in
|
||||
both files refer to nested dictionaries, then the content from these
|
||||
dictionaries will be recursively merged.
|
||||
|
||||
.. code-block:: yaml
|
||||
For example, keeping the ``top.sls`` the same, assume the following
|
||||
modifications to the pillar SLS files:
|
||||
|
||||
base:
|
||||
'*':
|
||||
- packages
|
||||
- services
|
||||
|
||||
And a ``packages.sls`` file like:
|
||||
``packages.sls``:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -206,7 +217,7 @@ And a ``packages.sls`` file like:
|
||||
package-name: bind9
|
||||
version: 9.9.5
|
||||
|
||||
And a ``services.sls`` file like:
|
||||
``services.sls``:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
@ -214,7 +225,7 @@ And a ``services.sls`` file like:
|
||||
port: 53
|
||||
listen-on: any
|
||||
|
||||
The resulting pillar will be as follows:
|
||||
The resulting pillar dictionary will be:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@ -230,11 +241,9 @@ The resulting pillar will be as follows:
|
||||
version:
|
||||
9.9.5
|
||||
|
||||
.. note::
|
||||
Pillar files are applied in the order they are listed in the top file.
|
||||
Therefore conflicting keys will be overwritten in a 'last one wins' manner!
|
||||
For example, in the above scenario conflicting key values in ``services``
|
||||
will overwrite those in ``packages`` because it's at the bottom of the list.
|
||||
Since both pillar SLS files contained a ``bind`` key which contained a nested
|
||||
dictionary, the pillar dictionary's ``bind`` key contains the combined contents
|
||||
of both SLS files' ``bind`` keys.
|
||||
|
||||
Including Other Pillars
|
||||
=======================
|
||||
@ -266,34 +275,118 @@ With this form, the included file (users.sls) will be nested within the 'users'
|
||||
key of the compiled pillar. Additionally, the 'sudo' value will be available
|
||||
as a template variable to users.sls.
|
||||
|
||||
.. _pillar-in-memory:
|
||||
|
||||
Viewing Minion Pillar
|
||||
=====================
|
||||
In-Memory Pillar Data vs. On-Demand Pillar Data
|
||||
===============================================
|
||||
|
||||
Once the pillar is set up the data can be viewed on the minion via the
|
||||
``pillar`` module, the pillar module comes with functions,
|
||||
:mod:`pillar.items <salt.modules.pillar.items>` and :mod:`pillar.raw
|
||||
<salt.modules.pillar.raw>`. :mod:`pillar.items <salt.modules.pillar.items>`
|
||||
will return a freshly reloaded pillar and :mod:`pillar.raw
|
||||
<salt.modules.pillar.raw>` will return the current pillar without a refresh:
|
||||
Since compiling pillar data is computationally expensive, the minion will
|
||||
maintain a copy of the pillar data in memory to avoid needing to ask the master
|
||||
to recompile and send it a copy of the pillar data each time pillar data is
|
||||
requested. This in-memory pillar data is what is returned by the
|
||||
:py:func:`pillar.item <salt.modules.pillar.item>`, :py:func:`pillar.get
|
||||
<salt.modules.pillar.get>`, and :py:func:`pillar.raw <salt.modules.pillar.raw>`
|
||||
functions.
|
||||
|
||||
Also, for those writing custom execution modules, or contributing to Salt's
|
||||
existing execution modules, the in-memory pillar data is available as the
|
||||
``__pillar__`` dunder dictionary.
|
||||
|
||||
The in-memory pillar data is generated on minion start, and can be refreshed
|
||||
using the :py:func:`saltutil.refresh_pillar
|
||||
<salt.modules.saltutil.refresh_pillar>` function:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
salt '*' pillar.items
|
||||
salt '*' saltutil.refresh_pillar
|
||||
|
||||
This function triggers the minion to asynchronously refresh the in-memory
|
||||
pillar data and will always return ``None``.
|
||||
|
||||
In contrast to in-memory pillar data, certain actions trigger pillar data to be
|
||||
compiled to ensure that the most up-to-date pillar data is available. These
|
||||
actions include:
|
||||
|
||||
- Running states
|
||||
- Running :py:func:`pillar.items <salt.modules.pillar.items>`
|
||||
|
||||
Performing these actions will *not* refresh the in-memory pillar data. So, if
|
||||
pillar data is modified, and then states are run, the states will see the
|
||||
updated pillar data, but :py:func:`pillar.item <salt.modules.pillar.item>`,
|
||||
:py:func:`pillar.get <salt.modules.pillar.get>`, and :py:func:`pillar.raw
|
||||
<salt.modules.pillar.raw>` will not see this data unless refreshed using
|
||||
:py:func:`saltutil.refresh_pillar <salt.modules.saltutil.refresh_pillar>`.
|
||||
|
||||
|
||||
How Pillar Environments Are Handled
|
||||
===================================
|
||||
|
||||
When multiple pillar environments are used, the default behavior is for the
|
||||
pillar data from all environments to be merged together. The pillar dictionary
|
||||
will therefore contain keys from all configured environments.
|
||||
|
||||
The :conf_minion:`pillarenv` minion config option can be used to force the
|
||||
minion to only consider pillar configuration from a single environment. This
|
||||
can be useful in cases where one needs to run states with alternate pillar
|
||||
data, either in a testing/QA environment or to test changes to the pillar data
|
||||
before pushing them live.
|
||||
|
||||
For example, assume that the following is set in the minion config file:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
pillarenv: base
|
||||
|
||||
This would cause that minion to ignore all other pillar environments besides
|
||||
``base`` when compiling the in-memory pillar data. Then, when running states,
|
||||
the ``pillarenv`` CLI argument can be used to override the minion's
|
||||
:conf_minion:`pillarenv` config value:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
salt '*' state.apply mystates pillarenv=testing
|
||||
|
||||
The above command will run the states with pillar data sourced exclusively from
|
||||
the ``testing`` environment, without modifying the in-memory pillar data.
|
||||
|
||||
.. note::
|
||||
Prior to version 0.16.2, this function is named ``pillar.data``. This
|
||||
function name is still supported for backwards compatibility.
|
||||
When running states, the ``pillarenv`` CLI option does not require a
|
||||
:conf_minion:`pillarenv` option to be set in the minion config file. When
|
||||
:conf_minion:`pillarenv` is left unset, as mentioned above all configured
|
||||
environments will be combined. Running states with ``pillarenv=testing`` in
|
||||
this case would still restrict the states' pillar data to just that of the
|
||||
``testing`` pillar environment.
|
||||
|
||||
Viewing Pillar Data
|
||||
===================
|
||||
|
||||
To view pillar data, use the :mod:`pillar <salt.modules.pillar>` execution
|
||||
module. This module includes several functions, each of them with their own
|
||||
use. These functions include:
|
||||
|
||||
- :py:func:`pillar.item <salt.modules.pillar.item>` - Retrieves the value of
|
||||
one or more keys from the :ref:`in-memory pillar datj <pillar-in-memory>`.
|
||||
- :py:func:`pillar.items <salt.modules.pillar.items>` - Compiles a fresh pillar
|
||||
dictionary and returns it, leaving the :ref:`in-memory pillar data
|
||||
<pillar-in-memory>` untouched. If pillar keys are passed to this function
|
||||
however, this function acts like :py:func:`pillar.item
|
||||
<salt.modules.pillar.item>` and returns their values from the :ref:`in-memory
|
||||
pillar data <pillar-in-memory>`.
|
||||
- :py:func:`pillar.raw <salt.modules.pillar.raw>` - Like :py:func:`pillar.items
|
||||
<salt.modules.pillar.items>`, it returns the entire pillar dictionary, but
|
||||
from the :ref:`in-memory pillar data <pillar-in-memory>` instead of compiling
|
||||
fresh pillar data.
|
||||
- :py:func:`pillar.get <salt.modules.pillar.get>` - Described in detail below.
|
||||
|
||||
|
||||
Pillar "get" Function
|
||||
=====================
|
||||
The :py:func:`pillar.get <salt.modules.pillar.get>` Function
|
||||
============================================================
|
||||
|
||||
.. versionadded:: 0.14.0
|
||||
|
||||
The :mod:`pillar.get <salt.modules.pillar.get>` function works much in the same
|
||||
way as the ``get`` method in a python dict, but with an enhancement: nested
|
||||
dict components can be extracted using a `:` delimiter.
|
||||
dictonaries can be traversed using a colon as a delimiter.
|
||||
|
||||
If a structure like this is in pillar:
|
||||
|
||||
@ -330,23 +423,8 @@ This makes handling nested structures much easier.
|
||||
'qux')``) to get the salt function, instead of the default dictionary
|
||||
behavior.
|
||||
|
||||
|
||||
Refreshing Pillar Data
|
||||
======================
|
||||
|
||||
When pillar data is changed on the master the minions need to refresh the data
|
||||
locally. This is done with the ``saltutil.refresh_pillar`` function.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
salt '*' saltutil.refresh_pillar
|
||||
|
||||
This function triggers the minion to asynchronously refresh the pillar and will
|
||||
always return ``None``.
|
||||
|
||||
|
||||
Set Pillar Data at the Command Line
|
||||
===================================
|
||||
Setting Pillar Data at the Command Line
|
||||
=======================================
|
||||
|
||||
Pillar data can be set at the command line like the following example:
|
||||
|
||||
@ -354,7 +432,7 @@ Pillar data can be set at the command line like the following example:
|
||||
|
||||
salt '*' state.apply pillar='{"cheese": "spam"}'
|
||||
|
||||
This will add a Pillar key of ``cheese`` with its value set to ``spam``.
|
||||
This will add a pillar key of ``cheese`` with its value set to ``spam``.
|
||||
|
||||
.. note::
|
||||
|
||||
@ -364,7 +442,7 @@ This will add a Pillar key of ``cheese`` with its value set to ``spam``.
|
||||
a security concern in some cases.
|
||||
|
||||
|
||||
Master Config In Pillar
|
||||
Master Config in Pillar
|
||||
=======================
|
||||
|
||||
For convenience the data stored in the master configuration file can be made
|
||||
@ -372,14 +450,13 @@ available in all minion's pillars. This makes global configuration of services
|
||||
and systems very easy but may not be desired if sensitive data is stored in the
|
||||
master configuration. This option is disabled by default.
|
||||
|
||||
To enable the master config from being added to the pillar set ``pillar_opts``
|
||||
to ``True``:
|
||||
To enable the master config from being added to the pillar set
|
||||
:conf_minion:`pillar_opts` to ``True`` in the minion config file:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
pillar_opts: True
|
||||
|
||||
|
||||
Minion Config in Pillar
|
||||
=======================
|
||||
|
||||
|
||||
@ -223,18 +223,35 @@ def item(*args, **kwargs):
|
||||
Return one or more pillar entries
|
||||
|
||||
pillar
|
||||
if specified, allows for a dictionary of pillar data to be made
|
||||
If specified, allows for a dictionary of pillar data to be made
|
||||
available to pillar and ext_pillar rendering. these pillar variables
|
||||
will also override any variables of the same name in pillar or
|
||||
ext_pillar.
|
||||
|
||||
.. versionadded:: 2015.5.0
|
||||
|
||||
delimiter
|
||||
Delimiter used to traverse nested dictionaries.
|
||||
|
||||
.. note::
|
||||
This is different from :py:func:`pillar.get
|
||||
<salt.modules.pillar.get>` in that no default value can be
|
||||
specified. :py:func:`pillar.get <salt.modules.pillar.get>` should
|
||||
probably still be used in most cases to retrieve nested pillar
|
||||
values, as it is a bit more flexible. One reason to use this
|
||||
function instead of :py:func:`pillar.get <salt.modules.pillar.get>`
|
||||
however is when it is desirable to retrieve the values of more than
|
||||
one key, since :py:func:`pillar.get <salt.modules.pillar.get>` can
|
||||
only retrieve one key at a time.
|
||||
|
||||
.. versionadded:: 2015.8.0
|
||||
|
||||
CLI Examples:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
salt '*' pillar.item foo
|
||||
salt '*' pillar.item foo:bar
|
||||
salt '*' pillar.item foo bar baz
|
||||
'''
|
||||
ret = {}
|
||||
|
||||
Loading…
Reference in New Issue
Block a user