From 39a82467f1e110ee6ea0281183dd582b6277ad71 Mon Sep 17 00:00:00 2001 From: Justin Findlay Date: Wed, 29 Jul 2015 23:26:51 -0600 Subject: [PATCH] expand minion reauth scalability documentation Related to #25447. --- doc/topics/tutorials/intro_scale.rst | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/doc/topics/tutorials/intro_scale.rst b/doc/topics/tutorials/intro_scale.rst index 74330266e8..00e0573898 100644 --- a/doc/topics/tutorials/intro_scale.rst +++ b/doc/topics/tutorials/intro_scale.rst @@ -67,17 +67,23 @@ subsequent retry until reaching `acceptance_wait_time_max`. Too many minions re-authing --------------------------- -This is most likely to happen in the testing phase, when all minion keys have -already been accepted, the framework is being tested and parameters change -frequently in the masters configuration file. +This is most likely to happen in the testing phase of a salt deployment, when +all minion keys have already been accepted, but the framework is being tested +and parameters are frequently changed in the salt master's configuration +file(s). -In a few cases (master restart, remove minion key, etc.) the salt-master generates -a new AES-key to encrypt its publications with. The minions aren't notified of -this but will realize this on the next pub job they receive. When the minion -receives such a job it will then re-auth with the master. Since Salt does minion-side -filtering this means that all the minions will re-auth on the next command published -on the master-- causing another "thundering herd". This can be avoided by -setting the +The salt master generates a new AES key to encrypt its publications at certain +events such as a master restart or the removal of a minion key. If you are +encountering this problem of too many minions re-authing against the master, +you will need to recalibrate your setup to reduce the rate of events like a +master restart or minion key removal (``salt-key -d``). + +When the master generates a new AES key, the minions aren't notified of this +but will discover it on the next pub job they receive. When the minion +receives such a job it will then re-auth with the master. Since Salt does +minion-side filtering this means that all the minions will re-auth on the next +command published on the master-- causing another "thundering herd". This can +be avoided by setting the .. code-block:: yaml @@ -247,4 +253,4 @@ If the job cache is necessary there are (currently) 2 options: - ext_job_cache: this will have the minions store their return data directly into a returner (not sent through the master) - master_job_cache (New in `2014.7.0`): this will make the master store the job - data using a returner (instead of the local job cache on disk). \ No newline at end of file + data using a returner (instead of the local job cache on disk).