From 84c6b620055ea8d0095188b2bc8550d4929ca24b Mon Sep 17 00:00:00 2001 From: Erik Johnson Date: Sun, 29 Dec 2013 20:13:11 -0600 Subject: [PATCH] More refinement of master/minion troubleshooting docs Added an nmap example for checking if ports are open, removed some redundant wording, and reworded other parts to be more concise. --- doc/topics/troubleshooting/master.rst | 20 +++---- doc/topics/troubleshooting/minion.rst | 75 +++++++++++++++++---------- 2 files changed, 59 insertions(+), 36 deletions(-) diff --git a/doc/topics/troubleshooting/master.rst b/doc/topics/troubleshooting/master.rst index 5bdf37ffff..a861491a57 100644 --- a/doc/topics/troubleshooting/master.rst +++ b/doc/topics/troubleshooting/master.rst @@ -11,7 +11,7 @@ the foreground: .. code-block:: bash - salt-master -l debug + # salt-master -l debug Anyone wanting to run Salt daemons via a process supervisor such as `monit`_, `runit`_, or `supervisord`_, should omit the ``-d`` argument to the daemons and @@ -129,23 +129,26 @@ Or with the following Salt state: Live Python Debug Output ======================== -If the master seems to be unresponsive, a SIGUSR1 can be passed to -the processes to display where in the code they are running. If encountering a -situation like this, this debug information can be invaluable. First make -sure the master is running in the foreground: +If the master seems to be unresponsive, a SIGUSR1 can be passed to the +salt-master threads to display what piece of code is executing. This debug +information can be invaluable in tracking down bugs. + +To pass a SIGUSR1 to the master, first make sure the minion is running in the +foreground. Stop the service if it is running as a daemon, and start it in the +foreground like so: .. code-block:: bash - salt-master -l debug + # salt-master -l debug Then pass the signal to the master when it seems to be unresponsive: .. code-block:: bash - killall -SIGUSR1 salt-master + # killall -SIGUSR1 salt-master When filing an issue or sending questions to the mailing list for a problem -with an unresponsive daemon this information can be invaluable. +with an unresponsive daemon, be sure to include this information if possible. Commands Time Out or Do Not Return Output ========================================= @@ -167,4 +170,3 @@ the ``root_dir`` setting. This can result in unintended behavior if you are expecting files such as ``/etc/salt/pki`` to be pulled from the location specified with ``-c``. Modify the ``root_dir`` setting to address this behavior. - diff --git a/doc/topics/troubleshooting/minion.rst b/doc/topics/troubleshooting/minion.rst index 0865b7fd09..45468610b4 100644 --- a/doc/topics/troubleshooting/minion.rst +++ b/doc/topics/troubleshooting/minion.rst @@ -11,7 +11,7 @@ the foreground: .. code-block:: bash - salt-minion -l debug + # salt-minion -l debug Anyone wanting to run Salt daemons via a process supervisor such as `monit`_, `runit`_, or `supervisord`_, should omit the ``-d`` argument to the daemons and @@ -25,25 +25,44 @@ run them in the foreground. What Ports does the Minion Need Open? ===================================== -No ports need to be opened up on each minion. If you've put both your Salt -master and minion in debug mode and don't see an acknowledgment that your -minion has connected, it could very well be a firewall. +No ports need to be opened on the minion, as it makes outbound connections to +the master. If you've put both your Salt master and minion in debug mode and +don't see an acknowledgment that your minion has connected, it could very well +be a firewall interfering with the connection. See our :doc:`firewall +configuration ` page for help opening the firewall +on various platforms. -You can check port connectivity from the minion with the nc command: +If you have netcat installed, you can check port connectivity from the minion +with the ``nc`` command: .. code-block:: bash - nc -v -z salt.master.ip 4505 - nc -v -z salt.master.ip 4506 + $ nc -v -z salt.master.ip.addr 4505 + Connection to salt.master.ip.addr 4505 port [tcp/unknown] succeeded! + $ nc -v -z salt.master.ip.addr 4506 + Connection to salt.master.ip.addr 4506 port [tcp/unknown] succeeded! -There is also a :doc:`firewall configuration` -document that might help as well. +The `Nmap`_ utility can also be used to check if these ports are open: -If you've enabled the right TCP ports on your operating system or Linux -distribution's firewall and still aren't seeing connections, check that no -additional access control system such as `SELinux`_ or `AppArmor`_ is blocking -Salt. +.. code-block:: bash + # nmap -sS -q -p 4505-4506 salt.master.ip.addr + + Starting Nmap 6.40 ( http://nmap.org ) at 2013-12-29 19:44 CST + Nmap scan report for salt.master.ip.addr (10.0.0.10) + Host is up (0.0026s latency). + PORT STATE SERVICE + 4505/tcp open unknown + 4506/tcp open unknown + MAC Address: 00:11:22:AA:BB:CC (Intel) + + Nmap done: 1 IP address (1 host up) scanned in 1.64 seconds + +If you've opened the correct TCP ports and still aren't seeing connections, +check that no additional access control system such as `SELinux`_ or +`AppArmor`_ is blocking Salt. + +.. _`Nmap`: http://nmap.org/ .. _`SELinux`: https://en.wikipedia.org/wiki/Security-Enhanced_Linux .. _`AppArmor`: http://wiki.apparmor.net/index.php/Main_Page @@ -56,15 +75,15 @@ running any Salt module locally on a minion. These range from the original intent of salt-call, development assistance, to gathering more verbose output from calls like :mod:`state.highstate `. -When creating your state tree, it is generally recommended to invoke -:mod:`state.highstate ` with ``salt-call``. This -displays far more information about the highstate execution than calling it -remotely. For even more verbosity, increase the loglevel with the same argument -as ``salt-minion``: +When initially creating your state tree, it is generally recommended to invoke +:mod:`state.highstate ` from the minion with +``salt-call``. This displays far more information about the highstate execution +than calling it remotely. For even more verbosity, increase the loglevel with +the same argument as ``salt-minion``: .. code-block:: bash - salt-call -l debug state.highstate + # salt-call -l debug state.highstate The main difference between using ``salt`` and using ``salt-call`` is that ``salt-call`` is run from the minion, and it only runs the selected function on @@ -75,21 +94,23 @@ system `. Live Python Debug Output ======================== -If the master seems to be unresponsive, a SIGUSR1 can be passed to -the processes to display where in the code they are running. If encountering a -situation like this, this debug information can be invaluable. First make -sure the minion is running in the foreground: +If the minion seems to be unresponsive, a SIGUSR1 can be passed to the process +to display what piece of code is executing. This debug information can be +invaluable in tracking down bugs. + +To pass a SIGUSR1 to the minion, first make sure the minion is running in the +foreground. Stop the service if it is running as a daemon, and start it in the +foreground like so: .. code-block:: bash - salt-minion -l debug + # salt-minion -l debug Then pass the signal to the minion when it seems to be unresponsive: .. code-block:: bash - killall -SIGUSR1 salt-minion + # killall -SIGUSR1 salt-minion When filing an issue or sending questions to the mailing list for a problem -with an unresponsive daemon this information can be invaluable. - +with an unresponsive daemon, be sure to include this information if possible.