2012-02-17 21:04:05 +00:00
|
|
|
===============
|
|
|
|
|
Troubleshooting
|
|
|
|
|
===============
|
|
|
|
|
|
2012-03-15 00:09:19 +00:00
|
|
|
The intent of the troubleshooting section is to introduce solutions to a
|
2012-02-17 21:04:05 +00:00
|
|
|
number of common issues encountered by users and the tools that are available
|
|
|
|
|
to aid in developing states and salt code.
|
|
|
|
|
|
|
|
|
|
Running in the Foreground
|
|
|
|
|
=========================
|
|
|
|
|
|
|
|
|
|
A great deal of information is available via the debug logging system, if you
|
|
|
|
|
are having issues with minions connecting or not starting run the minion and/or
|
|
|
|
|
master in the foreground:
|
|
|
|
|
|
|
|
|
|
# salt-master -l debug
|
|
|
|
|
# salt-minion -l debug
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Using salt-call
|
|
|
|
|
===============
|
|
|
|
|
|
|
|
|
|
The salt-call command was originally developed for aiding in the development
|
|
|
|
|
of new salt modules. Since then many applications have arisen for the salt-call
|
|
|
|
|
command that is bundled with the salt minion. These range from the original
|
2012-03-15 00:09:19 +00:00
|
|
|
intent of the salt-call, development assistance, to gathering large amounts of
|
2012-02-17 21:04:05 +00:00
|
|
|
data from complex calls like state.highstate.
|
|
|
|
|
|
2012-03-15 00:09:19 +00:00
|
|
|
When developing the state tree it is generally recommended to invoke
|
2012-02-17 21:04:05 +00:00
|
|
|
state.highstate with salt-call, this displays a great deal more information
|
2012-03-15 00:09:19 +00:00
|
|
|
about the highstate execution than if it is called remotely.
|
2012-02-17 21:04:05 +00:00
|
|
|
|
2012-03-01 18:23:48 +00:00
|
|
|
Too many open files
|
|
|
|
|
===================
|
2012-02-17 21:04:05 +00:00
|
|
|
|
2012-03-01 18:23:48 +00:00
|
|
|
The salt-master needs at least 2 sockets per host that connects to it, one for
|
2012-03-15 00:09:19 +00:00
|
|
|
the Publisher and one for response port. Thus, large installations may upon
|
2012-03-01 18:23:48 +00:00
|
|
|
scaling up the number of minions accessing a given master, encounter:
|
|
|
|
|
|
|
|
|
|
12:45:29,289 [salt.master ][INFO ] Starting Salt worker process 38
|
|
|
|
|
Too many open files
|
|
|
|
|
sock != -1 (tcp_listener.cpp:335)
|
|
|
|
|
|
2012-03-15 00:09:19 +00:00
|
|
|
The solution to this would be to check the number of files allowed to be
|
2012-03-01 18:23:48 +00:00
|
|
|
opened by the user running salt-master (root by default):
|
|
|
|
|
|
|
|
|
|
[root@salt-master ~]# ulimit -n
|
|
|
|
|
1024
|
|
|
|
|
|
|
|
|
|
And modify that value to be at least equal to the number of minions x 2.
|
2012-03-15 00:09:19 +00:00
|
|
|
This setting can be changed in limits.conf as the nofile value(s),
|
2012-03-01 18:23:48 +00:00
|
|
|
and activated upon new a login of the specified user.
|
|
|
|
|
|
|
|
|
|
So, an environment with 1800 minions, would need 1800 x 2 = 3600 as a minimum
|
