* * Improve init script: specifically manage salt configurations rather than arbitrary salt processes
Unfortunately SysV init scripts tend to rummage through PIDs filtering for
appropriate processes to manage. Unfortunately the filters are usually weak
and don't account for similar processes run by other users, PIDs of dead
processes being re-used for completely different executables, etc.. These
weaknesses can result in killing unrelated processes with potentially serious
results.
These improvements to the SysV init script is a complete rewrite with the
following improvements:
* Specifically manage individual salt configurations rather than looking for
salt minion-like processes.
* Obtain salt minion information from the salt configuration - use the
information to manage the specifically configured process.
* Drop all of the platform-specific helper functions that allow the
previously-mentioned weaknesses.
+ Unfortunately this means that the output information may not match the
specific platform (this could easily be corrected).
* Now can manage multiple salt processes started by different users
+ Unfortunately starts/stops/restarts as a group and is unable to manage
them both as a group or as individual processes (this could easily be
corrected)
The new initscript also allows various control variables to be overridden by
environment variables or through settings put in ``/etc/sysconf/salt`` or
``/etc/default/salt``.
:SALTMINION_DEBUG: Dump each line expansion before execution, output system
information on failure. Default: unset
:SALTMINION_BINDIR: Location of ``salt-minion``, ``salt-call`` and other
executables. Default: ``/usr/bin``
:SALTMINION_SYSCONFDIR: The parent directory for the ``salt`` configuration
directory and the ``sysconfig`` or ``default``
directory.
* Add lines that went missing in the rebase+squash
* fix sorting by latest version when called with an attribute
* remove reference to master_alive_check
* Fixes saltstack/salt#28262
* Resolve memory leak in authentication
* outputter virt_list does not exist anymore
* Update proxmox documentation
* Fix documentation on boto_asg and boto_elb modules and states
* modules.win_timezone: don't list all zones in debug log
* Correcty index glusterfs bricks
Fixes issue #32311
* Cleaner deprecation process with decorators
* Add deprecation decorator scaffold
* Capture type error and unhandled exceptions while function calls
* Aware of the current and future version of deprecation
* Implement initially is_deprecated decorator
* Add an alias for the capitalization
* Fix capitalization easier way
* Remove an extra line
* Add successor name to the deprecation decorator.
* Granulate logging and error messages.
* Implement function swapper
* Raise later the caught exception
* Clarify exception message
* Save function original name
* Remove an extra line
* Hide an alternative hidden function name in the error message, preserving the error itself
* Rename variable as private
* Add a method to detect if a function is using its previous version
* Message to the log and/or raise an exception accordingly to the status of used function
* Log an error along with the exception
* Add internal method documentation
* Add documentation and usage process for decorator "is_deprecated"
* Add documentation and process usage for the decorator "with_deprecated"
* Hide private method name
* Fix PEP8, re-word the error message
* Deprecate basic uptime function
* Add initial decorator unit test
* Rename old/new functions, mock versions
* Move frequent data to the test setup
* Add logging on EOL exception
* Rename and document high to low version test on is_deprecated
* Implement a test on low to high version of is_deprecated decorator
* Add a correction to the test description
* Remove a dead code
* Implement a test for high to low version on is_deprecated, using with_successor param
* Correct typso adn mistaeks
* Implement high to low version with successor param on is_deprecated
* Setup a virtual name for the module
* Implement test for with_deprecated should raise an exception if same deprecated function not found
* Implement test for with_deprecated an old function is picked up if configured
* Correct test description purpose
* Implement test with_deprecated when no deprecation is requested
* Add logging test to the configured deprecation request
* Add logging testing when deprecated version wasn't requested
* Implement test EOL for with_deprecated decorator
* Correct test explanation
* Rename the test
* Implement with_deprecated no EOL, deprecated other function name
* Implement with_deprecated, deprecated other function name, EOL reached
* Add test description for the with_deprecated + with_name + EOL
* Fix confusing test names
* Add logging test to the is_deprecated decorator when function as not found.
* Add more test point to each test, remove empty lines
* Bugfix: at certain conditions a wrong alias name is reported to the log
* Fix a typo in a comment
* Add test for the logging
* Disable a pylint: None will _never_ be raised
* Fix test for the deprecated "status.uptime" version
* Bugfix: Do not yank raised exceptions
* Remove unnecessary decorator
* Add test for the new uptime
* Add test for the new uptime fails when /proc/uptime does not exists
* Rename old test case
* Skip test for the UTC time, unless freeze time is used.
* Fix pylint
* Fix documentation
* Bugfix: proxy-pass the docstring of the decorated function
* Lint fix
* Fixessaltstack/salt#28262 for 2015.5 branch
* Update master config docs
* Improve git_pillar documentation/logging
* Add note about different behavior of top file in git_pillar
* Make log entry for a missing pillar SLS file more accurate for git_pillar
* FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info.
* FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info.
* Update glusterfs_test to be inline with #32312
* Fix salt-cloud paralell provisioning
Closes#31632
* Ignore Raspbian in service.py __virtual__ (#32421)
* Ignore Raspbian in service.py __virtual__
This prevents more than one execution module from trying to load as the
service virtual module.
Refs: #32413
* pack __salt__ before loading provider overrides
We can (and should) pack here since we're just packing a reference to the
object. __salt__ needs to be available when we're loading our provider
overrides
* Fix broken __salt__ dict in provider override
Using ret.items() here sets ``__salt__`` to its items (tuple containing
function name and reference), breaking usage of ``__salt__`` inside
overridden functions.
* Merge #32293 with test fixes (#32418)
* Fix issue #11497
* Remove check for working directory presence in tests
* Fix Domainname introspection
Default value needs to be extracted from the container itself,
because dockerd set Domainname value when network_mode=host.
* Add pgjsonb_queue to queue doc index
* Pylint fixes
* Pass parser options into batch mode
Resolves#31738
* Changed the target file in file.symlink test (#32443)
* Argument name in docs should match actual arg name (#32445)
Fixes#31851
* tests.integration: bypass MacOS TMPDIR, gettempdir (#32447)
Updates 0edd532, 8f558a5.
When logging in as root over `ssh root@host`, `$TMPDIR` and
`tempfile.gettempdir()` are both set to a variation of:
```
/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/
```
When logging in as root over `sudo -i`, `$TMPDIR` is unset and
`tempfile.gettempdir()` is set to `/tmp`.
My guess is that the second case is an unintended or uncorrected omision
by Apple as they have introduced the longer, randomized temp path in a
recent version of MacOS.
* Issue #28706: Fix state user.present behavior. (#32448)
- As mentionned in issue #28706, state user.present no longer remove
user from groups if the keyword 'groups' with empty value '[]' is not
explicitly set, salt will assume current groups are still wanted.
* tests.integration: fix 4230c8a
* Move the tables of virtual modules to individual documentation pages
* Add new doc pages to toctree
* Add external ref to windows package manager docs
* Improve docstrings
* Add documentation on virtual module provider overrides to the module docs
* Clarify the scope of the provider param in states.
* Add link to provider override docs to all package providers
* Add link to provider override docs to all service providers
* Add link to provider override docs to all user providers
* dd link to provider override docs to all shadow providers
* Add link to provider override docs to all group providers
* Backport 31164 and 31364 (#32474)
* Don't send REQ while another one is waiting for response.
The message has to be removed from the queue the only *after* it's
already processed to don't confuse send() functionality that expects
empty queue means: there's no active sendings.
* Fixed zeromq ReqMessageClient destroy
* Add link to provider override docs to opkg.py
This is a companion to https://github.com/saltstack/salt/pull/32458, but
this module was not added until the 2016.3 branch, so the documentation
is being updated there for this module.
* Add documentation for some master/minion configs (#32454)
Refs #32400
Adds docs for:
- cli_summary
- event_return_queue
- event_return_whitelist
- event_return_blacklist
- file_recv_max_size
- fileserver_followsymlinks
- fileserver_ignoresymlinks
- fileserver_limit_traversal
* Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440)
* Automatically detect MySQL password column
* Fix changing password in MySQL 5.7
* Fix lint test
* Fix unit tests (?)
They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that.
* Additional unit test fix
* Only unsub if we have a jid
Closes#32479
Updates 0edd532, 8f558a5.
When logging in as root over `ssh root@host`, `$TMPDIR` and
`tempfile.gettempdir()` are both set to a variation of:
```
/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/
```
When logging in as root over `sudo -i`, `$TMPDIR` is unset and
`tempfile.gettempdir()` is set to `/tmp`.
My guess is that the second case is an unintended or uncorrected omision
by Apple as they have introduced the longer, randomized temp path in a
recent version of MacOS.
Conflicts:
- salt/cli/daemons.py
- salt/config/__init__.py
- salt/exceptions.py
- salt/pillar/foreman.py
- salt/returners/local_cache.py
- salt/states/saltmod.py
- salt/utils/__init__.py
- salt/utils/cloud.py
The merge conflicts in salt/cli/daemons.py are still present in this commit.
@s0undt3ch is going to resolve them after this commit.
__process_multiprocessing_logging_queue:
- A leak was observed in this process on Windows. On Windows, creating a
new process doesn't fork (copy the parent process image). Due to this, we
need to setup extended logging inside this process. The leak was due to
the temp null and temp logging handlers being active which just store
messages but don't process them. Setting up extended logging removes
those handlers and fixes the leak (as well as properly dealing with
incoming logging messages).
setup_multiprocessing_logging:
- A leak was obversed in processes that used this on Windows. Similar
to above, due to lack of fork, the new process will have the temp null
and temp logging handlers enabled, causing the leak. In this function,
removing those handlers prevents the leak.
Signed-off-by: Sergey Kizunov <sergey.kizunov@ni.com>
This rounds out initial (alpha) support for a salt-ssh test runner.
At present, one must manually specify the --ssh flag to ensure that
the ssh deamons are spun up properly.
Conflicts:
tests/integration/__init__.py
tests/integration/files/conf/_ssh/sshd_config
tests/integration/files/conf/sshd_config
tests/integration/files/ext-conf/sshd_config
Since salt's cli tools now try to change the user to the configured one, when running the tests suite as non-root, we need to adapt the user setting on the configuration files.
* We were instantiating the integration testing local client too soon in the process. Deferring it to a little latter made it all work.
* Added the ability to dump the master and minion testing configurations to files. Just prepend the `runtests` call with `DUMP_SALT_CONFIG=1` and you'll get the computed configuration files in `/tmp/salttest/{master,minion}`.
* We now also un-subscribe from the event's once done.
* Added `salt.utils.event.LocalClientEvent` just to differentiate from `salt.utils.event.MasterEvent` in the logs so we know who's who. It's just a subclass of the latter.
We now support runtime created files that can be used like `salt://my-temp-file.txt`.
Just make sure you write them to `os.path.join(integration.TMP_STATE_TREE, 'my-temp-file.txt')`.
The methods `assertSaltTrueReturn`, `assertSaltFalseReturn` and `assertSaltNoneReturn` now reuse the code from `__getWithinSaltReturn` making `__assertReturn` obsolete and for that removed.
* Created a method which gets a deep keyed value in a salt state return. For example, allow getting, `ret['changes']['new']`.
* Created a method which tries to make sure the correct type of the passed keys for the above added method.
* Also added were the methods, `assertInSaltReturn`, `assertNotInSaltReturn`, `assertInSaltReturnRegexpMatches`, which simplified the existing `assertInSaltComment`, `assertNotInSaltComment`, `assertSaltCommentRegexpMatches`, `assertSaltStateChangesEqual` and `assertSaltStateChangesNotEqual`, also allowing to do similar tests in every key(deep or not) of a salt state return dictionary.
* Two new methods were added to `SaltReturnAssertsMixIn`, `SaltReturnAssertsMixIn.assertSaltStateChangesEqual()` and `SaltReturnAssertsMixIn.assertSaltStateChangesNotEqual()`, in order to be able to test salt state returns. These methods, besides asserting that the state was correctly applied using `SaltReturnAssertsMixIn.assertSaltTrueReturn()`, are also able to test for what's in the `changes` key of salt's return, it can even "go inside" the `changes` dictionary, just have a look at what was done in `tests.integration.states.ssh`.
