* Add Ingest pipeline methods to Elasticsearch execution module
* Refactor Elasticsearch execution module and properly handle exceptions
* Throw CommandExecutionError in methods applicable for different Elasticsearch versions
* Refactor Elasticsearch states to reflect execution module changes
* Add state for managing Elasticsearch pipelines
* Fix few typos in Elasticsearch module, return None when deleted document doesn't exist
* Implements stats and health methods for Elasticsearch
* Add Elasticsearch methods to open/close index, manage search templates and repositories
* Merge existing Elasticsearch states into single one, add Search Template handling
* Add index alias state for Elasticsearch, fix documentation
* Catch all global exceptions in Elasticsearch states, unit test all of them
* Implement few Unit tests for Elasticsearch execution module, merge fixes into deprecated elastic states
* Implement additional unit tests for Elasticsearch execution module
* Finalize Elasticsearch module documentation
With the merging of #39996, salt/modules/docker.py now imports
salt.utils.docker. However, our loader appends the module dir (in
this case salt/modules/) to sys.path temporarily for the length of the
loading process. So, as the docker execution module tries to import
salt.utils.docker, when salt.utils.docker attempts to do an "import
docker", and docker-py is *not* installed, this results in
salt/modules/docker.py (the docker execution module) being loaded in its
place, which results in tracebacks in the minion log.
Renaming the docker execution module keeps this import shadowing from
occurring. Note that we don't need to do this for the placeholder
salt/states/docker.py as it does not import salt.utils.docker.
Alternatives should say it is setting the symlink to `path` instead of
`current` which it is already set to
ssh_auth.absent should say it is going to remove the key, unless the status is
set to `add`, then it is already absent
Fixes#40322Fixes#40321
This does the following:
- Splits states for container/volume/image/network management into four
separate state modules.
- Preserves backward compatibility by making ``docker.image_present``
invoke ``docker_image.present``, etc.
- Changes how Salt detects that a container needs to be replaced.
Instead of comparing each passed argument to the named container's
configuration, it creates a temporary container, and compares that
container to the named container. If the two differ, then the older
container is removed, and the new one is renamed and started, becoming
the named container.
- Removes the unit tests for container management and replaces them with
integration tests.
- Adds unit tests for the new salt.utils.docker
The previous way this test was written was over-mocking some of the
state returns as well as tested and asserted against incorrect
behavior as shown in issue #39255. (The test asserted that a return
of '' from the call to _checkblk() was correct, which was hiding a
bug as noted in #39255 and fixed in #39272.)
Stabilizes these tests by avoiding errors when cheese, crackers
order changes in the comment and gives errors like:
- 'comment': 'Kernel modules crackers, cheese are already present',
? --------
+ 'comment': 'Kernel modules cheese, crackers are already present',
? ++++++++
* openvswitch: add a basic unit test for openvswitch
* openvswitch: enclose comments_* variables into a map
* openvswitch: only prepare error messages if type parameter is present
* openvswitch: add test case: create a GRE port
* openvswitch: remove trailing whitespace
* openvswitch: add missing "comments" variable
* openvswitch: pep8 compliance
* openvswitch: mock dig.check_ip
* openvswitch: rename "type" argument into "tunnel_type"
* openvswitch: reference bridge_exists before assignment
* openvswitch: reference port_list before assignment
* openvswitch: add upgrade note about "type" option renaming
* archive.extracted: don't try to cache local sources
This will keep us from trying to cache file when we already have it
locally, which will help significantly with larger archives.
* Update tests to reflect change in cache behavior
* Add mock return for /bin/tar
This fixes failing tests for Ubuntu 14
Also removed some mocks that were not needed anymore for the
calls to ``file.contains_regex_multiline`` because that function
is no longer called in the state file.
* adding optional response template parameters for normal and error cases
* making optional response templates affect checksum calculation in order to correctly determine if the service needs updating
* Remove trailing whitespace
* removing trailing whitespaces
* correcting invalid mock values in unit test code
* [PY3] Change log.warn statements to log.warning
log.warn is deprecated in Python 3. This change removes the
occurance of deprecation warnings for using log.warn. For example:
```
17:24:23 [WARNING ] /testing/salt/utils/verify.py:522: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
17:24:23 log.warn('Insecure logging configuration detected! Sensitive data may be logged.')
17:24:23
17:24:23 [WARNING ] Insecure logging configuration detected! Sensitive data may be logged.
```
* Tweak unit.utils.verify_test to using warning instead of warn
That function's namespace looks like this:
```
def create_internet_gateway(self, dry_run=False):
```
So when we pass in the vpc_id opbject in the test, the check later
in the function sets `dry_run=True` since the vpc_id opbject exists.
This later throws JSONResponseErrors because the `DryRun` flag is
set. This error raising functionality was added in the most recent
version of moto, which exposed this bug.
This fixes the three boto_vpc_test unit state tests. We'll see if
other tests need to be addressed in other files on a full test run.
This fixes a number of annoyances with the archive.extracted state.
Rather than re-state the changes again here, they can be observed in the
carbon release notes, where they have been documented.
* Move salt.utils.mkstemp() into salt.utils.files
Also use a default prefix
* update refs to salt.utils.mkstemp
* Update test to use new path to mkstemp helper func
* disable current at module on solaris, only compute job_kw_regtex once
* provide at_solaris (virtual=at) for Solaris-like systems
* add proper maintainer header
* forgot to update the docs again
* add notice that at module only works on linux, openbsd, and freebsd
* salt.states.at needed a lot of love
* update at_state checks to pass valid date to at.at, at.jobcheck, at.atrm,...
* today I learn about mod_watch, and I want it for at
* Update boto_dynamodb state/module for GSI management
* fix trailing whitespace
* modify format strings to conform to file style
* fix lint errors
* fix all but 1 test
* add example
* Fixing skipped boto tests to prevent errors if boto3 does not exists.
* Fix tests to prevent errors when libcloud is not present
* Fix PortageConfigTestCase in case of portage is not present
These tests have never run automatically because of an incorrect file name.
Added a skipIf on these tests as they are currently non-functioning and the
module they're testing has been deprecated.
* Openstack keystone V3 API support in keystone modules.
* Python3 compatibility unicode to six.text_type
* Return values of tests: test_user_absent, test_endpoint_absent,
test_role_absent, test_service_absent, test_tenant_absent
* Fixed tests states.test_role_present and states.service_present
* Fixed module tests (not all)
* These are the old tests and need to be rewritten anyway
* fix pylint error
* fix keystone state tests
contextlib.nested is deprecated in Python 3. Instead of having two
different ways of handling nested (PY2) vs ExitStacks (PY3), just use
nested with statements for both.
Ensure that lack of version does not result in blank version being used,
as to avoid certain Java EE applications from having issues with paths
when parallell execution is not "supported".
This fixes#24990 by both not adding a blank version if there is no
version to exract, but also by allowing you to overwrite that there
should be *no* version whether or not your WAR-file ends with `-1.2.3`.
Tests added for specifically dealing with no-to-version scenarios,
as well as several minor documentation tweaks that hopefully aren't too
interruptive!
InfluxDB versions newer than 0.8 introduced significant changes in
database concepts, user administration and Python API.
We therefore decided to rename the old InfluxDB states to 'influxdb08*'
and to reserve the 'influxdb' name for InfluxDB 0.9+ compatible states
and modules.
These tests have never run automatically because of an incorrect file name.
Added a skipIf on these tests as they are currently non-functioning and the
module they're testing has been deprecated.
The "table" argument is already part of the function signature, this means
that flush() will always force the "filter" table even when the user sets
a different one.
Due to changes made in https://github.com/saltstack/salt/pull/34004,
specifically the "Universal method for everywhere" addition, we
no longer needed to mock the return values as this gets the correct
path.
As per [their blog post of the 27th April](https://blog.readthedocs.com/securing-subdomains/) ‘Securing subdomains’:
> Starting today, Read the Docs will start hosting projects from subdomains on the domain readthedocs.io, instead of on readthedocs.org. This change addresses some security concerns around site cookies while hosting user generated data on the same domain as our dashboard.
Test Plan: Manually visited all the links I’ve modified.
* Support remote sources in a source list
This commit modifies the source_list check so that remote sources
(http(s), ftp, etc.) are not fetched more than once. To do so, it
adds the use of ``__context__`` in ``cp.cache_file`` and
``file.source_list`` to prevent multiple fetches of a single file in the
same salt run.
* Update tests
Added __context__ to test cases to reflect usage of __context__, and
also added file.source_list to mocked funcs for archive.extracted unit
test.
* Ensure rh_service not used on CloudLinux 7
Add CloudLinux to RHEL-derived distros excluded from rh_service use in osrelease >= 7
* Fix binary search and replace (#32542)
* Don't return None from eval_master (#32555)
Raise an exception instead. Because eval master if returns should return
a tuple.
* redact passwords and hashes from user.present updates
Fixes#32381
* Better log message on minion restart if master couldn't be reached. (#32576)
* Revert PR #32480 and apply #32314 with fixes / documentation (#32558)
* Revert "Fix loop in maint.flo"
This reverts commit 5196cd6a6e5db3c7b1a47b1740881bbd3e87ea3d.
* Revert "Clear VCS fsbackend and git_pillar locks on master start"
This reverts commit 7e3caa9bae1ac4de62db9924374e35a8b826937e.
* Revert "Add functions to remove VCS fsbackend update locks and git_pillar update/checkout locks"
This reverts commit 4c2db32419022501eae2a695ec488693e043d189.
* prevent eternal gitfs lock due to process crash
* Use salt.utils.fopen() instead of open()
* Make pid locking work for more than just gitfs
Also, make logging more descriptive, to aid in troubleshooting.
* Add git_pillar_global_lock config option default value
* Document proper usage of {gitfs,git_pillar}_global_lock
* Fix comments value in salt.states.pkgrepo example (#32604)
'comments' option adds '#' automatically. Example contains `#http://mirror.centos.org/centos/$releasever/os/$basearch/` string which becomes prefixed with '##' in generated file.
* alphabetize directories for dynamic modules (#32599)
Also add engines and proxy minions to the list.
* Expand on the open-source vs open-core FAQ
* Language clarification.
* Fix some mistakes in the salt-ssh thin shell script (#32583)
* [[ is bash, not compatible with /bin/sh
* check if python command exists before calling it
* Deprecate 'user' and 'group' in state cmd (#32613)
* Remove unused 'group' argument
* Fix unit testing of cmd.mod_run_check without group arg
* Deprecate 'user/group' in cmd.run
* Deprecate 'user'/'group' in cmd.script
* Deprecate 'user' in cmd.wait
* Deprecate 'user'/'group' in cmd.wait_script
* Fix mod_run_check without 'group'
* Push deprecation back one release
* Fix mac_service and mac_system modules (#32587)
* Fix mac_service module
* Add integration tests for new functions
* Start will not enable the service beforehand
* Remove unused variables
* Enable blocking on min_party until minimum set of nodes are available
* Fix the comment in min_party for salt-pr-lint-n
* Add test cases to the min_party method
* Fix the lint issues causing build failure
* Change the comment for test_min_party
* CR comments for modules/zk_concurrency.py
* Adding more examples for blocking/min_nodes
* Change the comment formatting
* Removing space at end of line
* Comments changes to rebuild
* Remove unused 'group' argument
* Fix unit testing of cmd.mod_run_check without group arg
* Deprecate 'user/group' in cmd.run
* Deprecate 'user'/'group' in cmd.script
* Deprecate 'user' in cmd.wait
* Deprecate 'user'/'group' in cmd.wait_script
* Fix mod_run_check without 'group'
* Push deprecation back one release
* fix sorting by latest version when called with an attribute
* remove reference to master_alive_check
* Fixes saltstack/salt#28262
* Resolve memory leak in authentication
* outputter virt_list does not exist anymore
* Update proxmox documentation
* Fix documentation on boto_asg and boto_elb modules and states
* modules.win_timezone: don't list all zones in debug log
* Correcty index glusterfs bricks
Fixes issue #32311
* Cleaner deprecation process with decorators
* Add deprecation decorator scaffold
* Capture type error and unhandled exceptions while function calls
* Aware of the current and future version of deprecation
* Implement initially is_deprecated decorator
* Add an alias for the capitalization
* Fix capitalization easier way
* Remove an extra line
* Add successor name to the deprecation decorator.
* Granulate logging and error messages.
* Implement function swapper
* Raise later the caught exception
* Clarify exception message
* Save function original name
* Remove an extra line
* Hide an alternative hidden function name in the error message, preserving the error itself
* Rename variable as private
* Add a method to detect if a function is using its previous version
* Message to the log and/or raise an exception accordingly to the status of used function
* Log an error along with the exception
* Add internal method documentation
* Add documentation and usage process for decorator "is_deprecated"
* Add documentation and process usage for the decorator "with_deprecated"
* Hide private method name
* Fix PEP8, re-word the error message
* Deprecate basic uptime function
* Add initial decorator unit test
* Rename old/new functions, mock versions
* Move frequent data to the test setup
* Add logging on EOL exception
* Rename and document high to low version test on is_deprecated
* Implement a test on low to high version of is_deprecated decorator
* Add a correction to the test description
* Remove a dead code
* Implement a test for high to low version on is_deprecated, using with_successor param
* Correct typso adn mistaeks
* Implement high to low version with successor param on is_deprecated
* Setup a virtual name for the module
* Implement test for with_deprecated should raise an exception if same deprecated function not found
* Implement test for with_deprecated an old function is picked up if configured
* Correct test description purpose
* Implement test with_deprecated when no deprecation is requested
* Add logging test to the configured deprecation request
* Add logging testing when deprecated version wasn't requested
* Implement test EOL for with_deprecated decorator
* Correct test explanation
* Rename the test
* Implement with_deprecated no EOL, deprecated other function name
* Implement with_deprecated, deprecated other function name, EOL reached
* Add test description for the with_deprecated + with_name + EOL
* Fix confusing test names
* Add logging test to the is_deprecated decorator when function as not found.
* Add more test point to each test, remove empty lines
* Bugfix: at certain conditions a wrong alias name is reported to the log
* Fix a typo in a comment
* Add test for the logging
* Disable a pylint: None will _never_ be raised
* Fix test for the deprecated "status.uptime" version
* Bugfix: Do not yank raised exceptions
* Remove unnecessary decorator
* Add test for the new uptime
* Add test for the new uptime fails when /proc/uptime does not exists
* Rename old test case
* Skip test for the UTC time, unless freeze time is used.
* Fix pylint
* Fix documentation
* Bugfix: proxy-pass the docstring of the decorated function
* Lint fix
* Fixessaltstack/salt#28262 for 2015.5 branch
* Update master config docs
* Improve git_pillar documentation/logging
* Add note about different behavior of top file in git_pillar
* Make log entry for a missing pillar SLS file more accurate for git_pillar
* FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info.
* FreeBSD supports packages in format java/openjdk7 so the prior commit broke that functionality. Check freebsd/pkg#1409 for more info.
* Update glusterfs_test to be inline with #32312
* Fix salt-cloud paralell provisioning
Closes#31632
* Ignore Raspbian in service.py __virtual__ (#32421)
* Ignore Raspbian in service.py __virtual__
This prevents more than one execution module from trying to load as the
service virtual module.
Refs: #32413
* pack __salt__ before loading provider overrides
We can (and should) pack here since we're just packing a reference to the
object. __salt__ needs to be available when we're loading our provider
overrides
* Fix broken __salt__ dict in provider override
Using ret.items() here sets ``__salt__`` to its items (tuple containing
function name and reference), breaking usage of ``__salt__`` inside
overridden functions.
* Merge #32293 with test fixes (#32418)
* Fix issue #11497
* Remove check for working directory presence in tests
* Fix Domainname introspection
Default value needs to be extracted from the container itself,
because dockerd set Domainname value when network_mode=host.
* Add pgjsonb_queue to queue doc index
* Pylint fixes
* Pass parser options into batch mode
Resolves#31738
* Changed the target file in file.symlink test (#32443)
* Argument name in docs should match actual arg name (#32445)
Fixes#31851
* tests.integration: bypass MacOS TMPDIR, gettempdir (#32447)
Updates 0edd532, 8f558a5.
When logging in as root over `ssh root@host`, `$TMPDIR` and
`tempfile.gettempdir()` are both set to a variation of:
```
/private/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/
```
When logging in as root over `sudo -i`, `$TMPDIR` is unset and
`tempfile.gettempdir()` is set to `/tmp`.
My guess is that the second case is an unintended or uncorrected omision
by Apple as they have introduced the longer, randomized temp path in a
recent version of MacOS.
* Issue #28706: Fix state user.present behavior. (#32448)
- As mentionned in issue #28706, state user.present no longer remove
user from groups if the keyword 'groups' with empty value '[]' is not
explicitly set, salt will assume current groups are still wanted.
* tests.integration: fix 4230c8a
* Move the tables of virtual modules to individual documentation pages
* Add new doc pages to toctree
* Add external ref to windows package manager docs
* Improve docstrings
* Add documentation on virtual module provider overrides to the module docs
* Clarify the scope of the provider param in states.
* Add link to provider override docs to all package providers
* Add link to provider override docs to all service providers
* Add link to provider override docs to all user providers
* dd link to provider override docs to all shadow providers
* Add link to provider override docs to all group providers
* Backport 31164 and 31364 (#32474)
* Don't send REQ while another one is waiting for response.
The message has to be removed from the queue the only *after* it's
already processed to don't confuse send() functionality that expects
empty queue means: there's no active sendings.
* Fixed zeromq ReqMessageClient destroy
* Add link to provider override docs to opkg.py
This is a companion to https://github.com/saltstack/salt/pull/32458, but
this module was not added until the 2016.3 branch, so the documentation
is being updated there for this module.
* Add documentation for some master/minion configs (#32454)
Refs #32400
Adds docs for:
- cli_summary
- event_return_queue
- event_return_whitelist
- event_return_blacklist
- file_recv_max_size
- fileserver_followsymlinks
- fileserver_ignoresymlinks
- fileserver_limit_traversal
* Automatically detect correct MySQL password column for 5.7 and fix setting passwords (#32440)
* Automatically detect MySQL password column
* Fix changing password in MySQL 5.7
* Fix lint test
* Fix unit tests (?)
They will still fail if "authentication_string" is legitimately the right column name, but I don't know what to do about that.
* Additional unit test fix
* Only unsub if we have a jid
Closes#32479
