Fixes#38683
When two states combine the `require`, `failhard`, and `order` options,
the `order` option should be ignored because `require` is present. Then
the `failhard` option should make the state run fail.
The check for "failhard" in the `check_failhard` function in the state
compiler was too broad. We want "failhard" to be true AND tag to be in
the `running` dict. Without the parens around the first OR statement,
we were bypassing the AND requirement because failhard was found.
Since the second state's tag was not found in the running dict, the state
run stacktraces on a KeyError.
This test occassionally fails on the develop branch and I cannot
reproduce it. @cachedout recommended to add the flaky decorator
to the test until we can circle back around and look at this more
closely.
The -G/--grains/--grains-tests option was being ignored because a suite
was not defined for it. This adds the option, and also adds a dunder
init to satisfy the test parser.
This does the following:
- Splits states for container/volume/image/network management into four
separate state modules.
- Preserves backward compatibility by making ``docker.image_present``
invoke ``docker_image.present``, etc.
- Changes how Salt detects that a container needs to be replaced.
Instead of comparing each passed argument to the named container's
configuration, it creates a temporary container, and compares that
container to the named container. If the two differ, then the older
container is removed, and the new one is renamed and started, becoming
the named container.
- Removes the unit tests for container management and replaces them with
integration tests.
- Adds unit tests for the new salt.utils.docker
The tornado web aplication that was set up in the archive tests, and then
duplicated in the remote file integration tests, starts the web server,
but never stops it. This creates a stacktrace that hangs the other test
file that attempts to start the web server.
The Application class has a `listen()` function, but not a `stop()` function.
The change uses the `HTTPServer` class to set up the listening server, but
also has the necessary `stop()` function. (The `listen()` function from the
`Application` class just calls out to the `HTTPServer`'s `listen()` function,
so this works nicely here.)
We can then call the `stop()` function in the `tearDownClass` class method.
I also removed some duplicate STATE_DIR definitions.
Put the skipIf and the destructiveTest decorators on the class instead
of each function. Also added a user.absent to the tearDown to reduce
code duplication.
Python 3 didn't like some of the stuff that we were doing with
``subprocess.check_call()`` in these tests, so to fix this I have redone
that stuff with calls to functions in the git execution module. In order
to avoid problems with running tests with no global gitconfig, I needed
to add an argument called ``git_opts`` to most of the funcs in the git
execution module (well I didn't *need* to do it to most of the funcs, it
just seemed like we shouldn't only be supporting this argument in a
single function).
This new ``git_opts`` argument is specifically for passing arguments to
the git command itself (not the subcommand). For example, ``git -c
user.name="Foo Bar" commit .....`` is different than running ``git
commit -c user.name="Foo Bar" .....``, because the ``commit`` subcommand
for git also accepts ``-c``.
The tearDown appears to only be removing the grain if it matches a
specific value. This may be leading to the grain value not being blank
at the time the next test is run.
Instead of only deleting the grain if it matches a specific value,
instead delete all items from that grain to ensure that it is empty for
the next test.
This commit also modifies the master config for the test suite to use
this new option, so that a missing GitPython/Pygit2 will not crash the
test suite's master. This should make the test suite run smoother by
only negatively impacting those tests which require the pillar data
provided by git_pillar, when a valid provider is not available.
This uses a function in the runtests_helpers custom module to perform
all the logic, and only returns what failed the test. This saves us from
having to return the entire contents of sys.doc (as well as log all of
the function calls), and also removes the need to run sys.doc in batches
to get around the "max message size" issue.
Since we have recently changed the test suite to use new-style
git_pillar, GitPython or Pygit2 is a hard dep for the test suite.
Additionally, when starting up the daemons, if no IPv4 addresses can be
detected (which can happen on docker containers which tend to have
minimal installs) then the suite will time out trying to detect whether
or not the minion/sub-minion has connected, which while it does not
prove fatal for the test suite, it does make the suite take several
minutes to start up and begin running tests. This is because the test
suite invokes the manage.joined runner, which in turn invokes
salt.utils.network.ip_addrs() to get the system's IP addresses to match
against those which are connected. If it can't get the IP addresses,
then the manage.joined runner returns an empty list, and the test suite
believes that no minions have connected, and the function that
periodically runs manage.joined will eventually time out.
This commit adds messages to the console when no suitable gitfs provider
is installed, and when salt.utils.network.ip_addrs() returns an empty
list, to hopefully prompt the user to install the missing requisites.
A recent PR of mine removed the logic in symlink_list and fell back to
the cached file list generated in _file_lists(). However, this code
dates back from before the fileserver backends' symlink_list() functions
were modified to return a dict mapping links to their destinations.
This fixes the code in _file_lists() so that it returns the correct
data. It also fixes the fact that '.' was showing up in the dir list
produced by _file_lists(), and updates the associated integration test
to include the cachedir in the mocked opts.
* Typo in comment
* First convert to string if not already a string. Then to bytes under Py3.
The reason being that jids from the CLI, at least the one fed in
integration.runners.jobs.ManageTest.test_loopup_jid is loaded as an
integer, and, while the Py2 code converts JIDs to strings, under Py3, we
assume JID's are already strings.
* Mark tests which require root permissions to run
* Allow declaring that the function IS a class method.
```
Python 3.5.3 (default, Jan 21 2017, 00:29:12)
[GCC 6.3.1 20170109] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Foo:
... def bar(self):
... print('bar')
...
>>> import inspect
>>> inspect.ismethod(Foo.bar)
False
>>> inspect.ismethod(Foo().bar)
True
```
On Python 2, `inspect.ismethod` returns `True` for bound and unbound
methods while on Python 3 it only returns `True` for bound methods.
The explicit `is_class_method` is to avoid instantiating the class just
to get the function signature.
* Always decode responses to the Python version native string implementation
* Just compare the objects as matching list.
Asserting same item count doesn't make that much sense.
* Py3 compatibility
* Fix saltnado tests under Py3
* Python 3 compatibility
* Show me the full traceback
* Revert "Convert fileserver data from bytes to strings"
This reverts commit e53972f8c6.
* Revert "Under Py3, we get `bytes` when using the roots backend directly"
This reverts commit 9f73b240c1.
* Convert from bytes to str if not a binary file
* Py3 compatibility fixes.
Convert file contents from bytes to string if not a binary file
This prevents git_pillar certificate errors. Additionally, sometimes
/dev/rtc becomes unresponsive which leads the ``hwclock --compare``
command to spin indefinitely, ultimately causing the entire test run to
fail. This changeset uses a SIGALRM handler to abort the hwclock command
if no output is seen within 3 seconds and try a second time. If it times
out the second time as well, then an error will be logged but it will
not be fatal to the test (or hang it indefinitely).
In #39060, some tests were added to check for the hardware clock
getting reset when possible. The test attempted to gate the test
when access to the hardware clock was impossible, but the check
wasn't quite right and caused problems with the resulting tests.
This change makes the `has_settable_hwclock()` function public
in order to make the test gate work correctly. This function
could also be useful at the CLI level.
When running `self.run_function('status._has_settable_hwclock()')`
from the test suite, the return is a string noting that the
function is unavailable. This change allows for the `if not` check
to evaluate the return of the function.
* Add config params for custom refspecs
* Add exceptions for errors encountered modifying git config
* Make the refspecs a configurable parameter
* Make refspecs a per-remote parameter
* Update master config template to include custom refspecs opts
* Add documentation for new config params
* Update GitFS walkthrough with a section on custom refspecs
* Remove dulwich support from salt.utils.gitfs
Dulwich still lacks important features, including (but not limited to)
the following:
- Lack of the necessary support for checking out a ref needed for
git_pillar/winrepo support
- No support in its config objects for multivar git config items, making
it impossible to detect when repos have multiple refspecs set for a
given git remote
Given this information, and the fact that it trails as a distant third
to Pygit2 and GitPython, Salt will cease to support Dulwich as a git
interface moving forward.
* Excise references to dulwich from documentation
* Add mention of custom refspecs to Nitrogen release notes
* Add gitfs_refspecs to mocked opts in gitfs integration tests
Also remove dulwich from unit tests
* Add information about opts argument not being intended for CLI use
* Apply fix from #38705 to 2016.3 branch
This is a better fix and covers more use cases than the sudo_user one.
* Remove saltenv param from internal state func call
This was probably redundant in the first place, but since state.sls,
state.highstate, etc. accept a saltenv param and the actual state
functions do not, this results in multiple values passed for the saltenv
param.
Remove this argument and let file.get_managed reference __env__
internally.
* Update archive tests to match 2016.11 branch
This keeps trimmed output from failing the test. We are still testing
with a specific module lower down in the test, so this doesn't reduce
our test coverage.
This test fails often due to being trimmed, thanks to
salt.utils.dicttrim trimming values > max_event_size.
This commit changes this test so that it runs sys.doc twice, ensuring
that the return from sys.doc (for now) is not trimmed.
This keeps trimmed output from failing the test. We are still testing
with a specific module lower down in the test, so this doesn't reduce
our test coverage.
This test fails often due to being trimmed, thanks to
salt.utils.dicttrim trimming values > max_event_size.
This commit changes this test so that it runs sys.doc twice, ensuring
that the return from sys.doc (for now) is not trimmed.
Update hardware clock so that date/time changes persist
through reboot.
Add hwclock check to system integration test. Bump tolerance
of _same_times() helper to account for longer run-time of set
operations. Updating hardware rtc's can take more than 2 seconds.
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Reviewed-by: Sergey Kizunov <sergey.kizunov@ni.com>
Reviewed-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Reviewed-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
- Added the "CLI Example:" syntax to function that were missing it
- Added docs to functions missing any docs and CLI examples
- Added to functions to allow_failure in sysmod test as needed
- Also adjusted the ordering of the allow_failure structure to make it
easier to see which modules/funcitons were included (alphabetize)
Previously file.directory correctly sets the permissions of the target
of a symlink, but it did not return the correct result during a dry run.
This change plumbs the follow_symlinks parameter to test-only functions.
Batch execution was removed from NetapiClient and Saltnado in
previous commits. This change is a follow up that removes related
test cases and doc references.
With support for cmd.run --> cmd.shell aliasing removed in this PR,
some of the tests looking for the old behavior need to be adjusted.
1. `test_template_default_enabled` was renamed to `test_template_shell`
and the cmd.run call was changed to a cmd.shell call. Since the
aliasing for cmd.run to cmd.shell has been removed for templates, this
test is no longer necessary. Instead of removing the test, we can use
it to check that cmd.shell works correctly in a state + template
context.
2. The `python_shell=False` kwarg was removed from the template in the
`test_template_default_disabled` test since python_shell=False should be
the default now. (Note the test was renamed for clarity.)
