the running tests run second because of alphabetical order, and stopping sshd
does not kill off the running connection, so we know that sshd has to be on
arch for us to connect, so stop it with dead and start it back with running.
It is also not possible to stop the systemd-journald process without shutting
down the full machine.
Carbon 1.1.1 for some reason added six to their setup.py, which breaks
this test since it's not installed into the virtualenv. This PR forces
this test to use a version of carbon which does not dep on six in its
setup.py.
Carbon 1.1.1 for some reason added six to their setup.py, which breaks
this test since it's not installed into the virtualenv. This PR forces
this test to use a version of carbon which does not dep on six in its
setup.py.
This makes the 2.x usage invalid syntax and forces the use of print as a
function. This adds the import to the files which I've updated in the
last couple of days but forgot to add it.
This updates the file state and execution modules to use
unicode_literals. Since the serializers and the cmd module are touched
by the file state/exec module, those are also updated here, as well as
the cmd state module, for good measure.
Additionally, I found that salt.utils.data.decode_dict (and decode_list)
are misnamed for what they actually do. Since they *encode* the
contents, the functions should be named encode_dict and encode_list,
respectively. And we also should have counterparts which actually
decode, so I've added them. The compatibility functions from salt.utils
still use the old "decode" names to preserve backward compatibility, but
they now invoke the renamed "encode" functions in salt.utils.data. Note
that this means that the compatibility functions
salt.utils.decode_dict/list, and their cognates in salt.utils.data now
do different things, but since the move to salt.utils.data is also
happening in the Oxygen release this is as good a time as any to correct
this oversight.
I've updated the jinja filter docs to include information on the renamed
jinja filters, and also added a section on jinja filter renaming to the
Oxygen release notes. There was another filter that I renamed during the
process of moving functions from salt.utils which I did not properly
document in the release notes, so this is now included along with the
others.
CentOS 7 still ships a version of Docker with API version 1.24 which
does not support this argument. This commit uses `docker.rm` to remove
the container manually after the state is run, in cases where we were
using auto_remove=True.
Much Improved Support for Docker Networking
===========================================
The `docker_network.present` state has undergone a full rewrite, which
includes the following improvements:
Full API Support for Network Management
---------------------------------------
The improvements made to input handling in the
`docker_container.running` state for 2017.7.0 have now been expanded to
docker_network.present`. This brings with it full support for all
tunable configuration arguments.
Custom Subnets
--------------
Custom subnets can now be configured. Both IPv4 and mixed IPv4/IPv6
networks are supported.
Network Configuration in :py:func:`docker_container.running` States
-------------------------------------------------------------------
It is now possible to configure static IPv4/IPv6 addresses, as well as
links and labels.
Improved Handling of Images from Custom Registries
==================================================
Rather than attempting to parse the tag from the passed image name, Salt
will now resolve that tag down to an image ID and use that ID instead.
Due to this change, there are some backward-incompatible changes to
image management. See below for a full list of these changes.
Backward-incompatible Changes to Docker Image Management
--------------------------------------------------------
Passing image names to the following functions must now be done using separate
`repository` and `tag` arguments:
- `docker.build`
- `docker.commit`
- `docker.import`
- `docker.load`
- `docker.tag`
- `docker.sls_build`
Additionally, the `tag` argument must now be explicitly passed to the
`docker_image.present` state, unless the image is being pulled from a
docker registry.
We have `output_loglevel` for controlling logging, but no corresponding
option to suppress the output from the command from being returned. This
adds a `hide_output` option to do that.
Additionally, this removes the rest of the references to the
long-since-deprecated `quiet` argument from the cmd execution module,
and adds a warning to states which use this argument. `quiet` appears to
have been incompletely removed following its deprecation. It was not
being passed to _check_loglevel(), so it was essentially ignored, but it
was still being accepted as an argument by several functions. Those
references to what has for some time now been a no-op argument are now
removed.
Elide terminology "execution failed/succeeded" since the result of these
conditions does not indicate a failure in state execution. Instead
simply report the result of each condition.
Some states are complicated and multiple subparts,
or maybe cross-call into __states__ if they manage subresources.
In these cases, they will have multiple comments.
Make this more ergonomic by supporting a list of strings as the
value for ret['comment'] in state returns and documenting this.
By joining comments on newlines, it is possible to combine
single-line and multi-line comments cleanly, as opposed to e.g. commas.
The driving impetus for this is some of the boto modules.
An update to the boto_sqs module is included as an example.
Add a check that outgoing state return data has the right shape,
and add a testcase as well.
Fix the NPM state tests and the saltmod runner & wheel state functions
to comply with the prescribed format.
This fails on centos 6 because its node is too old to support the version of
hawk bumped here https://github.com/request/request/pull/2751, we can still
test the functionality. This will pull from github, and install a specific tag
version, and we still do the uninstall using the github path.
This should be more stable.
This moves the following functions from salt.utils to salt.utils.versions:
- warn_until
- kwargs_warn_until
- compare_versions (as salt.utils.versions.compare)
- version_cmp
This PR is part of what will be an ongoing effort to use explicit
unicode strings in Salt. Because Python 3 does not suport Python 2's raw
unicode string syntax (i.e. `ur'\d+'`), we must use
`salt.utils.locales.sdecode()` to ensure that the raw string is unicode.
However, because of how `salt/utils/__init__.py` has evolved into the
hulking monstrosity it is today, this means importing a large module in
places where it is not needed, which could negatively impact
performance. For this reason, this PR also breaks out some of the
functions from `salt/utils/__init__.py` into new/existing modules under
`salt/utils/`. The long term goal will be that the modules within this
directory do not depend on importing `salt.utils`.
A summary of the changes in this PR is as follows:
* Moves the following functions from `salt.utils` to new locations
(including a deprecation warning if invoked from `salt.utils`):
`to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`,
`dequote`, `is_hex`, `is_bin_str`, `rand_string`,
`contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`,
`which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`,
`is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`,
`is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`,
`is_openbsd`, `is_aix`
* Moves the functions already deprecated by @rallytime to the bottom of
`salt/utils/__init__.py` for better organization, so we can keep the
deprecated ones separate from the ones yet to be deprecated as we
continue to break up `salt.utils`
* Updates `salt/*.py` and all files under `salt/client/` to use explicit
unicode string literals.
* Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils
import foo` becomes `import salt.utils.foo as foo`).
* Renames the `test.rand_str` function to `test.random_hash` to more
accurately reflect what it does
* Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`)
such that it returns a string matching the passed size. Previously
this function would get `size` bytes from `os.urandom()`,
base64-encode it, and return the result, which would in most cases not
be equal to the passed size.
This is the error
```
npm ERR! As of npm@5, the npm cache self-heals from corruption issues and
data extracted from the cache is guaranteed to be valid. If you want to make
sure everything is consistent, use 'npm cache verify' instead.
npm ERR!
npm ERR! If you're sure you want to delete the entire cache, rerun this command
with --force.
```
This test uses file.accumulated to accumulate data, to be used in the
file.blockreplace state. However, because this data is stored in a
dictionary, the iteration order is different on different versions of
Python. As it is very clearly stated that the accumulator data structure
is a dictionary, rather than modifying the file.accumulated behavior
this commit just updates the test so that it searches for all values
that it should find between the block markers.
This test is checking behavior that seems to have been specific to the
carbon package from PyPI. Since this is a Python 2-only module, and has
deps that do not exist on Python 3, it should not be run on Python 3.
This PR also removes the warnings that this default with change in
the Nitrogen release, updates the documetation accordingly, and adjusts
some of the affected tests.
Something else is installing pytz (likely via pip), causing filesystem
conflicts when the pkg integration tests try to install python2-pytz as
a dep of python2-django.
This commit also changes a bunch of the other tests so that they are
explicitly skipped instead of just being no-ops in cases where they are
not configured to be run. This provides more accurate information in the
overall test results.
This further abstracts some of the setup and teardown code so it can be
used for git-over-http tests.
It also moves the code that was originally added to the archive
state integration tests to create a local http server into
salt.support.helpers so that it can be more easily and portably used.
This does the following:
- Splits states for container/volume/image/network management into four
separate state modules.
- Preserves backward compatibility by making ``docker.image_present``
invoke ``docker_image.present``, etc.
- Changes how Salt detects that a container needs to be replaced.
Instead of comparing each passed argument to the named container's
configuration, it creates a temporary container, and compares that
container to the named container. If the two differ, then the older
container is removed, and the new one is renamed and started, becoming
the named container.
- Removes the unit tests for container management and replaces them with
integration tests.
- Adds unit tests for the new salt.utils.docker
The tornado web aplication that was set up in the archive tests, and then
duplicated in the remote file integration tests, starts the web server,
but never stops it. This creates a stacktrace that hangs the other test
file that attempts to start the web server.
The Application class has a `listen()` function, but not a `stop()` function.
The change uses the `HTTPServer` class to set up the listening server, but
also has the necessary `stop()` function. (The `listen()` function from the
`Application` class just calls out to the `HTTPServer`'s `listen()` function,
so this works nicely here.)
We can then call the `stop()` function in the `tearDownClass` class method.
I also removed some duplicate STATE_DIR definitions.
Put the skipIf and the destructiveTest decorators on the class instead
of each function. Also added a user.absent to the tearDown to reduce
code duplication.
Python 3 didn't like some of the stuff that we were doing with
``subprocess.check_call()`` in these tests, so to fix this I have redone
that stuff with calls to functions in the git execution module. In order
to avoid problems with running tests with no global gitconfig, I needed
to add an argument called ``git_opts`` to most of the funcs in the git
execution module (well I didn't *need* to do it to most of the funcs, it
just seemed like we shouldn't only be supporting this argument in a
single function).
This new ``git_opts`` argument is specifically for passing arguments to
the git command itself (not the subcommand). For example, ``git -c
user.name="Foo Bar" commit .....`` is different than running ``git
commit -c user.name="Foo Bar" .....``, because the ``commit`` subcommand
for git also accepts ``-c``.
* Apply fix from #38705 to 2016.3 branch
This is a better fix and covers more use cases than the sudo_user one.
* Remove saltenv param from internal state func call
This was probably redundant in the first place, but since state.sls,
state.highstate, etc. accept a saltenv param and the actual state
functions do not, this results in multiple values passed for the saltenv
param.
Remove this argument and let file.get_managed reference __env__
internally.
* Update archive tests to match 2016.11 branch
Previously file.directory correctly sets the permissions of the target
of a symlink, but it did not return the correct result during a dry run.
This change plumbs the follow_symlinks parameter to test-only functions.
Also removed skipIf logic when on Python 2.6. This fix resolves the
same test failures on 2.6 as well as Ubuntu 12 (which is apparently
running Pyhton 2.7 on our test images from Linode these days).
Also removed skipIf logic when on Python 2.6. This fix resolves the
same test failures on 2.6 as well as Ubuntu 12 (which is apparently
running Pyhton 2.7 on our test images from Linode these days).
The test that runs these states is testing for behavior that was
obsoleted by virtualenv 13.0. Ensure that we have older virtualenv
available, and then create a venv with that older version. Use the
2nd virtualenv to attempt the "weird" install.
* git.latest: fail gracefully for misconfigured remote repo
When the remote repo's HEAD refers to a nonexistent ref, this was
causing a traceback when we tried to check if the upstream tracking
branch needed to be changed after cloning the repo. This commit fixes
this traceback by gracefully failing the state when the remote HEAD is
not present in the ``git ls-remote`` output, but the desired remote
revision doesn't exist.
Additionally, a similar graceful failure now happens if the state is run
again after we gracefully fail the first time, and we need to set the
tracking branch. Trying to set the tracking branch when there is no
local branch would fail with an ambiguous error like "fatal: branch
'master' does not exist", so before we even attempt to set the tracking
branch, the state is failed with a more descriptive comment.
* Add integration test for #36242
The test that runs these states is testing for behavior that was
obsoleted by virtualenv 13.0. Ensure that we have older virtualenv
available, and then create a venv with that older version. Use the
2nd virtualenv to attempt the "weird" install.
* Fixing integration tests if azure is not present
* Fixing integration tests failures if 'git' command is missing
Skip git state integration tests if 'git' does not exists
Prevent OSError if 'git' command not found during _git_version()
* Fix PillarModuleTest::test_pillar_items: 'info' does not exist in pillar
* Fixing integration tests if azure is not present
* Fixing integration tests failures if 'git' command is missing
Skip git state integration tests if 'git' does not exists
Prevent OSError if 'git' command not found during _git_version()
