We had a similar check to ensure "foo: bar" isn't loaded as a dict, this
adds a check to ensure that we only load lists when there is a leading
bracket.
pip.installed state calls pip.freeze to check for existing installation
and to verify installation post-install. This patch propagates the
env_vars passed to pip.installed to the pip.freeze function. It also modifies
existing pip unit tests to test new functionality and adds an integration test
that verifies the expected correct behavior
Fixes#46127
Rework _run_os_grains_tests() to check all os grains including osfinger.
Fix osfinger on Debian (e.g. use "Debian-9" instead of
"Debian GNU/Linux-9").
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
Use the full content of /etc/os-release for the Ubuntu 16.04 (artful)
os_grains test case. This reveals a bug that 'oscodename' is not
correctly calculated if /etc/os-release provides PRETTY_NAME:
FAIL: test_ubuntu_os_grains (tests.unit.grains.test_core.CoreGrainsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/grains/test_core.py", line 439, in test_ubuntu_os_grains
self._run_ubuntu_os_grains_tests(_os_release_map)
File "tests/unit/grains/test_core.py", line 446, in _run_ubuntu_os_grains_tests
self._run_os_grains_tests(os_release_map)
File "tests/unit/grains/test_core.py", line 262, in _run_os_grains_tests
self.assertEqual(os_grains.get('oscodename'), os_release_map['oscodename'])
AssertionError: 'Ubuntu 16.04.3 LTS' != 'xenial'
- Ubuntu 16.04.3 LTS
+ xenial
fixes#34423 for Ubuntu
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
Instead of storing pre-parsed os-release files, add support for reading
os-release files from disk.
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
To simplify running unit tests against os_data(), make the
_parse_os_release() function always callable to avoid needing to mock
os.path.isfile().
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
_run_ubuntu_os_grains_tests is only used once and does not provide any
useful abstraction since the introduction of _run_os_grains_tests.
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
_run_suse_os_grains_tests() and _run_ubuntu_os_grains_tests() share most
of their logic. Combine the common part in _run_os_grains_tests() and
let the remaining small parts live in their origin functions.
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
When the Kubernetes client is not installed, the import of
salt.modules.kubernetes will still succeed, but HAS_LIBS will be set to
False (since the library import will be covered by a try-except clause).
Therefore expect the salt.modules.kubernetes to always succeed and check
kubernetes.HAS_LIBS instead for the presence of the kubernetes library.
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
`_get_extra_opts()` and `_get_branch_option()` were unnecessarily
quoting the value, causing it to be interpreted as a literal quote by
`subprocess.Popen()`.
Also, because there were separate helpers for repo options,
disableexcludes, branch options, and extra options, and specifically
because `_get_extra_opts()` parses *all* kwargs, any of the options from
the other helper funcs would end up being added to the command line
twice if `_get_extra_opts()` was used.
This commit consolidates all of the kwarg inspection and CLI opts
construction to a single helper function. It also adds unit tests to
make sure that we are formatting our commands properly.
Additionally, it makes a minor fix in `refresh_db()` which was not
accounted for when we changed the osmajorrelease grain to an integer in
2017.7.0.
This sorting was done mainly for the benefit of the test suite, but
Python 3 will raise an error when you try to sort a mixture of int and
tuple types, so sorting breaks down when there are UDP ports.
Instead, this just leaves them as an unsorted list when passed to the
API, and the test suite does the sorting before the assertEqual.
The logic which ensures that we expose ports which are being bound,
even when not explicitly configured, was done incorrectly. UDP ports
were being passed to the API as '1234/udp' instead of (1234, 'udp').
This results in the port not being exposed properly.
The logic has been corrected. Additionally both the "ports" input
translation function, as well as the post-processing code (where the
port numbers configured in port_bindings were being added) both
contained code to "fix" any ports which were configured using
'portnum/tcp', as these must be passed to the API simply as integers. To
reduce code duplication, this normalization is now only performed at the
very end of the post-processing function, after ports have been
translated, and any missing ports from the port_bindings have been
added.
The unit test for the port_bindings input translation code, which was
written based upon the same incorrect reading of the API docs that
resulted in the incorrect behavior, have been updated to confirm the
(now) correct behavior. The unit test for the ports input translation
code has been updated to reflect the new normalization behavior.
Finally, an integration test has been added to ensure that we properly
expose UDP ports which are added as part of the post-processing
function.
When running the unit tests with the locale set to POSIX, some Unicode
tests fail:
$ LC_ALL=POSIX python3 ./tests/runtests.py --unit
[...]
======================================================================
ERROR: test_list_products (unit.modules.test_zypper.ZypperTestCase)
[CPU:0.0%|MEM:73.2%]
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/modules/test_zypper.py", line 236, in
test_list_products
'stdout': get_test_data(filename)
File "tests/unit/modules/test_zypper.py", line 53, in get_test_data
return rfh.read()
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
828: ordinal not in range(128)
======================================================================
ERROR: test_non_ascii (unit.templates.test_jinja.TestGetTemplate)
[CPU:0.0%|MEM:73.2%]
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/templates/test_jinja.py", line 341, in test_non_ascii
result = fp.read()
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5:
ordinal not in range(128)
======================================================================
ERROR: test_non_ascii_encoding
(unit.templates.test_jinja.TestGetTemplate)
[CPU:0.0%|MEM:73.2%]
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/templates/test_jinja.py", line 303, in
test_non_ascii_encoding
fp_.read(),
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5:
ordinal not in range(128)
----------------------------------------------------------------------
Therefore open files in binary mode and explicitly decode them with
utf-8 instead of their default locale.
When boto and moto are not installed, the BotoVpcTestCaseBase() should
be skipped. _get_boto_version() and _get_moto_version() require that
boto and moto are installed, but are called anyway in the skipIf
condition.
The test_run unit tests end result when a command is not found. When
_run() is called without setting cwd, it will use the home directory as
working directory. When the home directory does not exist, the unit test
will fail:
Traceback (most recent call last):
File "tests/unit/modules/test_cmdmod.py", line 231, in test_run
ret = cmdmod._run('foo', use_vt=True).get('stderr')
File "salt/modules/cmdmod.py", line 536, in _run
.format(cwd)
salt.exceptions.CommandExecutionError: Specified cwd
'/sbuild-nonexistent' either not absolute or does not exist
Therefore set cwd to the current directory since the working directory
is not used in this test case.
This fixes one failing test of #45627.
Signed-off-by: Benjamin Drung <benjamin.drung@profitbricks.com>
Cherry-picked and squashed the two commits from the develop branch
(details at the end of this message)
The unit test module didn't exist at all on 2017.7, hence a conflict during
cherry picking, resolved by introducing only the new tests (the other ones
don't pass as is on 2017.7)
Also, the mock in unit tests for mine.get had to be adapted,
because the return of salt.utils.minions.CkMinions._check_compound_minions()
takes a different form on 2017.7 (just the list of minions).
Original cherry-picked commits:
commit 115ebef6a089dedf8dbadd3b4cca768adbb4a710
Author: Georges Racinet <gracinet@anybox.fr>
Date: Wed Nov 8 18:02:29 2017 +0100
Unit test for backwards compatibility of mine.get
Could not find calls to RemoteFuncs in unit tests, therefore started a
new one, hope it's in the appropriate place.
The test replaces the caceh by its own (very limited) one. It's quite
possible it could be improved by reusing another one meant for unit tests.
commit dc884478399355f3fbea8626a8366b933580dee0
Author: Georges Racinet <gracinet@anybox.fr>
Date: Mon Oct 30 13:49:13 2017 +0100
Backwards compat for mine.get on pre-Nitrogen minions
With this change, the master will accept `expr_form` from
its minions doing a `mine.get`, which is what minions versions
before Nitrogen would do. This solves issue #42713.
In case both `tgt_type` and `expr_form` are present, the former
gets precedence.
Performance-wise, this adds a single dict lookup for minions
running Nitrogen onwards, and two for pre-Nitrogen minions.
This is, in my opinion, acceptable.
This also modifies the virtual_subtype code slightly, due to an issue I
noticed in testing. We were reading from `/proc/1/cgroup` twice (once
for LXC, and then again for Docker).
PR #42064 modified the YAML dumper to display unicode text as a unicode
literal rather than with the !!python/unicode extension prefix. This
updates the test to reflect the change in Salt's behavior.
Remove while loop that was hanging when `salt` wasn't in the path
Add salt.util.path.safe_path function to check for unsafe paths
Pass root_dir to all calls to `verify_env`
The `beacons_before_connect` and `scheduler_before_connect` features
that came from PR #38289 had code that was lost and so the features
don't work. Added them back in and also added unit tests to help
prevent this type of issue in the future.
Signed-off-by: Sergey Kizunov <sergey.kizunov@ni.com>
