This sorting was done mainly for the benefit of the test suite, but
Python 3 will raise an error when you try to sort a mixture of int and
tuple types, so sorting breaks down when there are UDP ports.
Instead, this just leaves them as an unsorted list when passed to the
API, and the test suite does the sorting before the assertEqual.
This sorting was done mainly for the benefit of the test suite, but
Python 3 will raise an error when you try to sort a mixture of int and
tuple types, so sorting breaks down when there are UDP ports.
Instead, this just leaves them as an unsorted list when passed to the
API, and the test suite does the sorting before the assertEqual.
The logic which ensures that we expose ports which are being bound,
even when not explicitly configured, was done incorrectly. UDP ports
were being passed to the API as '1234/udp' instead of (1234, 'udp').
This results in the port not being exposed properly.
The logic has been corrected. Additionally both the "ports" input
translation function, as well as the post-processing code (where the
port numbers configured in port_bindings were being added) both
contained code to "fix" any ports which were configured using
'portnum/tcp', as these must be passed to the API simply as integers. To
reduce code duplication, this normalization is now only performed at the
very end of the post-processing function, after ports have been
translated, and any missing ports from the port_bindings have been
added.
The unit test for the port_bindings input translation code, which was
written based upon the same incorrect reading of the API docs that
resulted in the incorrect behavior, have been updated to confirm the
(now) correct behavior. The unit test for the ports input translation
code has been updated to reflect the new normalization behavior.
Finally, an integration test has been added to ensure that we properly
expose UDP ports which are added as part of the post-processing
function.
The logic which ensures that we expose ports which are being bound,
even when not explicitly configured, was done incorrectly. UDP ports
were being passed to the API as '1234/udp' instead of (1234, 'udp').
This results in the port not being exposed properly.
The logic has been corrected. Additionally both the "ports" input
translation function, as well as the post-processing code (where the
port numbers configured in port_bindings were being added) both
contained code to "fix" any ports which were configured using
'portnum/tcp', as these must be passed to the API simply as integers. To
reduce code duplication, this normalization is now only performed at the
very end of the post-processing function, after ports have been
translated, and any missing ports from the port_bindings have been
added.
The unit test for the port_bindings input translation code, which was
written based upon the same incorrect reading of the API docs that
resulted in the incorrect behavior, have been updated to confirm the
(now) correct behavior. The unit test for the ports input translation
code has been updated to reflect the new normalization behavior.
Finally, an integration test has been added to ensure that we properly
expose UDP ports which are added as part of the post-processing
function.
This reverts commit 20bf4eed1d.
This change breaks publisher_acls.
1) The key_dir's permissions are controlled by `permissive_pki_access` which is
not required by publisher_acls. By default, it is also changed back to 700
each time that the salt-master restarts, so it will have to be chmodded each
time.
2) The default directory for these keys is changed, which will break a lot of
users publisher_acls setups on an upgrade to Oxygen, and require them to go
back in to chmod new directories.
I was going through and switching out the key dir to default back to
/var/cache/salt/master, and allow it to be changed, and also be able to specify
that it is a sensitive dir, but once I ran across the `permissive_pki_access`
stuff, I thought it was better to just revert this change and try again against
Fluorine, since we do not have a lot of tests in this area around publisher_acl.
This adds a workaround for a bug fixed upstream on 2 Feb 2018, which
caused a branch or tag containing a unicode character to raise a
UnicodeDecodeError. Additionally, it changes how we handle version
analysis in salt.utils.gitfs. We should be using the LooseVersion from
salt.utils.versions instead of distutils.version.
First, `shlex.split()` will raise an exception when passed a unicode
type with unicode characters in the string. This modifies our
`shlex.split()` helper to first convert the passed string to a `str`
type, and then return a decoded copy of the result of the split.
Second, this uses our `to_unicode` helper to more gracefully decode the
stdout and stderr from the command. Unit tests have been added to
confirm that the output is properly decoded, including instances where
decoding fails because the return from the command contains binary data.
When running the unit tests with the locale set to POSIX, some Unicode
tests fail:
$ LC_ALL=POSIX python3 ./tests/runtests.py --unit
[...]
======================================================================
ERROR: test_list_products (unit.modules.test_zypper.ZypperTestCase)
[CPU:0.0%|MEM:73.2%]
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/modules/test_zypper.py", line 236, in
test_list_products
'stdout': get_test_data(filename)
File "tests/unit/modules/test_zypper.py", line 53, in get_test_data
return rfh.read()
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position
828: ordinal not in range(128)
======================================================================
ERROR: test_non_ascii (unit.templates.test_jinja.TestGetTemplate)
[CPU:0.0%|MEM:73.2%]
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/templates/test_jinja.py", line 341, in test_non_ascii
result = fp.read()
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5:
ordinal not in range(128)
======================================================================
ERROR: test_non_ascii_encoding
(unit.templates.test_jinja.TestGetTemplate)
[CPU:0.0%|MEM:73.2%]
----------------------------------------------------------------------
Traceback (most recent call last):
File "tests/unit/templates/test_jinja.py", line 303, in
test_non_ascii_encoding
fp_.read(),
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 5:
ordinal not in range(128)
----------------------------------------------------------------------
Therefore open files in binary mode and explicitly decode them with
utf-8 instead of their default locale.
This removes the lowercase normalization and handles case-insensitive
path matches. It also combines local path and system path inspection
into a single helper function in both add() and remove(), for simplicity.
Additionally, the add/remove unit tests have been rewritten to test
a wider array of use cases.
- `salt/modules/rbenv.py`: Forces use of str types in the custom env dict
passed to `cmd.run_all`.
- `salt/modules/syslog_ng.py`: Ditches janky PATH munging in favor of
the stable and long-existing support built into cmdmod.py
- `salt/modules/win_path.py`: Forces use of str types in path
modification functions.
- `salt/states/win_path.py`: Completely rewritten. Duplicated code from
the execution module removed in favor of calls to the execution
module. Tests junked and 14 new tests written.
- `salt/utils/path.py`: The `which()` function was mistakenly modified
in 20033ee to inject the directories in the POSIX default path into
the PATH environment variable (even for Windows!). We never used the
PATH to find the executables, we simply cycled through the dirs one by
one and looked for an executable file matching the named path. The
code that modifies the path is now removed. In addition, `which()` now
uses `salt.utils.path.join()`, which gracefully handles mismatched
str and unicode directory components to prevent decode errors.
`join()` has also been simplified to use `salt.utils.data.decode()` to
normalize directory components to unicode.
Both string_escape and unicode_escape don't like unicode content. This
scraps the escape encoding and simply does a series of string
replacements to disabmibguate quotes, newlines, and tab characters.
