A recent PY3-compatibility fix was made to `check_whitelist_blacklist`.
The function was previously checking for the `__iter__` attribute to
check if the blacklist and/or whitelist was a sequence of expressions
or a single expression. However, the `str` type has the `__iter__`
attribute in PY3 while it does not in PY2. To fix this, the check was
changed to see if the blacklist and/or whitelist were `list` types. This
however causes problems in `salt.daemons.masterapi` when we pass `set`
types.
To fix this, we just check if the whitelist and/or blacklist are string
types, rather than that they are _not_ some sort of sequence type.
Many funcs in this module only convert the input to bytes on Python 3
because the hashing and base64-encoding funcs used in this module
require bytes as input. However, on Python 2, to get bytes the input
will be coerced to a str type if it is a unicode type. This will result
in a UnicodeEncodeError when the unicode string contains non-ascii
characters.
This commit ensures that all input which is going to be encoded/hashed
is converted to a bytes type first, and also ensures that all
base64-encoded results are returned as a unicode string on Python 2 to
conform with changes that have been made in 2018.3 to use unicode types
where possible.
This adds a new decorator which creates a temporary directory and cleans
it after the test completes. It also modifies an existing decorator for
creating temporary files so that it accepts arguments, which will be
passed through to salt.utils.files.mkstemp().
PyYAML's scanner is not written to handle Python's unicode literal
strings, since they are not part of the YAML spec. With all strings
loaded in the YAML context now being unicode types, this causes errors
when scanning the document to construct the next node. The unicode
string literal is misidentified as a plain scalar and any colons within
it are interpreted as delimiting a mapping node. Since no whitespace appears
after the colon, this causes scanning error.
To fix it, we check for this scanning error and reset the scanner's
position and check if the first char is the "u" prefix. If so, and if
the following char is a quote, then we attempt to scan for a string
literal instead.
Adds additional tests for unicode scenarios
Fixes an issue with reg.py when deleting unicode keys
Puts the destructive tests in a try/except block so the delete occurs no
matter what
Uses a randomly generated key name for testing
We had a similar check to ensure "foo: bar" isn't loaded as a dict, this
adds a check to ensure that we only load lists when there is a leading
bracket.
