This allows for optional normalization of unicode strings, which will
make testing more reliable. In Mac OS, os.listdir() will produce certain
unicode glyphs using separate code points for the base character and
diacritic mark, while others will use a single code point to represent
the same unicode glyph.
>>> a = u'\u0065\u0301'
>>> b = u'\u00e9'
>>> print(a)
é
>>> print(b)
é
>>> a == b
False
>>> import salt.utils.stringutils
>>> salt.utils.stringutils.to_unicode(a, normalize=True) == b
True
>>>
`_get_extra_opts()` and `_get_branch_option()` were unnecessarily
quoting the value, causing it to be interpreted as a literal quote by
`subprocess.Popen()`.
Also, because there were separate helpers for repo options,
disableexcludes, branch options, and extra options, and specifically
because `_get_extra_opts()` parses *all* kwargs, any of the options from
the other helper funcs would end up being added to the command line
twice if `_get_extra_opts()` was used.
This commit consolidates all of the kwarg inspection and CLI opts
construction to a single helper function. It also adds unit tests to
make sure that we are formatting our commands properly.
Additionally, it makes a minor fix in `refresh_db()` which was not
accounted for when we changed the osmajorrelease grain to an integer in
2017.7.0.
This prevents a failed decode of undecodable data from resulting in a
traceback by catching the exception and just using the original value in
the changes dict.
While running an sdist, distutils logs when it creates paths. However,
if the paths are unicode _and_ contain unicode code points, the log
message results in a `UnicodeEncodeError` as distutils attempts to
perform string replacement using a str format string and a unicode path.
Paths recently added in the test suite's files to test unicode
compatibility have caused this issue to surface.
This fixes the resulting traceback by ensuring that Python 2 only passes
utf-8 encoded bytestrings when making the release tree.
This preserves the custom mock_open we backported from unittest.mock,
but otherwise ditches unittest.mock as it does not have
MagicMock.assert_called in Python releases before 3.6.
This allows us to maintain a uniform mock version across all platforms
and Python releases.
