This removes the need to hard code contents in the test. The roots
fileserver just reads from the file in binary mode, so we should be able
to do the same in the test to confirm the correct behavior of the roots
backend.
On Windows, in test_symlink_list we monkeypatch the file_roots. However,
we don't ever reverse that monkeypatching.
This would not have affected other test classes, because the opts we are
monkeypatching are ones created specifically for that test class. And
due to the lexicographic order in which the tests are run,
test_symlink_list is run last. However, if we were to ever add a test to
that test class which would run after test_symlink_list, it would be
affected by the monkeypatching.
This helper was designed to ensure that the paths returned were all
unicode types, but on a couple platforms having the path be a unicode
type causes a traceback deep in the stdlib within posixpath.
To work around this, this commit makes the helper a generator that
simply decodes each tuple returned from os.walk as we iterate.
This adds a unicode_literals import to the fileserver code, and also ensures
that we return unicode paths from the serialized file/env list caches
(which when deserialized are still encoded str types on PY2).
It also adds a test and modifies a few other existing tests to confirm
that unicode paths/envs are returned *as unicode* in the return data.
This PR is part of what will be an ongoing effort to use explicit
unicode strings in Salt. Because Python 3 does not suport Python 2's raw
unicode string syntax (i.e. `ur'\d+'`), we must use
`salt.utils.locales.sdecode()` to ensure that the raw string is unicode.
However, because of how `salt/utils/__init__.py` has evolved into the
hulking monstrosity it is today, this means importing a large module in
places where it is not needed, which could negatively impact
performance. For this reason, this PR also breaks out some of the
functions from `salt/utils/__init__.py` into new/existing modules under
`salt/utils/`. The long term goal will be that the modules within this
directory do not depend on importing `salt.utils`.
A summary of the changes in this PR is as follows:
* Moves the following functions from `salt.utils` to new locations
(including a deprecation warning if invoked from `salt.utils`):
`to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`,
`dequote`, `is_hex`, `is_bin_str`, `rand_string`,
`contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`,
`which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`,
`is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`,
`is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`,
`is_openbsd`, `is_aix`
* Moves the functions already deprecated by @rallytime to the bottom of
`salt/utils/__init__.py` for better organization, so we can keep the
deprecated ones separate from the ones yet to be deprecated as we
continue to break up `salt.utils`
* Updates `salt/*.py` and all files under `salt/client/` to use explicit
unicode string literals.
* Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils
import foo` becomes `import salt.utils.foo as foo`).
* Renames the `test.rand_str` function to `test.random_hash` to more
accurately reflect what it does
* Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`)
such that it returns a string matching the passed size. Previously
this function would get `size` bytes from `os.urandom()`,
base64-encode it, and return the result, which would in most cases not
be equal to the passed size.
