The imports were scattered around quite a bit, which made it hard
to see where multiple uses of salt/utils/* imports were being pulled
in.
This PR just adjusts the ordering so it's easier to see what is included
already for the future. (This is particularly useful for catching misuses
of salt.utils.__init__.py functions.)
This PR is part of what will be an ongoing effort to use explicit
unicode strings in Salt. Because Python 3 does not suport Python 2's raw
unicode string syntax (i.e. `ur'\d+'`), we must use
`salt.utils.locales.sdecode()` to ensure that the raw string is unicode.
However, because of how `salt/utils/__init__.py` has evolved into the
hulking monstrosity it is today, this means importing a large module in
places where it is not needed, which could negatively impact
performance. For this reason, this PR also breaks out some of the
functions from `salt/utils/__init__.py` into new/existing modules under
`salt/utils/`. The long term goal will be that the modules within this
directory do not depend on importing `salt.utils`.
A summary of the changes in this PR is as follows:
* Moves the following functions from `salt.utils` to new locations
(including a deprecation warning if invoked from `salt.utils`):
`to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`,
`dequote`, `is_hex`, `is_bin_str`, `rand_string`,
`contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`,
`which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`,
`is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`,
`is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`,
`is_openbsd`, `is_aix`
* Moves the functions already deprecated by @rallytime to the bottom of
`salt/utils/__init__.py` for better organization, so we can keep the
deprecated ones separate from the ones yet to be deprecated as we
continue to break up `salt.utils`
* Updates `salt/*.py` and all files under `salt/client/` to use explicit
unicode string literals.
* Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils
import foo` becomes `import salt.utils.foo as foo`).
* Renames the `test.rand_str` function to `test.random_hash` to more
accurately reflect what it does
* Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`)
such that it returns a string matching the passed size. Previously
this function would get `size` bytes from `os.urandom()`,
base64-encode it, and return the result, which would in most cases not
be equal to the passed size.
We can't rely on lists having the same order in Python3 the same
way we rely on them in Python2. If we sort them first, and then
compare them, this test will be more reliable.
Due to a bug in the equality handler, this test was failing on
CentOS 6 running Python 2.7.13. This PR splits up the test assertions
into several different pieces in order to cover all the possible
use cases, but also pass on CentOS 6.
* salt/crypt.py: clean up open filehandles
* salt/fileclient.py: clean up open filehandles
* salt/grains/core.py: clean up open filehandles
* salt/modules/cp.py: clean up open filehandles
* salt/modules/data.py: clean up open filehandles
* salt/modules/dnsutil.py: clean up open filehandles
* salt/modules/dockerng.py: clean up open filehandles
* salt/modules/inspectlib/collector.py: clean up open filehandles
* salt/modules/file.py: clean up open filehandles
* salt/modules/hosts.py: clean up open filehandles
* salt/modules/incron.py: clean up open filehandles
* salt/modules/dpkg.py: clean up open filehandles
* salt/modules/linux_sysctl.py: clean up open filehandles
* salt/modules/netbsd_sysctl.py: clean up open filehandles
* salt/modules/network.py: clean up open filehandles
* salt/modules/nftables.py: clean up open filehandles
* salt/modules/openbsd_sysctl.py: clean up open filehandles
* salt/modules/rh_ip.py: clean up open filehandles
* salt/modules/portage_config.py: clean up open filehandles
* salt/modules/status.py: clean up open filehandles
* salt/modules/tls.py: clean up open filehandles
* salt/modules/xapi.py: clean up open filehandles
* salt/modules/x509.py: clean up open filehandles
* salt/modules/virt.py: clean up open filehandles
* salt/modules/zcbuildout.py: clean up open filehandles
* salt/returners/local_cache.py: clean up open filehandles
* salt/utils/cloud.py: clean up open filehandles
* salt/states/pkgrepo.py: clean up open filehandles
* salt/states/x509.py: clean up open filehandles
* salt/transport/mixins/auth.py: clean up open filehandles
* salt/utils/__init__.py: clean up open filehandles
* salt/states/pkg.py: clean up open filehandles
* salt/utils/minion.py: clean up open filehandles
* salt/utils/openstack/nova.py: clean up open filehandles
* salt/utils/openstack/swift.py: clean up open filehandles
* salt/utils/process.py: clean up open filehandles
* salt/utils/templates.py: clean up open filehandles
* salt/utils/virt.py: clean up open filehandles
* tests/integration/__init__.py: clean up open filehandles
* tests/integration/cli/grains.py: clean up open filehandles
* tests/integration/client/standard.py: clean up open filehandles
* tests/integration/modules/hosts.py: clean up open filehandles
* tests/unit/utils/vt_test.py: clean up open filehandles
* tests/integration/shell/enabled.py: clean up open filehandles
* tests/integration/states/cmd.py: clean up open filehandles
* tests/integration/states/file.py: clean up open filehandles
* tests/integration/states/match.py: clean up open filehandles
* tests/unit/config_test.py: clean up open filehandles
* tests/unit/templates/jinja_test.py: clean up open filehandles
* tests/unit/utils/find_test.py: clean up open filehandles
* tests/integration/modules/state.py: clean up open filehandles
* Update dnsutil_test to reflect changes in fopen usage
