This PR is part of what will be an ongoing effort to use explicit
unicode strings in Salt. Because Python 3 does not suport Python 2's raw
unicode string syntax (i.e. `ur'\d+'`), we must use
`salt.utils.locales.sdecode()` to ensure that the raw string is unicode.
However, because of how `salt/utils/__init__.py` has evolved into the
hulking monstrosity it is today, this means importing a large module in
places where it is not needed, which could negatively impact
performance. For this reason, this PR also breaks out some of the
functions from `salt/utils/__init__.py` into new/existing modules under
`salt/utils/`. The long term goal will be that the modules within this
directory do not depend on importing `salt.utils`.
A summary of the changes in this PR is as follows:
* Moves the following functions from `salt.utils` to new locations
(including a deprecation warning if invoked from `salt.utils`):
`to_bytes`, `to_str`, `to_unicode`, `str_to_num`, `is_quoted`,
`dequote`, `is_hex`, `is_bin_str`, `rand_string`,
`contains_whitespace`, `clean_kwargs`, `invalid_kwargs`, `which`,
`which_bin`, `path_join`, `shlex_split`, `rand_str`, `is_windows`,
`is_proxy`, `is_linux`, `is_darwin`, `is_sunos`, `is_smartos`,
`is_smartos_globalzone`, `is_smartos_zone`, `is_freebsd`, `is_netbsd`,
`is_openbsd`, `is_aix`
* Moves the functions already deprecated by @rallytime to the bottom of
`salt/utils/__init__.py` for better organization, so we can keep the
deprecated ones separate from the ones yet to be deprecated as we
continue to break up `salt.utils`
* Updates `salt/*.py` and all files under `salt/client/` to use explicit
unicode string literals.
* Gets rid of implicit imports of `salt.utils` (e.g. `from salt.utils
import foo` becomes `import salt.utils.foo as foo`).
* Renames the `test.rand_str` function to `test.random_hash` to more
accurately reflect what it does
* Modifies `salt.utils.stringutils.random()` (née `salt.utils.rand_string()`)
such that it returns a string matching the passed size. Previously
this function would get `size` bytes from `os.urandom()`,
base64-encode it, and return the result, which would in most cases not
be equal to the passed size.
Previously, to make these run on Windows, I added the '.py'
extension. For example 'salt-master' => 'salt-master.py'
If this wasn't done, you would get an exception that looks like this
when spawning an addition process:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\salt\bin\lib\multiprocessing\forking.py",
line 380, in main
prepare(preparation_data)
File "C:\salt\bin\lib\multiprocessing\forking.py",
line 489, in prepare
file, path_name, etc = imp.find_module(main_name, dirs)
ImportError: No module named salt-master
Instead of adding the '.py' extension, I found another work-around that
seems to avoid the issue. The details are described in the file comments.
Signed-off-by: Sergey Kizunov <sergey.kizunov@ni.com>
Woot!
The conflicts below are intentional because they contained salt-api-isms
that needed to be included into the corresponding Salt files.
Conflicts:
debian/changelog
debian/control
doc/conf.py
doc/index.rst
opt_requirements.txt
salt/config.py
salt/loader.py
Applying latest changes for packaging
Fixed typos in man pages
Latest round of changes
More packaging fixups
Fixed salt-common typo
Fixed wildcarding in install files
Removed extra man pages
Removed trailing slash
Fixed links
Moved binaries to proper packages
Fixed man pages
Pathing
Perms tweak
Missing files
Fixed spacing
Fixed another lintian error
build the msgpack stuff
Updating rules and install files
Fixed shebang
Control updates
Fixed copyright file
Fixed lintian
Fixed overrides
cmd.retcode no longer uses subprocess.call since it is broken
fix issue with source_hash and trailing whitespace
Bye-bye pickle, hello msgpack
Add docs for new source powers
Add support for source_hash to be a source hash string
add pure python template type
add return clarifying that no states were found if no states are found
change some strings to use format
add code to cache jobs on the minion if option is set
serialize cache data
was caching the wrong line data
Add cache_jobs to the minion config template
add docs for new config param cache_jobs
make the minions return to the master in addition to returning to
returners
Add capability to designate multiple returns
only run the apache module if apachectl is installed
only load solr module if solr is installed
Debug statement used the wrong variable.
Only load nginx on machines that have nginx installed
Make it more like the apache module
- Log a message when the syndic or minion is exiting
- Cleanly exit and use the same message on all daemons
- Add places to handle a sigterm or sighup by KeyboardInterrupt
