Fix this exception which occurs in the exception handler
inside fire_args:
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/salt/minion.py", line 1493, in _thread_return
return_data = executor.execute()
File "/usr/lib64/python2.7/site-packages/salt/executors/direct_call.py", line 28, in execute
return self.func(*self.args, **self.kwargs)
File "/usr/lib64/python2.7/site-packages/salt/modules/saltutil.py", line 1437, in runner
prefix='run'
File "/usr/lib64/python2.7/site-packages/salt/utils/event.py", line 181, in fire_args
tag, tag_data, exc
UnboundLocalError: local variable 'tag' referenced before assignment
Fixes: https://github.com/saltstack/salt/issues/46120
The following code in fire_args can put at integer in the
suffix list:
tag_suffix = [jid, 'args']
Therefore, fix tagify to convert the integer to a string,
in order to avoid the following error:
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/salt/minion.py", line 1493, in _thread_return
return_data = executor.execute()
File "/usr/lib64/python2.7/site-packages/salt/executors/direct_call.py", line 28, in execute
return self.func(*self.args, **self.kwargs)
File "/usr/lib64/python2.7/site-packages/salt/modules/saltutil.py", line 1437, in runner
prefix='run'
File "/usr/lib64/python2.7/site-packages/salt/utils/event.py", line 175, in fire_args
tag = tagify(tag_suffix, prefix)
File "/usr/lib64/python2.7/site-packages/salt/utils/event.py", line 203, in tagify
return TAGPARTER.join([part for part in parts if part])
TypeError: sequence item 2: expected string, long found
`fnmatch.fnmatch()` does exact matches, so checking for an exact match
is an unnecessary extra step.
Additionally, while regex errors are caught, TypeErrors are not, so the
use of non-string expressions/values will result in a traceback. This
prevents those tracebacks.
These were initially marked for removal in "Fluorine", but we need
to give users 2 feature releases to update. The version that the
options will be removed in is "Neon". The related references have
been updated, as well as the relevant documentation.
Even when `multiprocessing` is set to `True`, there is a case where
multiple threads in the same process attempt to use the same LazyLoader
object. When using the reactor and reacting to an event that will call a
runner, `salt.utils.reactor.ReactWrap.runner` will invoke
`self.pool.fire_async(self.client_cache['runner'].low, args=(fun, kwargs))`
potentially multiple times, each time using a thread from
`salt.utils.process.ThreadPool`. Each thread will invoke
`salt.client.mixins.SyncClientMixin.low` which in turn will invoke its
`_low` and call `salt.utils.job.store_job`. `salt.utils.job.store_job`
will invoke the LazyLoader object for the returner.
Since the LazyLoader object is not thread safe, occasional failures will
occur which will reduce the reliability of the overall system.
Let's examine why a function such as `LazyLoader._load` is not thread safe.
Any time the GIL is released, it allows another thread to run. There are
various types of operations that could release the GIL, but in this
particular case they are file operations that happen in both
`refresh_file_mapping` and `_load_module`. Note that if you add `print`
statements, those also release the GIL (and make the problem more
frequent). In the failure case, `refresh_file_mapping` releases the
GIL, another thread loads the module, and then when the original thread
runs again it will fail when `_inner_load` runs the second time (after
`refresh_file_mapping`). The failure is because the module is already in
`self.loaded_files`, so it is skipped over and `_inner_load` returns
`False` even though the required `key` is already in `self._dict`. Since
adding in stuff like `print` statements, or other logic also adds points in
the code that allow thread switches, the most robust solution to such a
problem is to use a mutex (as opposed to rechecking if `key` now appears
in `self._dict` at certain checkpoints).
This solution adds such a mutex and uses it in key places to ensure
integrity.
Signed-off-by: Sergey Kizunov <sergey.kizunov@ni.com>
