This adds some debug logging which we can hopefully use to troubleshoot
https://github.com/saltstack/salt-jenkins/issues/341.
Also, this ensures that the lists of up/down minions are always sorted.
This will make future testing more reliable.
The .keys() function itself is fine, it's just what we do after calling
it that matters. Anywhere we try to index the dictionary view will
stacktrace in PY3.
The third argument that is passed to the salt.utils.cloud.filter_event
function is supposed to be a list. Most of the places calling this function
were using `foo.keys()`, which returns a list in PY2. But, in PY3, this
is a dictionary view.
This change updates the places where `filter_event` is using `.keys()` and
instead wraps the dict with list(). It also adds some extra type checking
in the filter_event function in case a list is not passed in.
This fixes cases where we did not re-parse the status line after a
redirect, causing the cached file to be zero-size as we ignored the
chunks being downloaded.
The code that appended them to the comment predates the warnings field
in state returns. This makes the warnings more uniform with the rest of
the Salt codebase.
tls module: convert bytes to string
gzip util: use BytesIO for 'wb' file
django module: ignore arguments order in assert
pillar module: correctly test pillar merge
Having always alive sessions introduces many issues
when working with legacy equipment. One of them
being that the device can decide by itself to kill
the connection, the sole reasons being lack of activity
on the CLI. (Especially when working with ancient
operating systems such Cisco IOS).
In that case, whenever we detect that the connection
was dropped, we'll try re-establishing the connection.
But it won't go through a loop, will try just once,
if still unable to execute anything, will fail loudly.
When working with SSH-based drivers, the Paramiko
is_alive flag doesn't help much, as the dumb device
doesn't close the connection properly.
When using the `salt` command, errors would occasionally appear that
look like:
```
File "...\salt\transport\tcp.py", line 883, in _stream_return
AttributeError: 'NoneType' object has no attribute 'StreamClosedError'
```
Upon investigation, it was discovered that in this case,
`SaltMessageClient._stream_return()` was completed after all the modules
such as the one containing `tornado.iostream.StreamClosedError` were
unloaded, so at that time, that symbol evaluated to `None` instead of a
valid exception class.
Instead, we would expect `_stream_return` to be completed at the time
that `SaltMessageClient.close()` is invoked. It was noted that
`_stream_return` was not completed when using the `SyncWrapper`, since
that object turns off the IO Loop immediately when its own future
completes, so that the IO Loop is not running when
`SaltMessageClient.close()` is invoked which doesn't give `_stream_return`
a chance to complete. Even when the IO Loop is closed in
`SyncWrapper.__del__()`, `_stream_return` is not completed. In this case,
it only seems to complete when the Python application shuts down which
is when the error appears.
Fix this issue by allowing the IO Loop to run again until the completion
of a specially created future that completes when `_stream_return` is
exiting.
Signed-off-by: Sergey Kizunov <sergey.kizunov@ni.com>
