Commit Graph

95314 Commits

Author SHA1 Message Date
Erik Johnson
baf1653221
Fix module name in title 2018-06-26 14:00:23 -05:00
Erik Johnson
17f7b18da1
Sort and update mocks in doc configuration 2018-06-26 14:00:02 -05:00
Nicole Thomas
0bb4ece4b6
Merge pull request #48290 from mirceaulinic/napalm-proxyless
Enhance the proxy_napalm_wrap decorator to allow "proxyless" execution
2018-06-26 09:50:31 -04:00
Nicole Thomas
c32b0e62ed
Merge pull request #48289 from isbm/isbm-ssh-tornado-concurrent-futures
Include missing module import for Salt SSH on Python 2.x
2018-06-26 09:47:54 -04:00
Nicole Thomas
6d5e6e7eb5
Merge pull request #48261 from cbosdo/virt-init
Virt init improvements
2018-06-26 09:41:38 -04:00
Mircea Ulinic
bf7baaebe2 Use copy.deepcopy instead of just dict.copy 2018-06-26 07:27:40 +00:00
Nicole Thomas
e5ef525a8c
Merge pull request #48180 from mtorromeo/fix-ini-no-section
Uniformed reports between ini.options_absent and ini.options_present
2018-06-25 16:31:13 -04:00
Nicole Thomas
906c7e80be
Merge pull request #48220 from mnieber/patch-1
Explain how to use a grain in a state
2018-06-25 16:29:17 -04:00
Nicole Thomas
1f5c6ee13a
Merge pull request #48235 from ixs/more_gpus
Update core.py grains to add Matrox and ASPEED.
2018-06-25 16:19:28 -04:00
Nicole Thomas
e0d76b0e64
Merge pull request #48264 from mirceaulinic/napalm-netmiko-gate
Adding a few helpers to gate functionality from netmiko into the NAPALM proxy minions
2018-06-25 16:14:16 -04:00
Nicole Thomas
7466f01f50
Merge pull request #48282 from sjorge/smartosdocker
The smartos state's vm_present should support docker
2018-06-25 16:12:56 -04:00
Nicole Thomas
b44f0f1d0f
Merge pull request #48291 from isbm/isbm-mask-rendered-data
Mask rendered data (in logs)
2018-06-25 16:10:59 -04:00
Daniel Wallace
1f50738d58
Merge pull request #48215 from KaiSforza/testlintnotify
Notify during tests using different credentials
2018-06-25 10:04:20 -05:00
Bo Maryniuk
36ffef5a69 Fix unit tests 2018-06-25 16:54:31 +02:00
Cédric Bosdonnat
d7c4f3d709
Virt disk profile handles pool for kvm
So far _disk_profile uses the pool property only for vmware
hypervisors. Use it also for KVM/QEMU for users to be able to choose
where to create their images.

The pool property can be either the path to a local folder or an storage
pool name.
2018-06-25 16:53:25 +02:00
Cédric Bosdonnat
f68d90ef4f
Add target path and type to virt.pool_info result
At times we need to know where the storage pool is pointing to and of
what type it is. virt.pool_info is a nice place for this.
2018-06-25 16:53:25 +02:00
Cédric Bosdonnat
ac6868a532
Simplify virt._disk_profile return value
Rather than returning a list of {name: dict_of_disk_props}, return
include the name as a property in each disk dictionary and remove one
level. This aligns disks with the structure returned for networks and
allows usage of list comrehensions to slightly simplify the calling code.
2018-06-25 16:53:25 +02:00
Cédric Bosdonnat
d6f0c59bc3
Don't create subfolders within virt:images
Disk images are currently created at a patch matching this rule:

    <virt:images>/<vmname>/<diskname>

In the future we want the user to be able to define in which libvirt
pool to create the image, rather than always in the default virt:images
folder. As libvirt doesn't allow volume names with a / in them, create
the qemu disk images in:

    <virt:images>/<vmname>_<diskname>
2018-06-25 16:53:25 +02:00
Cédric Bosdonnat
5479f88855
virt: move all disks computations in _disk_profile
Disk profile structure wasn't containing the image filename and path.
These were computed in two different places: one in _qemu_create_image()
and one in _gen_xml.

This commit moves all disks list computations in _disk_profile to get:
 - default values on both disk profile and user disks definitions
 - one place to compute all values
This should reduce error risks in future disk-related changes.
2018-06-25 16:53:25 +02:00
Cédric Bosdonnat
614a213b3a
Fix virt images default location in docs
virt images are not stored in /srv/salt/salt-images, but in
/srv/salt-images. Fix the documentation to reflect the truth.
2018-06-25 16:53:24 +02:00
Cédric Bosdonnat
b128f019b3
virt.init: move enable_qcow to disks parameter
enable_qcow is rather badly named since it doesn't tell the user what
that actually does. Thanks to the new disks parameter, this option can
now be set on a per-disk basis in the disks structure using a new
overlay_image property.

enable_qcow is now marked as deprecated
2018-06-25 16:53:24 +02:00
Cédric Bosdonnat
7555172e59
Allow overriding disks profile in virt.init
virt.init just allows the user to pass in a profile name to get the
list of disk devices to create. This is rather good for simple
cases, but requires the profile to be stored in the minion
configuration.

From that commit on, the user can use a disks parameter to
customize the disks from the template or add other ones. This could be
handy for the virt.running state for example.

This new parameter makes the image parameter useless: deprecating it.
2018-06-25 16:53:24 +02:00
Cédric Bosdonnat
535942ace2
Allow overriding NICs profile in virt.init
virt.init just allows the user to pass in a profile name to get the
list of network interfaces to create. This is rather good for simple
cases, but requires the profile to be stored in the minion
configuration.

From that commit on, the user can use an interfaces parameter to
customize the NICs from the template or add other ones. This could be
handy for the virt.running state for example.
2018-06-25 16:53:24 +02:00
Cédric Bosdonnat
0ac324acff
Support more display types in virt.init
virt.init's enable_vnc is too limiting for the current possibilities of
the libvirt stack. Deprecate it in favor of a graphics dictionary to
allow creating VMs with VNC, Spice, RDP or no graphics. This design
helps keeping the structure opened to support new parameters in the
future.
2018-06-25 16:53:24 +02:00
Cédric Bosdonnat
34d1b34fed
Finish removing the use of libvirt:hypervisor
Storing the hypervisor type in a configuration value doesn't make sense
since the libvirt host tells us what it is capable of. Instead use
the values from the guest domains provided by the virt.capabilities.

This is also the occasion to remove the use of the 'esxi' hypervisor in
as many places as possible since this is a synonym of 'vmware' and
'vmware' is the value provided by the libvirt esx driver.
2018-06-25 16:53:23 +02:00
Cédric Bosdonnat
6659e7beae
virt.get_disks provides complete backing chain
As mentioned in issue #48085, if a disk image has a backing file,
_parse_qemu_img_info() only returns the path to the backing file.
From a user point of view this is rather limited since there is no way
to know more on that file.

virt.get_disks now uses --backing-chain parameter to get the disk data
for all disks in the backing chain.
2018-06-25 16:53:23 +02:00
Cédric Bosdonnat
489b30c770
Fix qemu-img infos output parsing
As mentioned in issue #48085 qemu-img infos parsing is really fragile.
In order to fix the weaknesses of that parsing use the --output json
parameter of qemu-img infos and parse this rather than the human output.

Note that the following properties in virt.get_disks output have a
different format:

  * disk size, virtual size and snapshot vmsize are now in bytes, rather
    than in a human-friendly format
  * date is now complete and in iso format, but that won't bother
    anybody since that was broken before (only had the time part)

Since the image property was a duplicate of the file one, they have been
consolidated into a single file property.

All format-specific values are simply not provided, but as those were
in the snapshots list before it's rather likely no one will care.

Also write the parsed data into a dictionary rather than writing it as
YAML in a string and parse it later on.

This commit also adds a unit test for _parse_qemu_img_info() function.
2018-06-25 16:53:23 +02:00
Cédric Bosdonnat
256a123108
Refactoring virt.get_disks
virt.get_disks does not need to depend on libvirt:hypervisor value to
decice whether to extract data using qemu-img info on a disk image.
This needs to be run on all qemu images... and those can be also used
by a Xen VM nowadays.

The refactoring removes that dependency on the deprecated configuration
libvirt:hypervisor and uses the value for the <driver> type attribute
in the disk XML configuration.

Enhance the get_disk unit test to make sure that the qemu-img info is
parsed for a qemu image and not for others
2018-06-25 16:53:23 +02:00
Nicole Thomas
1cee5ec014
Merge pull request #48253 from steveno/add_file_grep_unit_tests
Add unit tests for file.grep
2018-06-25 10:31:08 -04:00
Bo Maryniuk
cb24c13bc6 Docstring fix for semantic meaning 2018-06-25 16:24:55 +02:00
Bo Maryniuk
a6a9bc6f97 rephrase the docstring 2018-06-25 16:15:34 +02:00
Nicole Thomas
9382834f14
Merge pull request #48269 from rallytime/stale-config-update
Reduce the number of days an issue is stale by 10
2018-06-25 10:08:38 -04:00
Nicole Thomas
210d558615
Merge pull request #48260 from mirceaulinic/netmiko-proxy
Adding the initial version of the Netmiko Proxy and Execution modules
2018-06-25 10:02:28 -04:00
Bo Maryniuk
b167b5c833 Prevent possible UnicodeDecodeError once key is non-ascii 2018-06-25 15:54:04 +02:00
Nicole Thomas
57bd7d8f17
Merge pull request #48288 from rmarchei/lsattr
fix lsattr.get in file.check_perms
2018-06-25 09:34:09 -04:00
Daniel Wallace
3932c7da06
Merge pull request #47637 from alexvasiu/dev/avasiu/nilrt_get_all_interfaces
modules/nilrt_ip.py: Fixed get_interfaces_details
2018-06-25 08:28:08 -05:00
Nicole Thomas
422aded3f6
Merge pull request #48284 from alexandruavadanii/libvirt-undefine-nvram
virt module: Allow NVRAM unlinking on DOM undefine
2018-06-25 09:27:16 -04:00
Bo Maryniuk
36cacdb49e Add unit test for ignoring masking once syntax isn't YAML's key/value set 2018-06-25 15:24:24 +02:00
Bo Maryniuk
794a233a70 Add unit test for masking key:value of YAML 2018-06-25 15:24:03 +02:00
Nicole Thomas
3d5739f8a9
Merge pull request #48214 from pm17788/develop
Add 'uuid', 'on_reboot', 'on_restart', and 'on_crash' settings to the…
2018-06-25 09:20:54 -04:00
Bo Maryniuk
0f7d17dd76 Use masking for Salt SSH to hide passwords in the debugging logs 2018-06-25 15:18:21 +02:00
Bo Maryniuk
4548ed5ad0 Mask sensitive data in the logs for the debug purposes 2018-06-25 15:18:00 +02:00
Bo Maryniuk
3ae8c16364 Update docstring 2018-06-25 15:17:42 +02:00
Bo Maryniuk
2336f51435 Implement a data masking function 2018-06-25 15:12:54 +02:00
Mircea Ulinic
243ba9c297 Adding a few helpers to gate functionality from netmiko into the napalm proxy minions
``netmiko`` is already a dependency of ``napalm``, therefore we are able to
expose all the ``netmiko`` functionalities from the ``netmiko`` execution
module without having to run under a netmiko Proxy Minion: see
https://github.com/saltstack/salt/pull/48260, which is the PR adding the netmiko
execution module.

These new functions added to the existing napalm module gate netmiko's features
to be reused into the napalm Proxy Minions, by forwarding the authentication
details and options which are already there.

For example, the following function goes through the ``netmiko`` Execution
Module, and napalm users can invoke it straight away to get direct access to
basic SSH primitives: ``salt '*' napalm.netmiko_call send_command 'show version'``.
(without any further configuration needed).
2018-06-25 11:51:03 +01:00
Mircea Ulinic
50457a5f76 Add the possibility to have the credentials for all devices into napalm_inventory
This is going to ease the CLI usage, so we can reduce everything to simply
specifying the host of the device, e.g.,

Having the following configuration in the opts/pillar:

```yaml
napalm:
  username: salt
  password: test

napalm_inventory:
  1.2.3.4:
    driver: eos
  edge01.bzr01:
    driver: junos
```

With the above available in the opts/pillar for a Minion, say ``server1``, the
user would be able to execute: ``salt 'server1' bgp.neighbors host=1.2.3.4`` or
``salt 'server1' net.arp host=edge01.bzr01``, etc.
2018-06-25 11:19:11 +01:00
Mircea Ulinic
7612f4524a Enhance the proxy_napalm_wrap decorator to allow proxyless execution
Usually the Salt proxies have been designed using a single methodology: for
each device you aim to manage, start one Proxy Minion. The NAPALM modules didn't
make an exception, however, beginning with https://github.com/saltstack/salt/pull/38339
(therefore starting with release Nitrogen), the functionality has been enhanced
in such a way that we can execute the code when we are able to install the
regular Minion directly on the network hardware.

There is another use case, for something, let's call it "proxyless" when we
don't particularly need to start a Proxy process for each device, but rather
simply invoke arbitrary functions. These changes make this possible, so with one
single Minion (whether Proxy or regular), one is able to execute Salt commands
going through the NAPALM library to connect to the remote network device by
specifying the connection details from the command line and/or opts/pillar, e.g.
``salt server1 bgp.neighbors driver=junos host=1.2.3.4 username=salt``.

If the ``server1`` Minion has the following block in the Pillar (for example):

```yaml
napalm:
  driver: junos
  username: salt
```

The user would only need to provide the rest of the credentials (depending on
each individual case):

``salt server1 bgp.neighbors host=1.2.3.4``.
2018-06-25 10:25:41 +01:00
Bo Maryniuk
cca8b2fa3e Include missing module import for Salt SSH on Python 2.x 2018-06-25 10:39:18 +02:00
Vasiu Alexandru
1b541a92fe
Merge branch 'develop' into dev/avasiu/nilrt_get_all_interfaces 2018-06-25 10:31:39 +03:00
Ruggero Marchei
efb3286eca fix lsattr.get in file.check_perms 2018-06-25 00:00:35 +02:00