valitydev/salt
14
0
Fork 0
mirror of https://github.com/valitydev/salt.git synced 2024-11-08 09:23:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #45467 from twangboy/win_exclude_hidden

Browse Source 
Exclude hidden directories in pkg.refresh_db
This commit is contained in:
Nicole Thomas 2018-02-13 11:07:07 -05:00 committed by GitHub
commit 30fb8f7be0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 294 additions and 190 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

390
doc/topics/windows/windows-package-manager.rst
View File

@ -301,25 +301,30 @@ can define multiple versions for the same piece of software. The lines following
the version are indented two more spaces and contain all the information needed
to install that package.
.. warning:: The package name and the ``full_name`` must be unique to all
other packages in the software repository.
.. warning::
The package name and the ``full_name`` must be unique to all other packages
in the software repository.
The version line is the version for the package to be installed. It is used when
you need to install a specific version of a piece of software.
.. warning:: The version must be enclosed in quotes, otherwise the yaml parser
will remove trailing zeros.
.. warning::
The version must be enclosed in quotes, otherwise the yaml parser will
remove trailing zeros.
.. note::
There are unique situations where previous versions are unavailable. Take
Google Chrome for example. There is only one url provided for a standalone
installation of Google Chrome.
.. note:: There are unique situations where previous versions are unavailable.
Take Google Chrome for example. There is only one url provided for a
standalone installation of Google Chrome.
(https://dl.google.com/edgedl/chrome/install/GoogleChromeStandaloneEnterprise.msi)
When a new version is released, the url just points to the new version. To
handle situations such as these, set the version to `latest`. Salt will
install the version of Chrome at the URL and report that version. Here's an
example:
.. code-block:: bash
.. code-block:: yaml
chrome:
latest:
@ -334,200 +339,237 @@ you need to install a specific version of a piece of software.
Available parameters are as follows:
:param str full_name: The Full Name for the software as shown in "Programs and
Features" in the control panel. You can also get this information by
installing the package manually and then running ``pkg.list_pkgs``. Here's
an example of the output from ``pkg.list_pkgs``:
:param str full_name:
The Full Name for the software as shown in "Programs and Features" in the
control panel. You can also get this information by installing the package
manually and then running ``pkg.list_pkgs``. Here's an example of the output
from ``pkg.list_pkgs``:
.. code-block:: bash
.. code-block:: bash
salt 'test-2008' pkg.list_pkgs
test-2008
----------
7-Zip 9.20 (x64 edition):
9.20.00.0
Microsoft .NET Framework 4 Client Profile:
4.0.30319,4.0.30319
Microsoft .NET Framework 4 Extended:
4.0.30319,4.0.30319
Microsoft Visual C++ 2008 Redistributable - x64 9.0.21022:
9.0.21022
Mozilla Firefox 17.0.1 (x86 en-US):
17.0.1
Mozilla Maintenance Service:
17.0.1
NSClient++ (x64):
0.3.8.76
Notepad++:
6.4.2
Salt Minion 0.16.0:
0.16.0
salt 'test-2008' pkg.list_pkgs
test-2008
----------
7-Zip 9.20 (x64 edition):
9.20.00.0
Microsoft .NET Framework 4 Client Profile:
4.0.30319,4.0.30319
Microsoft .NET Framework 4 Extended:
4.0.30319,4.0.30319
Microsoft Visual C++ 2008 Redistributable - x64 9.0.21022:
9.0.21022
Mozilla Firefox 17.0.1 (x86 en-US):
17.0.1
Mozilla Maintenance Service:
17.0.1
NSClient++ (x64):
0.3.8.76
Notepad++:
6.4.2
Salt Minion 0.16.0:
0.16.0
Notice the Full Name for Firefox: Mozilla Firefox 17.0.0 (x86 en-US). That's
exactly what's in the ``full_name`` parameter in the software definition file.
Notice the Full Name for Firefox: ``Mozilla Firefox 17.0.0 (x86 en-US)``.
That's exactly what's in the ``full_name`` parameter in the software
definition file.
If any of the software insalled on the machine matches one of the software
definition files in the repository the full_name will be automatically renamed
to the package name. The example below shows the ``pkg.list_pkgs`` for a
machine that already has Mozilla Firefox 17.0.1 installed.
If any of the software installed on the machine matches one of the software
definition files in the repository, the full_name will be automatically
renamed to the package name. The example below shows the ``pkg.list_pkgs``
for a machine that already has Mozilla Firefox 17.0.1 installed.
.. code-block:: bash
.. code-block:: bash
test-2008:
----------
7zip:
9.20.00.0
Microsoft .NET Framework 4 Client Profile:
4.0.30319,4.0.30319
Microsoft .NET Framework 4 Extended:
4.0.30319,4.0.30319
Microsoft Visual C++ 2008 Redistributable - x64 9.0.21022:
9.0.21022
Mozilla Maintenance Service:
17.0.1
Notepad++:
6.4.2
Salt Minion 0.16.0:
0.16.0
firefox:
17.0.1
nsclient:
0.3.9.328
.. important::
The version number and ``full_name`` need to match the output from
``pkg.list_pkgs`` so that the status can be verified when running a
highstate.
.. note::
It is still possible to successfully install packages using
``pkg.install``, even if the ``full_name`` or the version number don't
match. However, this can make troubleshooting issues difficult, so be
careful.
.. tip::
To force salt to display the full name when there's already an existing
package definition file on the system, you can pass a bogus ``saltenv``
parameter to the command like so: ``pkg.list_pkgs saltenv=NotARealEnv``
:param str installer:
The path to the ``.exe`` or ``.msi`` to use to install the package. This can
be a path or a URL. If it is a URL or a salt path (``salt://``), the package
will be cached locally and then executed. If it is a path to a file on disk
or a file share, it will be executed directly.
.. note::
If storing software in the same location as the winrepo it is best
practice to place each installer in its own directory rather than the
root of winrepo. Then you can place your package definition file in the
same directory. It is best practice to name the file ``init.sls``. This
will be picked up by ``pkg.refresh_db`` and processed properly.
:param str install_flags:
Any flags that need to be passed to the installer to make it perform a
silent install. These can often be found by adding ``/?`` or ``/h`` when
running the installer from the command-line. A great resource for finding
these silent install flags can be found on the WPKG project's wiki_:
.. warning::
Salt will not return if the installer is waiting for user input so it is
imperative that the software package being installed has the ability to
install silently.
:param str uninstaller:
The path to the program used to uninstall this software. This can be the
path to the same `exe` or `msi` used to install the software. It can also be
a GUID. You can find this value in the registry under the following keys:
- Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall
- Software\\Wow6432None\\Microsoft\\Windows\\CurrentVersion\\Uninstall
:param str uninstall_flags:
Any flags that need to be passed to the uninstaller to make it perform a
silent uninstall. These can often be found by adding ``/?`` or ``/h`` when
running the uninstaller from the command-line. A great resource for finding
these silent install flags can be found on the WPKG project's wiki_:
.. warning::
Salt will not return if the uninstaller is waiting for user input so it
is imperative that the software package being uninstalled has the
ability to uninstall silently.
Here are some examples of installer and uninstaller settings:
.. code-block:: yaml
test-2008:
----------
7zip:
9.20.00.0
Microsoft .NET Framework 4 Client Profile:
4.0.30319,4.0.30319
Microsoft .NET Framework 4 Extended:
4.0.30319,4.0.30319
Microsoft Visual C++ 2008 Redistributable - x64 9.0.21022:
9.0.21022
Mozilla Maintenance Service:
17.0.1
Notepad++:
6.4.2
Salt Minion 0.16.0:
0.16.0
firefox:
17.0.1
nsclient:
0.3.9.328
'9.20.00.0':
installer: salt://win/repo/7zip/7z920-x64.msi
full_name: 7-Zip 9.20 (x64 edition)
reboot: False
install_flags: '/qn /norestart'
msiexec: True
uninstaller: '{23170F69-40C1-2702-0920-000001000000}'
uninstall_flags: '/qn /norestart'
.. important:: The version number and ``full_name`` need to match the output
from ``pkg.list_pkgs`` so that the status can be verified when running
highstate.
Alternatively the ``uninstaller`` can also simply repeat the URL of an msi
file:
.. note:: It is still possible to successfully install packages using
``pkg.install`` even if they don't match. This can make troubleshooting
difficult so be careful.
.. code-block:: yaml
:param str installer: The path to the ``.exe`` or ``.msi`` to use to install the
package. This can be a path or a URL. If it is a URL or a salt path
(salt://), the package will be cached locally and then executed. If it is a
path to a file on disk or a file share, it will be executed directly.
7zip:
'9.20.00.0':
installer: salt://win/repo/7zip/7z920-x64.msi
full_name: 7-Zip 9.20 (x64 edition)
reboot: False
install_flags: '/qn /norestart'
msiexec: True
uninstaller: salt://win/repo/7zip/7z920-x64.msi
uninstall_flags: '/qn /norestart'
:param str install_flags: Any flags that need to be passed to the installer to
make it perform a silent install. These can often be found by adding ``/?``
or ``/h`` when running the installer from the command-line. A great resource
for finding these silent install flags can be found on the WPKG project's wiki_:
:param msiexec:
This tells salt to use ``msiexec /i`` to install the package and
``msiexec /x`` to uninstall. This is for ``.msi`` installations. Possible
options are: True, False or the path to ``msiexec.exe`` on your system
Salt will not return if the installer is waiting for user input so these are
important.
.. code-block:: yaml
:param str uninstaller: The path to the program used to uninstall this software.
This can be the path to the same `exe` or `msi` used to install the
software. It can also be a GUID. You can find this value in the registry
under the following keys:
7zip:
'9.20.00.0':
installer: salt://win/repo/7zip/7z920-x64.msi
full_name: 7-Zip 9.20 (x64 edition)
reboot: False
install_flags: '/qn /norestart'
msiexec: 'C:\Windows\System32\msiexec.exe'
uninstaller: salt://win/repo/7zip/7z920-x64.msi
uninstall_flags: '/qn /norestart'
- Software\\Microsoft\\Windows\\CurrentVersion\\Uninstall
- Software\\Wow6432None\\Microsoft\\Windows\\CurrentVersion\\Uninstall
:param bool allusers:
This parameter is specific to ``.msi`` installations. It tells ``msiexec``
to install the software for all users. The default is ``True``.
:param str uninstall_flags: Any flags that need to be passed to the uninstaller
to make it perform a silent uninstall. These can often be found by adding
``/?`` or ``/h`` when running the uninstaller from the command-line. A great
resource for finding these silent install flags can be found on the WPKG
project's wiki_:
:param bool cache_dir:
If ``True`` and the installer URL begins with ``salt://``, the entire
directory where the installer resides will be recursively cached. This is
useful for installers that depend on other files in the same directory for
installation.
Salt will not return if the uninstaller is waiting for user input so these are
important.
Here are some examples of installer and uninstaller settings:
.. code-block:: yaml
7zip:
'9.20.00.0':
installer: salt://win/repo/7zip/7z920-x64.msi
full_name: 7-Zip 9.20 (x64 edition)
reboot: False
install_flags: '/qn /norestart'
msiexec: True
uninstaller: '{23170F69-40C1-2702-0920-000001000000}'
uninstall_flags: '/qn /norestart'
Alternatively the ``uninstaller`` can also simply repeat the URL of the msi file.
.. code-block:: yaml
7zip:
'9.20.00.0':
installer: salt://win/repo/7zip/7z920-x64.msi
full_name: 7-Zip 9.20 (x64 edition)
reboot: False
install_flags: '/qn /norestart'
msiexec: True
uninstaller: salt://win/repo/7zip/7z920-x64.msi
uninstall_flags: '/qn /norestart'
:param msiexec: This tells salt to use ``msiexec /i`` to install the
package and ``msiexec /x`` to uninstall. This is for `.msi` installations.
Possible options are: True, False or path to msiexec on your system
7zip:
'9.20.00.0':
installer: salt://win/repo/7zip/7z920-x64.msi
full_name: 7-Zip 9.20 (x64 edition)
reboot: False
install_flags: '/qn /norestart'
msiexec: 'C:\Windows\System32\msiexec.exe'
uninstaller: salt://win/repo/7zip/7z920-x64.msi
uninstall_flags: '/qn /norestart'
:param str arch: This selects which ``msiexec.exe`` to use. Possible values:
``x86``, ``x64``
:param bool allusers: This parameter is specific to `.msi` installations. It
tells `msiexec` to install the software for all users. The default is True.
:param bool cache_dir: If true when installer URL begins with salt://, the
entire directory where the installer resides will be recursively cached.
This is useful for installers that depend on other files in the same
directory for installation.
.. warning::
Be aware that all files and directories in the same location as the
installer file will be copied down to the minion. If you place your
installer file in the root of winrepo (``/srv/salt/win/repo-ng``) and
``cache_dir: True`` the entire contents of winrepo will be cached to
the minion. Therefore, it is best practice to place your installer files
in a subdirectory if they are to be stored in winrepo.
:param str cache_file:
When installer URL begins with salt://, this indicates single file to copy
down for use with the installer. Copied to the same location as the
installer. Use this over ``cache_dir`` if there are many files in the
When the installer URL begins with ``salt://``, this indicates a single file
to copy down for use with the installer. It is copied to the same location
as the installer. Use this over ``cache_dir`` if there are many files in the
directory and you only need a specific file and don't want to cache
additional files that may reside in the installer directory.
Here's an example for a software package that has dependent files:
Here's an example for a software package that has dependent files:
.. code-block:: yaml
.. code-block:: yaml
sqlexpress:
'12.0.2000.8':
installer: 'salt://win/repo/sqlexpress/setup.exe'
full_name: Microsoft SQL Server 2014 Setup (English)
reboot: False
install_flags: '/ACTION=install /IACCEPTSQLSERVERLICENSETERMS /Q'
cache_dir: True
sqlexpress:
'12.0.2000.8':
installer: 'salt://win/repo/sqlexpress/setup.exe'
full_name: Microsoft SQL Server 2014 Setup (English)
reboot: False
install_flags: '/ACTION=install /IACCEPTSQLSERVERLICENSETERMS /Q'
cache_dir: True
:param bool use_scheduler: If true, windows will use the task scheduler to run
the installation. This is useful for running the salt installation itself as
the installation process kills any currently running instances of salt.
:param bool use_scheduler:
If ``True``, Windows will use the task scheduler to run the installation.
This is useful for running the Salt installation itself as the installation
process kills any currently running instances of Salt.
:param str source_hash: This tells salt to compare a hash sum of the installer
to the provided hash sum before execution. The value can be formatted as
``hash_algorithm=hash_sum``, or it can be a URI to a file containing the hash
sum.
For a list of supported algorithms, see the `hashlib documentation
<https://docs.python.org/2/library/hashlib.html>`_.
:param str source_hash:
This tells Salt to compare a hash sum of the installer to the provided hash
sum before execution. The value can be formatted as
``<hash_algorithm>=<hash_sum>``, or it can be a URI to a file containing the
hash sum.
Here's an example of source_hash usage:
For a list of supported algorithms, see the `hashlib documentation
<https://docs.python.org/2/library/hashlib.html>`_.
.. code-block:: yaml
Here's an example of source_hash usage:
messageanalyzer:
'4.0.7551.0':
full_name: 'Microsoft Message Analyzer'
installer: 'salt://win/repo/messageanalyzer/MessageAnalyzer64.msi'
install_flags: '/quiet /norestart'
uninstaller: '{1CC02C23-8FCD-487E-860C-311EC0A0C933}'
uninstall_flags: '/quiet /norestart'
msiexec: True
source_hash: 'sha1=62875ff451f13b10a8ff988f2943e76a4735d3d4'
.. code-block:: yaml
messageanalyzer:
'4.0.7551.0':
full_name: 'Microsoft Message Analyzer'
installer: 'salt://win/repo/messageanalyzer/MessageAnalyzer64.msi'
install_flags: '/quiet /norestart'
uninstaller: '{1CC02C23-8FCD-487E-860C-311EC0A0C933}'
uninstall_flags: '/quiet /norestart'
msiexec: True
source_hash: 'sha1=62875ff451f13b10a8ff988f2943e76a4735d3d4'
:param bool reboot: Not implemented

94
salt/modules/win_pkg.py
View File

@ -580,28 +580,77 @@ def _refresh_db_conditional(saltenv, **kwargs):
def refresh_db(**kwargs):
'''
Fetches metadata files and calls :py:func:`pkg.genrepo
<salt.modules.win_pkg.genrepo>` to compile updated repository metadata.
r'''
Generates the local software metadata database (`winrepo.p`) on the minion.
The database is stored in a serialized format located by default at the
following location:
`C:\salt\var\cache\salt\minion\files\base\win\repo-ng\winrepo.p`
This module performs the following steps to generate the software metadata
database:
- Fetch the package definition files (.sls) from `winrepo_source_dir`
(default `salt://win/repo-ng`) and cache them in
`<cachedir>\files\<saltenv>\<winrepo_source_dir>`
(default: `C:\salt\var\cache\salt\minion\files\base\win\repo-ng`)
- Call :py:func:`pkg.genrepo <salt.modules.win_pkg.genrepo>` to parse the
package definition files and generate the repository metadata database
file (`winrepo.p`)
- Return the report received from
:py:func:`pkg.genrepo <salt.modules.win_pkg.genrepo>`
The default winrepo directory on the master is `/srv/salt/win/repo-ng`. All
files that end with `.sls` in this and all subdirectories will be used to
generate the repository metadata database (`winrepo.p`).
.. note::
- Hidden directories (directories beginning with '`.`', such as
'`.git`') will be ignored.
.. note::
There is no need to call `pkg.refresh_db` every time you work with the
pkg module. Automatic refresh will occur based on the following minion
configuration settings:
- `winrepo_cache_expire_min`
- `winrepo_cache_expire_max`
However, if the package definition files have changed, as would be the
case if you are developing a new package definition, this function
should be called to ensure the minion has the latest information about
packages available to it.
.. warning::
Directories and files fetched from <winrepo_source_dir>
(`/srv/salt/win/repo-ng`) will be processed in alphabetical order. If
two or more software definition files contain the same name, the last
one processed replaces all data from the files processed before it.
For more information see
:ref:`Windows Software Repository <windows-package-manager>`
Kwargs:
saltenv (str): Salt environment. Default: ``base``
verbose (bool):
Return verbose data structure which includes 'success_list', a list
of all sls files and the package names contained within. Default
'False'
Return a verbose data structure which includes 'success_list', a
list of all sls files and the package names contained within.
Default is 'False'
failhard (bool):
If ``True``, an error will be raised if any repo SLS files failed to
If ``True``, an error will be raised if any repo SLS files fails to
process. If ``False``, no error will be raised, and a dictionary
containing the full results will be returned.
Returns:
dict: A dictionary containing the results of the database refresh.
.. Warning::
.. note::
A result with a `total: 0` generally means that the files are in the
wrong location on the master. Try running the following command on the
minion: `salt-call -l debug pkg.refresh saltenv=base`
.. warning::
When calling this command from a state using `module.run` be sure to
pass `failhard: False`. Otherwise the state will report failure if it
encounters a bad software definition file.
@ -647,10 +696,12 @@ def refresh_db(**kwargs):
)
# Cache repo-ng locally
log.info('Fetching *.sls files from {0}'.format(repo_details.winrepo_source_dir))
__salt__['cp.cache_dir'](
repo_details.winrepo_source_dir,
saltenv,
include_pat='*.sls'
path=repo_details.winrepo_source_dir,
saltenv=saltenv,
include_pat='*.sls',
exclude_pat=r'E@\/\..*?\/' # Exclude all hidden directories (.git)
)
return genrepo(saltenv=saltenv, verbose=verbose, failhard=failhard)
@ -787,6 +838,10 @@ def genrepo(**kwargs):
to process. If ``False``, no error will be raised, and a dictionary
containing the full results will be returned.
.. note::
- Hidden directories (directories beginning with '`.`', such as
'`.git`') will be ignored.
Returns:
dict: A dictionary of the results of the command
@ -810,9 +865,16 @@ def genrepo(**kwargs):
repo_details = _get_repo_details(saltenv)
for root, _, files in os.walk(repo_details.local_dest, followlinks=False):
# Skip hidden directories (.git)
if re.search(r'[\\/]\..*', root):
log.debug('Skipping files in directory: {0}'.format(root))
continue
short_path = os.path.relpath(root, repo_details.local_dest)
if short_path == '.':
short_path = ''
for name in files:
if name.endswith('.sls'):
total_files_processed += 1
@ -1243,11 +1305,11 @@ def install(name=None, refresh=False, pkgs=None, **kwargs):
# single files
if cache_dir and installer.startswith('salt:'):
path, _ = os.path.split(installer)
__salt__['cp.cache_dir'](path,
saltenv,
False,
None,
'E@init.sls$')
__salt__['cp.cache_dir'](path=path,
saltenv=saltenv,
include_empty=False,
include_pat=None,
exclude_pat='E@init.sls$')
# Check to see if the cache_file is cached... if passed
if cache_file and cache_file.startswith('salt:'):