salt/doc/topics/development/labels.rst

.. _labels-and-milestones:

============================
GitHub Labels and Milestones
============================

SaltStack uses several labeling schemes, as well as applying milestones, to triage incoming issues and pull requests in
the GitHub Issue Tracker. Most of the labels and milestones are used for internal tracking, but the following
definitions might prove useful for the community to discover the best issues to help resolve.

Milestones
==========

Milestones are most often applied to issues, as a milestone is assigned to every issue that has been triaged. However,
milestones can also be applied to pull requests. SaltStack uses milestones to track bugs or features that should be
included in the next major feature release, or even the next bug-fix release, as well as what issues are ready to be
worked on or what might be blocked. All incoming issues must have a milestone associated with them.

Approved
    Used to indicate that this issue has all of the needed information and is ready to be worked on.

Blocked
    Used to indicate that the issue is not ready to be worked on yet. This typically applies to issues that have been
    labeled with “Info Needed”, “Question”, “Expected Behavior”, “Won’t Fix for Now”, etc.

Dot or Bug-fix Release
    Used to help filter/identify what issues must be fixed before the release such as 2014.7.4 or 2015.2.3. This
    milestone is often used in conjunction with the ``Blocker`` label, but not always.

Feature Release
    Similar to the Dot or Bug-fix Release milestone, but for upcoming feature releases such as Boron, Carbon, etc.
    This milestone is often used in conjunction with the ``Blocker`` label, but not always.

Labels
======

Labels are used to facilitate the resolution of new pull requests and open issues. Most labels are confined to being
applied to either issues or pull requests, though some labels may be applied to both.

Issue Labels
------------

All incoming issues should be triaged with at least one label and a milestone. When a new issue comes in, it should be
determined if the issue is a bug or a feature request, and either of those labels should be applied accordingly. Bugs
and Feature Requests have differing labeling schemes, detailed below, where other labels are applied to them to further
help contributors find issues to fix or implement.

There are some labels, such as ``Question`` or some of the "Status" labels that may be applied as "stand alone" labels
in which more information may be needed or a decision must be reached on how to proceed. (See the "Bug Status Labels"
section below.)

Features
~~~~~~~~

The ``Feature`` label should be applied when a user is requesting entirely new functionality. This can include new
functions, modules, states, modular systems, flags for existing functions, etc. Features *do not* receive severity
or priority labels, as those labels are only used for bugs. However, they may receive "Functional Area" labels or "ZD".

Feature request issues will be prioritized on an "as-needed" basis using milestones during SaltStack's feature release
and sprint planning processes.

Bugs
~~~~

All bugs should have the ``Bug`` label as well as a severity, priority, functional area, and a status, as applicable.

Severity
^^^^^^^^

How severe is the bug? SaltStack uses four labels to determine the severity of a bug: ``Blocker``, ``Critical``,
``High``, and ``Medium``. This scale is intended to make the bug-triage process as objective as possible.

Blocker
    Should be used sparingly to indicate must-have fixes for the impending release.

Critical
    Applied to bugs that have data loss, crashes, hanging, unresponsive system, etc.

High Severity
    Any bug report that contains incorrect functionality, bad functionality, a confusing user experience, etc.

Medium Severity
    Applied to bugs that are about cosmetic items, spelling, spacing, colors, etc.

Priority
^^^^^^^^

In addition to using a bug severity to classify issues, a priority is also assigned to each bug to give further
granularity in searching for bugs to fix. In this way, a bug's priority is defined as follows:

P1
    Very likely. Everyone will see the bug.

P2
    Somewhat likely. Most will see the bug, but a few will not.

P3
    Half will see the bug, about half will not.

P4
    Most will not see the bug. Usually a very specific use case or corner case.

.. note::

    A bug's priority is relative to its functional area. If a bug report, for example, about ``gitfs`` includes details
    indicating that everyone who ``gitfs`` will run into this bug, then a ``P1`` label will be applied, even though
    Salt users who are not enabling ``gitfs`` will see the bug.

Functional Areas
^^^^^^^^^^^^^^^^

All bugs should receive a "Functional Area" label to indicate what region of Salt the bug is mainly seen in. This will
help internal developers as well as community members identify areas of expertise to find issues that can be fixed more
easily. Functional Area labels can also be applied to Feature Requests.

Functional Area Labels, in alphabetical order, include:

* Core
* Documentation
* Execution Module
* File Servers
* Multi-Master
* Packaging
* Pillar
* Platform Mgmt.
* RAET
* Returners
* Salt-API
* Salt-Cloud
* Salt-SSH
* Salt-Syndic
* State Module
* Windows
* ZMQ

Bug Status Labels
^^^^^^^^^^^^^^^^^

Status lables are used to define and track the state a bug is in at any given time. Not all bugs will have a status
label, but if a SaltStack employee is able to apply a status label, he or she will. Status labels are somewhat unique
in the fact that they might be the only label on an issue, such as ``Pending Discussion``, ``Info Needed``, or
``Expected Behavior`` until further action can be taken.

Cannot Reproduce
    Someone from the SaltStack team has tried to reproduce the bug with the given information but they are unable to
    replicate the problem. More information will need to be provided from the original issue-filer before proceeding.

Confirmed
    A SaltStack engineer has confirmed the reported bug and provided a simple way to reproduce the failure.

Duplicate
    The issue has been reported already in another report. A link to the other bug report must be provided. At that
    point the new issue can be closed. Usually, the earliest bug on file is kept as that typically has the most
    discussion revolving around the issue, though not always. (This can be a "stand-alone" label.)

Expected Behavior
    The issue reported is expected behavior and nothing needs to be fixed. (This can be a "stand-alone" label.)

Fixed Pending Verification
    The bug has been fixed and a link to the applicable pull request(s) has been provided, but confirmation is being
    sought from the community member(s) involved in the bug to test and confirm the fix.

Info Needed
    More information about the issue is needed before proceeding such as a versions report, a sample state, the command
    the user was running, or the operating system the error was occurring on, etc. (This can be a "stand-alone" label.)

Upstream Bug
    The reported bug is something that cannot be fixed in the Salt code base but is instead a bug in another library
    such a bug in ZMQ or Python. When an issue is labeled with ``Upstream Bug`` then a bug report in the upstream
    project must be filed (or found if a report already exists) and a link to the report must be provided to the issue
    in Salt for tracking purposes. (This can be a stand-alone label.)

Won't Fix for Now
    The SaltStack team has acknowledged the issue at hand is legitimate, but made the call that it’s not something
    they’re able or willing to fix at this time. These issues may be revisited in the future.

Other
~~~~~

There are a couple of other labels that are helpful in categorizing bugs that are not included in the categories above.
These labels can either stand on their own such as ``Question`` or can be applied to bugs or feature requests as
applicable.

Low Hanging Fruit
    Applied to bugs that should be easy to fix. This is useful for new contributors to know where some simple things
    are to get involved in contributing to salt.

Question
    Used when the issue isn’t a bug nor a feature, but the user has a question about expected behavior, how something
    works, is misunderstanding a concept, etc. This label is typically applied on its own with ``Blocked`` milestone.

Regression
    Helps with additional filtering for bug fixing. If something previously worked and now does not work, as opposed to
    something that never worked in the first place, the issue should be treated with greater urgency.

ZD
    Stands for “Zendesk” and is used to help track bugs that customers are seeing as well as community members. Bugs
    with this label should be treated with greater urgency.

Pull Request Labels
-------------------

SaltStack also applies various labels to incoming pull requests. These are mainly used to help SaltStack engineers
easily identify the nature the changes presented in a pull request and whether or not that pull request is ready to be
reviewed and merged into the Salt codebase.

Type of Change
~~~~~~~~~~~~~~

A "* Change" label is applied to each incoming pull request. The type of change label that is applied to a pull request
is based on a scale that encompasses the number of lines affected by the change in conjunction with the area of code
the change touches. For example, does the pull request change code in an execution or state module? Or is the change
affecting code in a core area of Salt such as in the transport, state compiler, or crypt files? These labels indicate
to the Salt engineering staff who should be reviewing and either discussing the changes with the contributor or merging
the changes into the codebase.

The change labels are listed in order of smallest/least critical code area change to largest/most critical area:

* Minor Change
* Medium Change
* Master Change
* Expert Change

Back-port Labels
~~~~~~~~~~~~~~~~

There are two labels that are used to keep track of what pull requests need to be back-ported to an older release branch
and which pull requests have already been back-ported.

Bugfix - back-port
    Indicates a pull request that needs to be back-ported. Once the back-port is completed, the back-porting pull request
    is linked to the original pull request and this label is removed.

Bugfix - [Done] back-ported
    Indicates a pull request that has been back-ported to another branch. The pull request that is responsible for the
    backport should be linked to this original pull request.

Testing Labels
~~~~~~~~~~~~~~

There are a couple of labels that the QA team uses to indicate the mergability of a pull request. If the pull request is
legitimately passing or failing tests, then one or more of these labels may be applied.

Lint
    If a pull request fails the test run, but the only failures are related pylint errors, this label will be applied to
    indicate that pylint needs to be fixed before proceeding.

Pending Changes
    Indicates that additional commits should be added to the original pull request before the pull request is merged
    into the codebase. These changes are unrelated to fixing tests and are generally needed to round out any unfinished
    pull requests.

Tests Passed
    Sometimes the Jenkins test run encounters problems, either tests that are known to have reliability issues or a
    test VM failed to build, but the problems are not related to the code changed in the pull request. This label is
    used to indicate that someone has reviewed the test failures and has deemed the failures to be non-pertinent.

Other Pull Request Labels
~~~~~~~~~~~~~~~~~~~~~~~~~

Awesome
    Applied to pull requests that implemented a cool new feature or fixed a bug in an excellent way.

Labels that Bridge Issues and Pull Requests
===========================================

Needs Testcase
    Used by SaltStack's QA team to realize where pain points are and to bring special attention to where some test
    coverage needs to occur, especially in areas that have regressed. This label can apply to issues or pull requests,
    which can also be open or closed. Once tests are written, the pull request containing the tests should be linked to
    the issue or pull request that originally had the ``Needs Testcase`` label. At this point, the ``Needs Testcase``
    label must be removed to indicate that tests no longer need to be written.

Pending Discussion
    If this label is applied to an issue, the issue may or may not be a bug. Enough information was provided about the
    issue, but some other opinions on the issue are desirable before proceeding. (This can be a "stand-alone" label.)
    If the label is applied to a pull request, this is used to signal that further discussion must occur before a
    decision is made to either merge the pull request into the code base or to close it all together.