diff --git a/docs/Contributing/Fleet-UI-Testing.md b/docs/Contributing/Fleet-UI-Testing.md new file mode 100644 index 000000000..564d0700d --- /dev/null +++ b/docs/Contributing/Fleet-UI-Testing.md @@ -0,0 +1,374 @@ +# Fleet UI testing + +This document contains the testing strategy and plan for the frontend codebase. The testing +strategy is a high-level overview of the **who**, **what**, **when** and **why** when it comes to testing. +The testing plan primarily outlines the **how** of testing and covers the different practices and +toolings used in testing. + +For instructions on using our testings tools, check out our [testing docs](../Contributing/Testing-and-local-development.md). + +**Table of contents** + +- [Testing strategy](#testing-strategy) + - [Testing philosophy](#testing-philosophy) + - [Who tests](#who-tests) + - [What we test](#what-we-test) + - [Shared utilities](#shared-utilities) + - [UI building blocks](#ui-building-blocks-resusable-components-and-hooks) + - [App widgets](#app-widgets) + - [User journeys](#user-journeys) +- [Testing plan](#testing-plan) + - [Types of tests](#types-of-tests) + - [Testing cheat sheet](#testing-cheat-sheet) + - [Manual testing](#manual-testing) + - [Static analysis](#static-analysis) + - [Unit testing](#unit-testing) + - [Shared utilities testing](#shared-utilities-testing) + - [UI building block testing](#ui-building-block-testing) + - [Reusable hook testing](#reusable-hooks-testing) + - [Integration testing](#integration-testing) + - [App widgets testing](#app-widget-testing) + - [E2E testing](#e2e-testing) + - [Tooling](#tooling) + - [ESLint and TypeScript](#eslint-and-typescript) + - [Jest](#jest) + - [Testing library](#testing-library) + - [Cypress](#cypress) + - [Additional examples](#additional-examples) + - [Roles and permissions](#roles-and-permissions) + - [Mac and Windows hosts](#mac-and-windows-hosts) + - [Error states](#error-states) + +--- + +## Testing strategy + +### Testing philosophy + +When we create tests, we keep in mind how an end user will be using this software. This idea +influences all other decisions when it comes to testing. Sometimes the end user is a company admin +using the Fleet UI, or a DevOps engineer using fleetctl in the terminal, or a developer using a +reusable UI component or utility or class. In any case, we should first think about the end user +when building our testing plan. Testing software from this perspective has many advantages, +including: + +- A focus on functionality and behavior over implementation details. This leads to better + maintainability of the testing suite which does not have to change as the implementation changes. +- A clear idea of what type of tests are useful and should be prioritized. +- A higher level of confidence that the software behaves as intended in real-world scenarios. + +### Who tests + +The **developer** is responsible for writing and maintaining tests as part of their work. They can get +help from QA when trying to decide what tests are sufficient for the feature. + +### What we test + +We break down our application into different types of systems that we can test in ways that best fit +their use cases. + +#### Shared utilities + +Shared utilities are generally simple JS functions used throughout the app. This includes code in the +`utilities` directory such as `utilites/url` and `utilities/string`. + +#### UI building blocks (reusable components and hooks) + +UI building blocks is reusable code that's primarily driven by props or arguments. This +includes components in the `components` directory like `Radio`, `FlashMessage`, and `Button` or +reusable hooks like `useDeepEffect`. + +#### App widgets + +App widgets are larger chunks of the application that are more specific (therefore not +reused) and are made up of the UI building blocks. They tend to have more context and dependencies +within them, which means they're tested differently than the building blocks. Examples include forms +like `LoginForm` and `ResetPasswordForm` or even simpler pages like `Registration Page` or `LoginPage` + +#### User journeys + +User journeys are the flows users take through the application to accomplish their goals. These are +typically the widest and can include navigating through multiple pages or working with multiple app +widgets on a page. This would include goals like **creating a new user or team** or **filtering +hosts by software vulnerabilities or policy results**. + +--- + +## Testing plan + +This section answers **how** we are testing our code by covering our testing tools and best practices. We +like to test the software at different layers (unit, integration, E2E), all of which have their +own usefulness and testing practices. + +> NOTE: Architecture plays a huge role in testing practices and tools. In our current landscape +(__as of 08/31/2022__), we do not have the best separation of concerns between our systems. As a +result, the most reliable way to test our software is as a whole with E2E tests. While this is ok +at some level, we are still missing a large chunk of important testing that would be better written +and maintained at the integration/unit level. We'd like to utilize separation of concerns more +between our systems as this will allow us to test more in the integration and unit layers, which are +quicker to run and generally easier to work with. + +### Types of tests + +We use a variety of testing to ensure that our software is working as intended. This includes: + +- End-to-end (E2E) +- Integration +- Unit +- Static +- Manual + +#### Testing cheat sheet + +| Systems | Type of Test | Who is the User | Example | Notes | +| ------- | ------------ | --------------- | ------- | ----- | +| Reusable utilities | Unit with Jest | Devs | String util, url util | Little to no dependencies. Function argument-based. | +| Reusable hooks | Unit with Jest & react-hooks Library | Devs | useToggleDisplayed Hook | Little to no dependencies. Function argument-based. | +| Reusable UI components | Unit with Jest & testing-library | Devs | Radio, button, and input components | Little to no dependencies. Props-based. | +| App widgets | Integration or E2E with testing-library or Cypress | End users | Create user form. Reset password form. | Less reusable code with more complex environment setup and dependencies. Depending on the case, can be done with integration or E2E. For integration, mock at the backend level; don't mock other UI systems. For E2E, don't mock other systems except for common network error states. | +| User journeys | E2E with Cypress | End users | Filtering a host by software. Creating a team as admin. | Full business flows. Little to no mocking of systems, except for common network error states. | +| N/A | Manual | N/A | // TODO | Manual testing can be used for all types of code. Examples would be for one-offs or states that would require extremely difficult testing setups. | + +#### Manual testing + +There will always be a space for manual testing. We use it for testing states that do not +occur very often in the application or are not worth the effort to test for unlikely edge cases. + +#### Static analysis + +This includes typing and linting to quickly ensure proper typings of data flowing through the +application, and that we are following coding conventions and styling rules. This gives us a first +line of defense against writing buggy code. + +#### Unit testing + +We unit test smaller reusable components that have little to no dependencies within them. These +components are primarily parametric-based and require no or minimal mocking to be tested +effectively. They tend to be small building blocks of our application (e.g., reusable UI components, +common utilities, reusable hooks). With unit testing, the end users tend to be other developers, so +we ensure these components work as expected when used as building blocks. + +##### Shared utilities testing + +We can test utility functions purely with Jest and don't have to worry about rendering components with +react-testing-library. Only Jest is needed with minimal mocking. + +[View the full url utility testing source](https://github.com/fleetdm/fleet/blob/main/frontend/utilities/url/url.tests.ts). + +```tsx +import { buildQueryStringFromParams } from "."; + +describe("url utilities", () => { + it("creates a query string from a params object", () => { + const params = { + query: "test", + page: 1, + order: "asc", + isNew: true, + }; + expect(buildQueryStringFromParams(params)).toBe( + "query=test&page=1&order=asc&isNew=true" + ); + }); + + it("filters out undefined values", () => { + const params = { + query: undefined, + page: 1, + order: "asc", + }; + expect(buildQueryStringFromParams(params)).toBe("page=1&order=asc"); + }); +}); +``` +##### UI building block testing + +We test the component to ensure it works how a developer would want to use it. There is minimal +mocking and we utilize react-testing-library and Jest spies to test. + +[View the full Radio component testing source](https://github.com/fleetdm/fleet/blob/main/frontend/components/forms/fields/Radio/Radio.tests.tsx). + +```tsx +import React from "react"; +import { noop } from "lodash"; +import { render, screen } from "@testing-library/react"; +import userEvent from "@testing-library/user-event"; + +import Radio from "./Radio"; + +describe("Radio - component", () => { + it("renders the radio label text from the label prop", () => { + render( + + ); + + const labelText = screen.getByText("Radio Label"); + expect(labelText).toBeInTheDocument(); + }); + + it("passes the radio input value when checked", async () => { + const user = userEvent.setup(); + const changeHandlerSpy = jest.fn(); + + render( + + ); + + const radio = screen.getByRole("radio", { name: "Radio Label" }); + await user.click(radio); + + expect(changeHandlerSpy).toHaveBeenCalled(); + expect(changeHandlerSpy).toHaveBeenCalledWith("radioValue"); + }); +}); +``` + +##### Reusable hooks testing + +// TODO + +#### Integration testing + +We use integration testing to strike a balance between speed and expense to write our tests. We use +them to test multiple reusable components that come together into less reusable app widgets. We also +try to use minimal mocking, but we can mock at the backend level if required. + +##### App widget testing + +This layer of tests is great for testing difficult to obtain states and edge cases as the test setup +is simpler than E2E tests. We highly utilize react-testing-library to interface with these components. + +[View the full ResetPasswordForm app widget testing source](https://github.com/fleetdm/fleet/blob/main/frontend/components/forms/ResetPasswordForm/ResetPasswordForm.tests.jsx). + +```tsx +import React from "react"; +import { render, screen } from "@testing-library/react"; + +import { renderWithSetup } from "test/testingUtils"; + +import ResetPasswordForm from "./ResetPasswordForm"; + +describe("ResetPasswordForm - component", () => { + const newPassword = "password123!"; + const submitSpy = jest.fn(); + + it("does not submit the form if the password is invalid", async () => { + const invalidPassword = "invalid"; + const { user } = renderWithSetup( + + ); + + await user.type(screen.getByLabelText("New password"), invalidPassword); + await user.type(screen.getByLabelText("Confirm password"), invalidPassword); + await user.click(screen.getByRole("button", { name: "Reset password" })); + + const passwordError = screen.getByText( + "Password must meet the criteria below" + ); + expect(passwordError).toBeInTheDocument(); + expect(submitSpy).not.toHaveBeenCalled(); + }); +}); +``` + +#### E2E testing + +Our E2E layer tests all the systems of the software (frontend and backend) together to +ensure the application works as intended. To support this, we rarely mock API responses at this +layer of testing. Exceptions include mocking network error responses and mocking external APIs, such +as Jira or Zendesk integration responses. At this level, we want to test the software as an actual user. + +[View the full labels flow user journey testing source](https://github.com/fleetdm/fleet/blob/main/cypress/integration/all/app/labelflow.spec.ts). + +```ts +describe("Labels flow", () => { + before(() => { + // ...setup + }); + after(() => { + // ...teardown + }); + + describe("Manage hosts page", () => { + beforeEach(() => { + // ...setup + }); + it("creates a custom label", () => { + cy.getAttached(".label-filter-select__control").click(); + cy.findByRole("button", { name: /add label/i }).click(); + cy.getAttached(".ace_content").type( + "{selectall}{backspace}SELECT * FROM users;" + ); + cy.findByLabelText(/name/i).click().type("Show all MAC users"); + cy.findByLabelText(/description/i) + .click() + .type("Select all MAC users."); + cy.getAttached(".label-form__form-field--platform > .Select").click(); + cy.getAttached(".Select-menu-outer").within(() => { + cy.findByText(/macOS/i).click(); + }); + cy.findByRole("button", { name: /save label/i }).click(); + cy.findByText(/label created/i).should("exist"); + }); + }); +}); +``` + + +### Tooling + +Here is a quick reference of the current tooling we are using at each layer of testing. + + + +#### ESLint and TypeScript + +We use these for our static analysis testing. These tools have been set up so +errors should appear in your editor if they are broken. + +#### Jest + +We use Jest as our frontend test runner, assertion library, and spy and mock utilities for unit and +integration testing. + +#### Testing library + +We rely heavily on the different libraries that are part of the testing-library ecosystem for our +unit and integration testing. These including react-testing-library, cypress-testing-library, +react-hooks, and user-events. The guiding principles of the testing-library tools align with our own +in that we believe tests should resemble real-world usage as closely as possible. + +#### Cypress + +We use Cypress with Cypress Testing Library as our E2E testing framework. We primarily rely on full +E2E software testing and rarely mock API responses, but we make exceptions for mocking common +network error responses and testing app integrations with external APIs. + +For more details on our E2E testing, check out our [Cypress documentation](../../cypress/README.md). + +### Additional examples + +#### Roles and permissions + +// TODO + +#### Mac and Windows hosts + +// TODO + +#### Error states + +// TODO + +