Fedora QA’s Kiwi TCMS instance documentation!¶

Indices and tables¶

Introduction¶

There have been a few discussions in the past about whether we should be using a more formal TCMS in Fedora and this is the latest attempt to figure out whether it’s worth changing how we’re storing test cases and results.

The current iteration of this list discussion

Kiwi TCMS is an open source (GPL2) test case management system (TCMS). Kiwi’s source is available on GitHub.

Logins and Access¶

This is meant as a demo and as such, it is missing features that would be implemented if we choose to go forward and use Kiwi as our production TCMS.

One of those missing features is FAS integration so logins are handled by a local user database and accounts are created manually. If you want access to the demo instance, check the list discussion for details on how to request an account.

Terminology Used¶

For the sake of making sure we’re all on pretty much the same page, let’s get some terminology and definitions out of the way. Much of the terminology is going to be based on what Kiwi uses since that is the center of this particular discussion.

TestCase

A set of actions performed on a target for the purpose of testing a single specific item.

TestPlan

A group of TestCases which each test separate parts of a single feature.

TestRun

An artifact produced when a user runs through 1 or more test cases in a test plan. Generally, only the most recent recent test run from a user for a given build and test plan are shown in metrics.

Version

In this context, a version is roughly a single Fedora release (36, etc.).

Product

A product in Kiwi roughly corresponds to a deliverable. The way that our demo instance is configured, every product and image produced in a compose is considered to be a product.

Build

A build rougly corresponds to a deliverable within a Fedora compose (e.g, Fedora-36-20220307.n.0 Desktop, Fedora 35 RC 1.1 Server)

How to use Kiwi¶

Adding test run to kiwi¶

Log in to kiwi. See Logins and Access if you don’t have login credentials - Fedora credentials will not work for the demo instance.

click on search -> search test plans

find the test plan you want to run (using ‘release blocking base environment (Workstation-live-x86_64)’ for this example) and click on it

Which brings you to a page like:

Select the test cases you will be running - you can either select test cases one by one or select all test cases in the plan by clicking on the box at the top

Click on the hamburger menu and “New Test Run”

Select the build that this test run will be against

Click on “Save” to start the test run

Go through each test case shown - clicking on the carrot will expand the test case to show all steps. Notes go into the text box.

Make sure to choose a final state for each test case using the colored icons below the text box and click on the “Save” button for each testcase when completed.

When the test run is complete, click on the orange check icon next to “Finished at:”