Fedora QA’s Kiwi TCMS instance documentation!

Indices and tables


There have been a few discussions in the past about whether we should be using a more formal TCMS in Fedora and this is the latest attempt to figure out whether it’s worth changing how we’re storing test cases and results.

The current iteration of this list discussion

Kiwi TCMS is an open source (GPL2) test case management system (TCMS). Kiwi’s source is available on GitHub.

Logins and Access

This is meant as a demo and as such, it is missing features that would be implemented if we choose to go forward and use Kiwi as our production TCMS.

One of those missing features is FAS integration so logins are handled by a local user database and accounts are created manually. If you want access to the demo instance, check the list discussion for details on how to request an account.

Terminology Used

For the sake of making sure we’re all on pretty much the same page, let’s get some terminology and definitions out of the way. Much of the terminology is going to be based on what Kiwi uses since that is the center of this particular discussion.


A set of actions performed on a target for the purpose of testing a single specific item.


A group of TestCases which each test separate parts of a single feature.


An artifact produced when a user runs through 1 or more test cases in a test plan. Generally, only the most recent recent test run from a user for a given build and test plan are shown in metrics.


In this context, a version is roughly a single Fedora release (36, etc.).


A product in Kiwi roughly corresponds to a deliverable. The way that our demo instance is configured, every product and image produced in a compose is considered to be a product.


A build rougly corresponds to a deliverable within a Fedora compose (e.g, Fedora-36-20220307.n.0 Desktop, Fedora 35 RC 1.1 Server)

How to use Kiwi

Adding test run to kiwi

  1. Log in to kiwi. See Logins and Access if you don’t have login credentials - Fedora credentials will not work for the demo instance.

  2. click on search -> search test plans

    screenshot of the "search test plans" screen in kiwi
  3. find the test plan you want to run (using ‘release blocking base environment (Workstation-live-x86_64)’ for this example) and click on it

    zoomed screenshot showing the link for a test plan from the "search test plans" screen

    Which brings you to a page like:

    screenshot of a testplan details screen in kiwi
  4. Select the test cases you will be running - you can either select test cases one by one or select all test cases in the plan by clicking on the box at the top

    zoomed screenshot of selecting testcases to run
  5. Click on the hamburger menu and “New Test Run”

    zoomed screenshot of starting a new test run
  6. Select the build that this test run will be against

    screenshot highlighting where to click on to select a build for a testrun
  7. Click on “Save” to start the test run

  8. Go through each test case shown - clicking on the carrot will expand the test case to show all steps. Notes go into the text box.

    screenshot of expanded testcase in testrun
  9. Make sure to choose a final state for each test case using the colored icons below the text box and click on the “Save” button for each testcase when completed.

    zoomed screenshot showing the pass/fail/save buttons for a testcase in a testrun
  10. When the test run is complete, click on the orange check icon next to “Finished at:”

    screenshot showing where to click upon testrun completion

Congratulations, you’ve just added a new test run to the TCMS!

screenshot of completed testrun