.. Fedora QA Kiwi TCMS Usage documentation master file, created by
   sphinx-quickstart on Fri Mar  4 10:11:51 2022.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Fedora QA's Kiwi TCMS instance documentation!
=====================================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:


Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`


* :ref:`TerminologyUsed`
* :ref:`adding-test-run`

.. _list discussion: https://lists.fedoraproject.org/NEED_TO_COMPLETE_THIS

Introduction
============


There have been a few discussions in the past about whether we should be using a more formal TCMS in Fedora and this
is the latest attempt to figure out whether it's worth changing how we're storing test cases and results.

The current iteration of this `list discussion`_

**REMEMBER TO FIX THIS LINK ONCE LIVE**


`Kiwi TCMS <https://kiwitcms.org/>`_ is an open source (GPL2) test case management system (TCMS).
`Kiwi's source is available on GitHub <https://github.com/kiwitcms/Kiwi>`_.


.. _KiwiLoginDetails:

Logins and Access
-----------------

This is meant as a demo and as such, it is missing features that would be implemented if we choose to go forward and
use Kiwi as our production TCMS.

One of those missing features is FAS integration so logins are handled by a local user database and accounts are created
manually. If you want access to the demo instance, check the `list discussion`_ for details on how to request
an account.

.. _TerminologyUsed:

Terminology Used
----------------

For the sake of making sure we're all on pretty much the same page, let's get some terminology and definitions out of
the way. Much of the terminology is going to be based on what Kiwi uses since that is the center of this particular
discussion.


.. _KiwiTestCase:

TestCase

    A set of actions performed on a target for the purpose of testing a single specific item.


.. _KiwiTestPlan:

TestPlan

    A group of TestCases which each test separate parts of a single feature.


.. _KiwiTestRun:

TestRun

    An artifact produced when a user runs through 1 or more test cases in a test plan. Generally, only the
    most recent recent test run from a user for a given build and test plan are shown in metrics.


.. _KiwiVersion:

Version

    In this context, a version is roughly a single Fedora release (36, etc.).


.. _KiwiProduct:

Product

    A product in Kiwi roughly corresponds to a deliverable. The way that our demo instance is configured, every product
    and image produced in a compose is considered to be a product.


.. _KiwiBuild:

Build

    A build rougly corresponds to a deliverable within a Fedora compose (e.g, Fedora-36-20220307.n.0 Desktop,
    Fedora 35 RC 1.1 Server)


How to use Kiwi
===============

.. _adding-test-run:

Adding test run to kiwi
-----------------------

 1. Log in to kiwi. See :ref:`KiwiLoginDetails` if you don't have login credentials - Fedora credentials will not work
    for the demo instance.

 #. click on search -> search test plans

    .. image:: img/20220324-base-testplan-search.png
       :width: 800
       :alt: screenshot of the "search test plans" screen in kiwi

 #. find the test plan you want to run (using 'release blocking base environment (Workstation-live-x86_64)' for this
    example) and click on it

    .. image:: img/20220324-base-testplan-search-select-testplan.png
       :width: 800
       :alt: zoomed screenshot showing the link for a test plan from the "search test plans" screen

    Which brings you to a page like:

    .. image:: img/20220324-base-testplan-details.png
       :width: 800
       :alt: screenshot of a testplan details screen in kiwi

 #. Select the test cases you will be running - you can either select test cases one by one or select all test cases
    in the plan by clicking on the box at the top

    .. image:: img/20220324-testplan-selecting-testcases.png
       :width: 800
       :alt: zoomed screenshot of selecting testcases to run


 #. Click on the hamburger menu and "New Test Run"

    .. image:: img/20220324-workstation-testplan-new-test-run-zoomed.png
       :width: 300
       :alt: zoomed screenshot of starting a new test run

 #. Select the build that this test run will be against

    .. image:: img/20220324-new-testrun-build-selection-circled.png
       :width: 800
       :alt: screenshot highlighting where to click on to select a build for a testrun

 #. Click on "Save" to start the test run

 #. Go through each test case shown - clicking on the carrot will expand the test case to show all steps. Notes go into
    the text box.

    .. image:: img/20220324-testrun-expanded-testcase.png
       :width: 800
       :alt: screenshot of expanded testcase in testrun

 #. Make sure to choose a final state for each test case using the colored icons below the text box and click on the
    "Save" button for each testcase when completed.

    .. image:: img/20220324-testrun-expanded-testcase-cropped-buttons.png
       :alt: zoomed screenshot showing the pass/fail/save buttons for a testcase in a testrun

 #. When the test run is complete, click on the orange check icon next to "Finished at:"

    .. image:: img/20220324-testrun-completed-last-step-circled.png
       :width: 800
       :alt: screenshot showing where to click upon testrun completion


Congratulations, you've just added a new test run to the TCMS!


.. image:: img/20220324-testrun-completed.png
    :width: 800
    :alt: screenshot of completed testrun

Links and Resources
===================

 * `Fedora's Kiwi TCMS Demo Instance <https://kiwidemo.fedorainfracloud.org>`_
 * `Documentation Git Repo <https://pagure.io/fedora-qa/fedora-kiwi-instance-docs>`_
 * `Fedora Test Case MD Extractions <https://pagure.io/fedora-qa/fedora-kiwi-instance-docs>`_
 * `Fedora TCMS Demo Migration Code <https://pagure.io/fedora-qa/fedora-tcms-migration>`_