.. _resultyaml-format:

==================
Task result format
==================

The required output format of the checks in Taskotron is YAML.

We recommend using the provided :py:class:`libtaskotron.check.CheckDetail` and the
helper method :py:meth:`libtaskotron.check.export_YAML` to create the YAML output.
If you can not use it, then please adhere to the specification described below.


YAML and reporting
==================

First up, we will provide you with two examples of the valid YAML output - the minimal
version, which just covers requirements, and then the extended version, which uses the
format to its full potential Taskotron-vise.


Minimal version
---------------

If you don't need anything fancy, then this is the minimalistic version of the output
which, at the same time, is valid for results reporting in Taskotron::

 results:
   - item: xchat-0.5-1.fc20
     type: koji_build
     outcome: PASSED
 
You just need to provide the ``item``, ``type`` and ``outcome`` values in
the YAML block.

The ``outcome`` field defines whether the check result was OK or not OK.
Values allowed at the moment, are ``PASSED``, ``INFO``, ``FAILED``, ``ERROR``, ``WAIVED``,
and ``NEEDS_INSPECTION``. Using other values will cause the ResultsDB reporting to fail.

``type`` describes "what" was tested on a high level - whether it was a build from Koji, DVD image,
update in Bodhi, or something completely different. At the moment, we have internally
standardized three ``type`` values for the most common items we test:

  * ``koji_build`` for testing new builds in Koji
  * ``bodhi_update`` to represent results tied to a specific Bodhi update
  * ``yum_repository`` for repo-sanity checks, and so on
  * ``compose`` for compose-related testing

You can, of course, use any other value for the ``type`` parameter, but if you have a check
for repositories, Koji builds or Bodhi updates, please stick to the already defined ones,
so the tools next in the chain will recognize your results properly.

``item`` is a string representation of the specific piece, that was tested - NVR for the ``koji_build``
type, update-name or update-id for the ``bodhi_update``, etc. Once again - you can set whatever
value you think is right, but sticking to the reasonable/obvious choices will ensure seamless
processing.

.. Note::

  ``item``, ``type`` and ``outcome`` are just some of the "reserved" key-names in the YAML block. Refer to the
  next section for the full list.


Full version
------------

Although the minimal version of the output is sufficient, you can use additional fields
to provide more information::

  results:
    - item: tzdata-2014f-1.fc20
      type: bodhi_update
      outcome: ABORTED
      artifact: upgradepath_details.html
      checkname: upgradepath
      note: from testing to stable
      arch: x86_64
      package: tzdata
    - item: bluez-5.34-1.fc23
      type: bodhi_update
      outcome: PASSED
      artifact: upgradepath_details.html
      checkname: upgradepath
      note: from testing to stable
      arch: x86_64
      package: bluez
    - item: tzdata-2014f-1.fc20
      type: bodhi_update
      outcome: PASSED
      artifact: depcheck_tzdata-2014f-1.fc20.txt
      checkname: depcheck
      note: for stable
      arch: x86_64

Going top-down, the new fields are ``artifact``, ``checkname``, ``note``, ``arch`` and ``package``.

``artifact`` contains the name of a file in the ``$artifactsdir``, that contains detailed
information about the check run. In this example the upgradepath check produced an user-friendly
report in HTML format, and the author decided to link it as an artifact.

``checkname`` allows overriding the value devised from the Taskotron formula. It is useful
when your task implements multiple checks, and you want to report the per-check results
separately (e.g. ``repo-sanity`` could output ``depcheck``, ``upgradepath`` and ``repoclosure``
results in one run).

``note`` adds a short valuable information that should be easily visible in the result overview.

``arch`` and ``package`` are custom fields that do not have any special meaning for the Taskotron
tooling, but provide additional information about the check results as such.


Note on the reserved field names
--------------------------------

``item``, ``type``, ``outcome``, ``note`` are the commonly used "reserved" fields. We also decided to use
``arch`` as a field with "recommended meaning" (to represent the architecture). You can then add any
number of custom key-value pairs (like the ``package`` in the above example), which will get stored
in ResultsDB, and can be then used for searching.

There are two more reserved fields - ``_internal`` and ``results``._