The required output format of the checks in Taskotron is YAML.
We recommend using the provided libtaskotron.check.CheckDetail and the helper method libtaskotron.check.export_YAML() to create the YAML output. If you can not use it, then please adhere to the specification described below.
First up, we will provide you with two examples of the valid YAML output - the minimal version, which just covers requirements, and then the extended version, which uses the format to its full potential Taskotron-vise.
If you don’t need anything fancy, then this is the minimalistic version of the output which, at the same time, is valid for results reporting in Taskotron:
results:
- item: xchat-0.5-1.fc20
type: koji_build
outcome: PASSED
You just need to provide the item, type and outcome values in the YAML block.
The outcome field defines whether the check result was OK or not OK. Values allowed at the moment, are PASSED, INFO, FAILED, ERROR, WAIVED, and NEEDS_INSPECTION. Using other values will cause the ResultsDB reporting to fail.
type describes “what” was tested on a high level - whether it was a build from Koji, DVD image, update in Bodhi, or something completely different. At the moment, we have internally standardized three type values for the most common items we test:
- koji_build for testing new builds in Koji
- bodhi_update to represent results tied to a specific Bodhi update
- yum_repository for repo-sanity checks, and so on
- compose for compose-related testing
You can, of course, use any other value for the type parameter, but if you have a check for repositories, Koji builds or Bodhi updates, please stick to the already defined ones, so the tools next in the chain will recognize your results properly.
item is a string representation of the specific piece, that was tested - NVR for the koji_build type, update-name or update-id for the bodhi_update, etc. Once again - you can set whatever value you think is right, but sticking to the reasonable/obvious choices will ensure seamless processing.
Note
item, type and outcome are just some of the “reserved” key-names in the YAML block. Refer to the next section for the full list.
Although the minimal version of the output is sufficient, you can use additional fields to provide more information:
results:
- item: tzdata-2014f-1.fc20
type: bodhi_update
outcome: ABORTED
artifact: upgradepath_details.html
checkname: upgradepath
note: from testing to stable
arch: x86_64
package: tzdata
- item: bluez-5.34-1.fc23
type: bodhi_update
outcome: PASSED
artifact: upgradepath_details.html
checkname: upgradepath
note: from testing to stable
arch: x86_64
package: bluez
- item: tzdata-2014f-1.fc20
type: bodhi_update
outcome: PASSED
artifact: depcheck_tzdata-2014f-1.fc20.txt
checkname: depcheck
note: for stable
arch: x86_64
Going top-down, the new fields are artifact, checkname, note, arch and package.
artifact contains the name of a file in the $artifactsdir, that contains detailed information about the check run. In this example the upgradepath check produced an user-friendly report in HTML format, and the author decided to link it as an artifact.
checkname allows overriding the value devised from the Taskotron formula. It is useful when your task implements multiple checks, and you want to report the per-check results separately (e.g. repo-sanity could output depcheck, upgradepath and repoclosure results in one run).
note adds a short valuable information that should be easily visible in the result overview.
arch and package are custom fields that do not have any special meaning for the Taskotron tooling, but provide additional information about the check results as such.
item, type, outcome, note are the commonly used “reserved” fields. We also decided to use arch as a field with “recommended meaning” (to represent the architecture). You can then add any number of custom key-value pairs (like the package in the above example), which will get stored in ResultsDB, and can be then used for searching.
There are two more reserved fields - _internal and results._