.. _writing-tasks-for-taskotron: =========================== Writing Tasks for Taskotron =========================== While the eventual target for most tasks will be to run in a production system, one of libtaskotron's features is that it can be run without the overhead of a full production-like system. To learn more about what can be done with taskotron, see :ref:`what-can-i-do-with-taskotron`. In order to write tasks, you must :ref:`install-libtaskotron` on your local machine if you aren't running from git. Examples ======== examplebodhi ------------ `examplebodhi task git repository `_ The examplebodhi task is a trivial example which takes an update and determines a PASS/FAIL results in a pseudo-random fashion. This task wasn't designed to be used in production but is useful as a starting point and for development purposes. rpmlint ------- `rpmlint task git repository `_ rpmlint is the simplest of the production tasks. It takes a koji build and runs rpmlint on it, reporting results as result-YAML. Creating a New Task =================== While running rpmlint is an easy and trivial example, that doesn't help with creating something new. To write a new task, you'll need some information: * What type of object will the task be run against? * What will be the name of the task? * What are the dependencies? The taskotron runner works on specially formatted YAML files (see :ref:`taskotron-formula-format` for details) called formulae which describe the task that is to be executed. In addition to the task formula, any other code or resources needed for task execution should be accessible by the runner. A directory layout could look like:: mytask/ mytask.yml mytask.py someresource.txt .. note:: Taskotron was designed to work with tasks stored in git repositories and while this is not strictly required for the local execution case, it is highly recommended that you limit tasks to one per directory. Non-Executed Task Information ----------------------------- For this example, let's write a task that runs on bodhi IDs. Looking at the non-executed task data needed in :file:`mytask.yml`: .. code-block:: yaml --- name: mytask desc: do something stupendous maintainer: somefasuser input: args: - arch - bodhi_id environment: rpm: - rpm_that_task_uses Executed Task Information ------------------------- Let's have this new task do the following: * download the rpms contained in the indicated bodhi update * run the function ``run_mytask`` in :file:`mytask.py` using a list of the downloaded rpms as input * report the results to resultsdb .. code-block:: python # this is a trivial example that doesn't do everything we want it to do, more # details will be added later in the example def run_mytask(rpmfiles, bodhi_id): print "Running mytask on %s" % bodhi_id .. code-block:: yaml actions: # we can use bodhi_id and arch as variables here because they're defined # for us by the runner which also verifies that all required input listed # above is present before executing the task - name: download bodhi update bodhi: action: download update_id: ${bodhi_id} arch: ${arch} export: bodhi_downloads # notice how the third item under python is 'rpmfiles' and this matches # the arg name in mytask.py. Anything outside of 'file' and 'callable' # are passed in as kwargs to the specified python callable. - name: run mytask python: file: mytask.py callable: run_mytask rpmfiles: ${bodhi_downloads} bodhi_id: ${bodhi_id} export: mytask_output # this assumes that the output from mytask is valid result-YAML - name: report mytask results to resultsdb resultsdb: results: ${mytask_output} .. this needs to be fixed, this output is doctored since it doesn't actually work. We can execute this new task using:: $ python runtask.py -i FEDORA-2016-8760e32d9b -t bodhi_id -a x86_64 ../task-mytask/mytask.yml The output from this task should end with:: Running mytask on FEDORA-2016-8760e32d9b [libtaskotron] 15:00:40 CRITICAL Traceback (most recent call last): File "runtask.py", line 10, in main.main() File "/home/mkrizek/devel/libtaskotron/libtaskotron/main.py", line 151, in main overlord.start() File "/home/mkrizek/devel/libtaskotron/libtaskotron/overlord.py", line 95, in start runner.execute() File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 56, in execute self._run() File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 93, in _run self._do_actions() File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 131, in _do_actions self._do_single_action(action) File "/home/mkrizek/devel/libtaskotron/libtaskotron/executor.py", line 152, in _do_single_action self.arg_data) File "/home/mkrizek/devel/libtaskotron/libtaskotron/directives/resultsdb_directive.py", line 196, in process raise TaskotronDirectiveError("Failed to load 'results': %s" % e.message) TaskotronDirectiveError: Failed to load 'results': Failed to parse YAML contents: empty input We're not actually passing result-YAML output from our task, so let's get that fixed. Actually Doing Something ------------------------ The Python code for ``mytask.py`` above doesn't really do anything at all, so let's make it do something: * Change the task to create a result from the input bodhi update id and list all of the rpms contained in that update as details. Generating result-YAML from python tasks in Taskotron isn't very difficult. Our original task looked like: .. code-block:: python # this is a trivial example that doesn't do everything we want it to do, more # details will be added later in the example def run_mytask(rpmfiles): for rpmfile in rpmfiles['downloaded_rpms']: print "Running mytask on %s" % rpmfile To create valid result-YAML, we want to populate a :py:class:`libtaskotron.check.CheckDetail` object and use :py:meth:`libtaskotron.check.export_YAML` to generate valid result-YAML with the data required for reporting. .. note:: If you can not use the :py:class:`libtaskotron.check.CheckDetail` directly, then create the YAML output according to :ref:`resultyaml-format`. .. code-block:: python from libtaskotron import check def run_mytask(rpmfiles, bodhi_id): """run through all passed in rpmfiles and emit result-YAML saying they all passed""" print "Running mytask on %s" % bodhi_id result = 'PASSED' note = 'from testing-pending to testing' detail = check.CheckDetail(bodhi_id, check.ReportType.BODHI_UPDATE, result, note) for rpmfile in rpmfiles['downloaded_rpms']: detail.store(rpmfile, printout=False) return check.export_YAML(detail) Now if we run the task we get output that ends with:: [libtaskotron] 10:01:03 INFO Reporting to ResultsDB is disabled. Once enabled, the following would get reported: results: - item: FEDORA-2016-8760e32d9b note: from testing-pending to testing outcome: PASSED type: bodhi_update Working with CheckDetail Objects -------------------------------- Check can have one of these outcomes (with increasing priority): ``PASSED``, ``INFO``, ``FAILED``, ``NEEDS_INSPECTION``, ``ABORTED`` and ``CRASHED``. These are available as a list (where higher index means bigger priority) in :py:attr:`CheckDetail.outcome_priority `. You can set outcome during :py:class:`libtaskotron.check.CheckDetail` creation, by setting :py:attr:`CheckDetail.outcome ` directly or by using :py:meth:`libtaskotron.check.CheckDetail.update_outcome` - it changes task outcome only if it has higher priority than current outcome. :py:meth:`libtaskotron.check.CheckDetail.store` is a convenience method if you receive some output gradually and you want to store it and also print it at the same time. You can also set check's output by setting :py:attr:`CheckDetail.output` or in :py:class:`CheckDetail ` constructor. .. code-block:: python import os import random from libtaskotron import check def random_choice(_): return random.choice(['PASSED', 'FAILED', 'ABORTED']) def run_mytask(rpmfiles, bodhi_id): """run through all passed in rpmfiles and emit result-YAML. randomly report failure or pass""" print "Running mytask on %s" % bodhi_id seed = random.random() random.seed(seed) results = [(rpmfile, random_choice(rpmfile)) for rpmfile in rpmfiles['downloaded_rpms']] detail = check.CheckDetail(bodhi_id, check.ReportType.BODHI_UPDATE) for rpmfile, outcome in results: # e.g. "xchat-1.2-3.fc20.x86_64.rpm: FAILED" detail.store('%s: %s' % (os.path.basename(rpmfile), outcome)) detail.update_outcome(outcome) detail.note = "seed was %s" % seed detail.checkname = "mytask" return check.export_YAML(detail) Running this code as ``python runtask.py -i tzdata-2014f-1.fc20 -t bodhi_id -a x86_64 ../task-mytask/mytask.yml`` ends with:: results: - item: tzdata-2014f-1.fc20 outcome: ABORTED note: seed was 0.45189 type: bodhi_update checkname: mytask For more information about APIs and ``CheckDetail`` usage, see :py:class:`CheckDetail class `. .. note:: Reporting is disabled in the default configuration for Taskotron. While it is possible to set up a local resultsdb instance to check reporting, we want to make it easier than that and will update these docs when that easier method is available. For now, if the resultsdb directive doesn't throw exceptions, it's reasonable to assume that the result-YAML output was constructed correctly and is reasonably valid.