Taskotron and Me

  • Flock 2014
  • Tim Flink - Fedora QA, QA Devel, Test Support Monkey

Presenter Notes

Outline

  • Introduction
  • What is Taskotron and How is it different?
  • Running Tasks Locally and Remotely
  • Future Plans
  • Questions

Presenter Notes

What this talk is and isn't

Is About

  • What Taskotron was designed to do
  • How to use taskotron locally
  • What Taskotron will be (noting what is and isn't done yet)

Isn't About

  • Details about Taskotron infrastructure
  • Test Runners
  • How to run tasks locally

Presenter Notes

Definitions

Check

Checking is a process of confirmation, verification and validation. All checks return binary "pass/fail" metrics and can be executed by machines.

Test

Testing is a process of exploration, discovery, investigation and learning. Checking is a subset of checking so while all checks are tests, not all tests are checks but in this context, tests cannot be automated and checks can be.

See [1], [2], [3] for more discussion on testing vs. checking.

Task

A task is a unit of work that can be executed by taskotron. Can be a check, doesn't have to be.

[1]Testing and Checking Refined
[2]Testing vs. Checking
[3]Tests vs. Checks: The Motive for Distinguishing

Presenter Notes

This isn't a talk about checking vs. testing, so don't harp on this too much. It's to make sure that my terminology is understood, not to evangelize. There are links for farther reading if folks are interested.

Where did Taskotron come from?

Why is it that we have so many things which would be beneficial to run at scale but no reasonable way to actually run them at scale?

  • static code checking
  • bundling checks
  • package-specific test suites that aren't a great fit for %check
  • distro health checks (depcheck, upgradepath etc.)

Presenter Notes

What is Taskotron?

Taskotron is first and foremost a system to enable Fedorans to automate tasks which should be executed on a pre-defined schedule. It is designed to do as much as it needs to but delegate the specialized functionality to appropriate code.

Of course, this includes the checks that used to run in AutoQA but it is in no way limited to just those things.

While Taskotron is similar in concept to AutoQA, it has been redesigned to work around many of the limitations that we found in the years of working on AutoQA.

Presenter Notes

No, really, what is Taskotron?

To get a bit more into the realm of the concrete, Taskotron does:

  • Kick off tasks based on received fedmsgs (or manually, in the local case)
  • Prepare an environment for task execution
  • Delegate task execution to specialized code (or runners)
  • Report task results to appropriate places

Presenter Notes

Note that a test runner isn't specified, there is no requirement for a specific test runner and as many runners could be supported as there are resources to support them.

How is this Different from AutoQA?

  • role of core devs
    • Fedorans can maintain their own tasks, don't need us to do it
  • less confusing naming
    • libtaskotron is the software, Taskotron is the system
  • more flexibility to do what we want to do

Presenter Notes

Why didn't you just use X?

At the time that Taskotron started, there was no other option that was:

  • open source
  • not tied to one runner, one distribution or one language
  • had the flexibility that we needed

Presenter Notes

Taskotron Design Principles

  • Minimize coupling where sane
    • Let us change things in smaller chunks
  • Python in the core unless there's a really good reason not to
    • That's what we know and have experience with.
  • REST and json between components
  • Tasks can be isolated in a single git repository
  • Tasks aren't restricted to one language, framework or runner

Presenter Notes

Not quite an absolute but exceptions will be rare and only if there are no other sane options

How Does This all Work?

Presenter Notes

Executing a Task Locally

  • works out of the box, no long setup process
  • output goes to stdout/stderr
  • reporting disabled by default
python runtask.py -i foo-1.2-3.fc99 -t koji_build -a x86_64 \
../task-rpmlint/task-rpmlint..yml

Presenter Notes

Output from Local Task

TAP version 13
1..1
ok - $CHECKNAME for Koji build datagrepper-0.4.2-1.fc21.noarch.rpm
  ---
  details:
    output: |
      datagrepper.noarch: W: spelling-error Summary(en_US) webapp -> web app, web-app, weapon
      datagrepper.noarch: W: spelling-error %description -l en_US webapp -> web app, web-app, weapon
      datagrepper.noarch: W: spelling-error %description -l en_US api -> pi, ape, apt
      datagrepper.noarch: W: spelling-error %description -l en_US datanommer -> manometer
      1 packages and 0 specfiles checked; 0 errors, 4 warnings.
  item: datagrepper-0.4.2-1.fc21.noarch.rpm
  outcome: PASSED
  summary: RPMLINT PASSED for datagrepper-0.4.2-1.fc21.noarch.rpm
  type: koji_build
  ...

Presenter Notes

How is Production Different?

  • If we are doing our job right, very little outside of:
    • Enabling Reporting
    • Pulling from git repo instead of executing locally

Presenter Notes

Running Tasks Remotely

From a high level, it isn't much more complicated:

Presenter Notes

Scheduling

Scheduling is handled by taskotron-trigger and works off of fedmsgs.

Right now it's using simple per-message-type logic but that will be changing once we have more tasks and a better idea of what the scheduling needs are.

Presenter Notes

Execution

In the production instance, all tasks are executed with buildbot. The process is similar to:

  • clone task repository
  • run task using similar entry point as the local run
    • report results as configured

Presenter Notes

So, What can I do with Taskotron?

  • Just about anything you can think of, within reason.

Presenter Notes

This is vague, but at the same time, we don't pretend to be experts in all of Fedora. The whole idea here is to enable automation as much as it is to get that work done. I'm not going to get up here and say that I'm the only authority on what needs to be automated and instead of trying to do all the automation ourselves, I want to see what ideas and approaches that other fedorans have.

Future Plans

Presenter Notes

Task Execution Features

  • Disposable Clients
    • May be OpenStack, may be more raw libvirt
  • Enhanced Syntax
    • package install

Presenter Notes

UX Enhancements

  • Dashboard
    • Single landing point for job, package history
    • Make it easy for people to find information, logs
  • Logging
    • Searchable
    • Analytics and statistics on logs?

Presenter Notes

What to do next?

Write Tasks

  • Use what exists, write new tasks
  • Documentation on how to do this is available

Report Bugs

  • If you find issues, tell us!
    • want it to work and work well

Presenter Notes

Conclusion

  • The only thing missing from Taskotron is you
  • Important Takeaways
    • You don't need a whole system set up to use Taskotron
    • Taskotron can do just about anything you're willing to write code for
    • The only way that Taskotron is going to succeed is for other groups to use it.
    • We are committed to having good documentation and helping out as much as we can. If something doesn't make sense or doesn't seem to work, come find me

Presenter Notes

Questions/Comments?

Presenter Notes

Presenter Notes