libtaskotron API docs

This isn’t final API documentation, more of an example of how we can do API docs.

Once more of our code is using sphinx-style docs, we can start finalizing the format for this page.

Note that the docs here are for libtaskotron library functions and don’t include directives or task yaml details.

arch_utils

class libtaskotron.arch_utils.Arches

Helper class for working with supported arches inside taskotron

all = ['i386', 'i486', 'i586', 'i686', 'x86_64', 'armhfp', 'arm7hl', 'noarch', 'src']

all supported architectures

base = ['i386', 'x86_64', 'armhfp']

base architectures

meta = ['noarch', 'src']

meta architectures

libtaskotron.arch_utils.basearch(arch=None)

This returns the ‘base’ architecture identifier for a specified architecture (e.g. i386 for i[3-6]86), to be used by YUM etc.

Parameters:arch (str) – an architecture to be converted to a basearch. If None, then the arch of the current machine is used. Returns:basearch, or arch if no basearch was found for it Return type:str

bodhi_utils

Utility functions for dealing with Bodhi

class libtaskotron.ext.fedora.bodhi_utils.BodhiUtils(client=None)

Helper Bodhi methods.

Variables:client (fedora.client.Bodhi2Client) – Bodhi2 client instance

Create a new BodhiUtils instance.

Parameters:client – custom Bodhi2Client instance. If None, a default Bodhi2Client instance is used.

build2update(builds, strict=False)

Find matching Bodhi updates for provided builds.

Parameters:

• builds (iterable of str) – builds to search for in N(E)VR format (foo-1.2-3.fc20 or foo-4:1.2-3.fc20)
• strict (bool) – if False, incomplete Bodhi updates are allowed. If True, every Bodhi update will be compared with the set of provided builds. If there is an Bodhi update which contains builds not provided in builds, that update is marked as incomplete and removed from the result - i.e. all builds from builds that were part of this incomplete update are placed in the second dictionary of the result tuple.

Returns:

a tuple of two dictionaries:

• The first dict provides mapping between builds and their updates where no error occured.

  {build (string): Bodhi update (Munch)}

• The second dict provides mapping between builds and their updates where some error occured. The value is None if the matching Bodhi update could not be found (the only possible cause of failure if strict=False). Or the value is a Bodhi update that was incomplete (happens only if strict=True).

  {build (string): Bodhi update (Munch) or None}

• The set of keys in both these dictionaries correspond exactly to builds. For every build provided in builds you’ll get an answer in either the first or the second dictionary, and there will be no extra builds that you haven’t specified.

Raises TaskotronValueError:  

if builds type is incorrect

get_update(updateid)

Get the last Bodhi update for the specified update ID.

Parameters:updateid (str) – update ID, e.g. ‘FEDORA-2015-13787’ Returns:Bodhi update object with that ID, or None when no such update is found Return type:munch.Munch

buildbot_utils

Utility functions for dealing with Buildbot

libtaskotron.buildbot_utils.get_urls(jobid, masterurl, task_stepname)

Form url to the job and its logs based on given jobid.

Parameters:

• jobid (str) – jobid from runner
• masterurl (str) – taskotron master url
• task_stepname (str) – taskotron master step name

Returns:

(job_url, log_url)

libtaskotron.buildbot_utils.parse_jobid(jobid)

Parse the incoming jobid which should either be ‘-1’ to indicate that dummy values should be used or in the form of <builder>/<buildid>.

If the passed in format is invalid, no exceptions will be raised but a warning message will be emitted to logs

Parameters:jobid (str) – jobid from runner Returns:(builder, buildid)

check

Helper tools for managing check status, outcome, and output.

class libtaskotron.check.CheckDetail(item, report_type=None, outcome=None, note='', output=None, keyvals=None, checkname=None, artifact=None)

Class encompassing everything related to the outcome of a check run. It contains convenience methods for tracking, evaluating and reporting check results.

For some checks, it’s trivial to parse its output at the end of its execution and evaluate the results. For some, however, it’s much easier to do this continuously during check execution. This is especially true for multi-item checks (a single check providing results for many items - builds, updates, etc). That’s when this class comes very handy (it can be used even for the simple use cases, of course).

Variables:

• outcome_priority – a tuple of outcome keywords sorted by priority from the least important to the most important
• item (str) – a description the item being tested; for example a build NVR (vpnc-0.5-1.fc20), update ID (FEDORA-2014-3309) or a repository name (f20-updates)
• report_type (str) – a definition of the type of the object being checked; for example ‘a Koji build’, ‘a Bodhi update’ or ‘a yum repository name’. The allowed values are attributes in ReportType. You don’t have to fill this in (or you can provide a custom string value), but the reporting directives react only to the known types (you can always find it in ResultsDB, though).
• outcome (str) –

  a keyword specifying the final outcome of the check. Available outcome keywords:

  • PASSED - everything went well
  • INFO - everything went well, but there is some important information that needs to be pointed out
  • FAILED - the item in question fails the check
  • NEEDS_INSPECTION - the outcome can’t be determined and a manual inspection is needed
  • ABORTED - the check aborted itself because of some unexpected problems, i.e. a necessary network server is not reachable. Running this check with the same arguments later can help to mitigate this problem.
  • CRASHED - the check crashed and did not provide usable results

  If no outcome has been set, this attribute returns NEEDS_INSPECTION.

  Raises TaskotronValueError if you try to assign an unknown keyword.

• note (str) – a few words or one-sentence note about the result of the check run (e.g. 5 WARNINGS, 1 ERROR for an rpmlint result). Should not unnecessarily duplicate item or outcome, if possible.
• output – output from the check run (a list of strings). You can easily populate this by using @method store(), or you can modify it directly (for example filter out some messages you don’t want to see in the check output).
• keyvals (dict) – all key-value pairs in this dictionary are stored in ResultsDB as ‘extra data’. This can be used to store e.g. a compose id or a kernel version, if that information is important for querying the results. Keep it as short as possible.
• checkname (str) – name of the check, which the CheckDetail belongs to. This is usually not needed, as the check name is devised from the task metadata, but if a single task produces results for multiple checks, this is the way to override the default behavior.
• artifact (str) – path to a file or directory placed in the artifacts directory, either absolute $artifactsdir/report.html or relative report.html. It will represent task output specific for this particular :attr:item.

broken()

Tells whether the check outcome is set to one of the broken states (i.e. ABORTED or CRASHED).

classmethod cmp_outcome(outcome1, outcome2)

Compare two outcomes according to outcome_priority and return -1/0/1 if outcome1 has lower/equal/higher priority than outcome2.

Parameters:

• outcome1 (str) – an outcome keyword to compare
• outcome2 (str) – an outcome keyword to compare

Raises TaskotronValueError:  

if any of the outcome keywords are not specified in outcome_priority

classmethod create_multi_item_summary(outcomes)

Create a string containing a sum of all outcomes, like this: 3 PASSED, 1 INFO, 2 FAILED

Parameters:outcomes – either one CheckDetail instance or an iterable of CheckDetails or an iterable of outcome strings

store(message, printout=True)

Add a string to the output, and print it optionally as well.

This is just a convenience method for most common use case, you can of course always access and modify output directly, and print or use logging facilities directly as well. This combines both into a single call.

Parameters:

• message (str) – a string to store in output
• printout (bool) – whether to print to standard output or not

update_outcome(outcome)

Update outcome with the provided value only and only if it has a higher priority than its current value (according to outcome_priority). Otherwise this call does nothing.

This is useful if your check performs a number of ‘sub-tasks’, each passing or failing, and you want to final outcome to be the worst/most serious one of those. You can just keep calling update_outcome() and the highest priority outcome type will be stored at outcome at the end.

Parameters:outcome (str) – the outcome value to assign to outcome if it has a higher priority than its current value. Handles None values gracefully (no action). Raises TaskotronValueError:  if any of the outcome keywords are not specified in outcome_priority

class libtaskotron.check.ReportType

Enum for different types of CheckDetail.report_type

BODHI_UPDATE = 'bodhi_update'

COMPOSE = 'compose'

KOJI_BUILD = 'koji_build'

YUM_REPOSITORY = 'yum_repository'

libtaskotron.check.export_YAML(check_details)

Generate YAML output used for reporting to ResultsDB.

Note: You need to provide all your CheckDetails in a single pass in order to generate a valid YAML output. You can’t call this method several times and then simply join the outputs simply as strings.

Parameters:check_details – iterable of CheckDetail instances or single instance of CheckDetail Returns:YAML output with results for every CheckDetail instance provided Return type:str Raises TaskotronValueError:  if CheckDetail.item or CheckDetail.outcome is empty for any parameter provided

Example output:

results:
  - item: xchat-0.5-1.fc20
    type: koji_build
    outcome: PASSED
    note: 5 ERRORS, 10 WARNINGS
    artifact: xchat-0.5-1.fc20.x86_64.log

libtaskotron.check.import_YAML(source)

Parses YAML and returns a list of CheckDetails.

Parameters:source (str) – YAML-formatted text Raises TaskotronValueError:  if YAML syntax is incorrect

config

Global configuration for Taskotron and relevant helper methods.

libtaskotron.config.CONF_DIRS = ['/home/tflink/code/taskotron/libtaskotron/conf', '/etc/taskotron']

A list of directories where config files are stored. The config files are loaded from these locations in the specified order, and only the first config file found is used (further locations are ignored). The first location is dynamic, relative to the package location and it reflects the usual config location in a git checkout.

libtaskotron.config.CONF_FILE = 'taskotron.yaml'

the name of our configuration file

libtaskotron.config.PROFILE_VAR = 'TASKOTRON_PROFILE'

environment variable name for setting config profile

libtaskotron.config.get_config()

Get the Config instance. This method is implemented using the singleton pattern - you will always receive the same instance, which will get auto-initialized on the first method call.

Returns:either Config or its subclass, depending on taskotron profile used. Raises TaskotronConfigError:  if config file parsing and handling failed

config_defaults

This includes the default values for Taskotron configuration. This is automatically loaded by config.py and then overridden by values from config files available in system-wide location.

class libtaskotron.config_defaults.Config

Global configuration for Taskotron (development profile).

The documentation for individual options is available in the config files (unless they’re not present in the config files, then they’re documented here).

Implementation notes:

• If you want to add a new option, put it here and optionally into the config file as well.
• If you modify a default value for some option, don’t forget to modify it in both places - here and in the config file (if present).
• Don’t assign None as a default value. We need to know a value type in order to check for correct type of user-provided values.

artifacts_baseurl = 'http://localhost/artifacts'

artifactsdir = '/var/lib/taskotron/artifacts'

bodhi_staging = False

buildbot_task_step = 'runtask'

cachedir = '/var/cache/taskotron'

client_taskdir = '/var/tmp/taskotron/taskdir'

config_filename = ''

download_cache_enabled = True

execdb_server = 'http://localhost:5003'

imagesdir = '/var/lib/taskotron/images'

imageurl = 'http://download.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-23-20151030.x86_64.qcow2'

koji_url = 'http://koji.fedoraproject.org/kojihub'

log_file_enabled = False

log_level_file = 'DEBUG'

log_level_stream = 'INFO'

logdir = '/var/log/taskotron'

minion_repos = ['https://copr-fe.cloud.fedoraproject.org/coprs/tflink/taskotron/repo/fedora-23/tflink-taskotron-fedora-23.repo']

pkg_url = 'http://kojipkgs.fedoraproject.org/packages'

profile = 'development'

report_to_resultsdb = False

resultsdb_frontend = 'http://localhost:5002'

resultsdb_server = 'http://localhost:5001/api/v1.0'

runtask_mode = 'local'

ssh_privkey = ''

taskotron_master = 'http://localhost/taskmaster'

tmpdir = '/var/tmp/taskotron'

class libtaskotron.config_defaults.ProductionConfig

Configuration for production profile. Inherits values from Config and overrides some. Read Config documentation.

download_cache_enabled = False

log_file_enabled = True

log_level_file = 'DEBUG'

log_level_stream = 'INFO'

profile = 'production'

report_to_resultsdb = True

runtask_mode = 'libvirt'

class libtaskotron.config_defaults.ProfileName

Enum of available profile names. These can be specified in the config file or as the environment variable.

DEVELOPMENT = 'development'

PRODUCTION = 'production'

TESTING = 'testing'

class libtaskotron.config_defaults.RuntaskModeName

Enum of available runtask mode names. These can be specified in the config file.

LIBVIRT = 'libvirt'

LOCAL = 'local'

class libtaskotron.config_defaults.TestingConfig

Configuration for testing suite profile. Inherits values from Config and overrides some. Read Config documentation.

artifactsdir = '/var/tmp/taskotron-test/artifacts'

cachedir = '/var/tmp/taskotron-test/cache'

log_level_file = 'DEBUG'

log_level_stream = 'DEBUG'

logdir = '/var/tmp/taskotron-test/log'

profile = 'testing'

tmpdir = '/var/tmp/taskotron-test/tmp'

exceptions

This module contains custom Taskotron exceptions

exception libtaskotron.exceptions.TaskotronConfigError

All errors related to Taskotron config files

exception libtaskotron.exceptions.TaskotronDirectiveError

All errors related to Taskotron directives

exception libtaskotron.exceptions.TaskotronError

Common ancestor for Taskotron related exceptions

exception libtaskotron.exceptions.TaskotronImageError

All generic image related errors

exception libtaskotron.exceptions.TaskotronImageNotFoundError

Requested image not found error

exception libtaskotron.exceptions.TaskotronImportError(e)

All issues with Extensions

exception libtaskotron.exceptions.TaskotronNotImplementedError

NotImplementedError for Taskotron classes, methods and functions

exception libtaskotron.exceptions.TaskotronRemoteError

All network and remote-server related errors

exception libtaskotron.exceptions.TaskotronValueError

Taskotron-specific ValueError

exception libtaskotron.exceptions.TaskotronYamlError

Error in YAML config file of the executed check

executor

class libtaskotron.executor.Executor(task_data, arg_data, workdir=None)

This class serves the purpose of actual task execution. It is instantiated by Overlord, PersistentMinion or DisposableMinion.

Create a new instance.

Parameters:

• task_data (dict) – data loaded from task formula
• arg_data (dict) – data from command line arguments
• workdir (str) – path to working directory, temporary is created if None

execute()

Execute the task (consists of preparing the task and running it).

exitcode = None

int exitcode set by exitcode directive, is None if exitcode directive was not used

file_utils

class libtaskotron.file_utils.Tee(*files)

Helper class for writing data to different streams.

libtaskotron.file_utils.download(url, dirname, filename=None, cachedir=None)

Download a file.

Parameters:

• url (str) – file URL to download
• dirname (str) – directory path; if the directory does not exist, it gets created (and all its parent directories).
• filename (str) – name of downloaded file; if not provided, the basename is extracted from URL
• cachedir (str) – If set, the file will be downloaded to a cache directory specified by this parameter. If the file is already present and of the same length, download is skipped. The requested destination file (dirname/filename) will be a symlink to the cached file. This directory is automatically created if not present.

Returns:

the path to the downloaded file

Return type:

str

Raises TaskotronRemoteError:  

if download fails

libtaskotron.file_utils.makedirs(fullpath)

This is the same as os.makedirs(), but does not raise an exception when the destination directory already exists.

Raises OSError:if directory doesn’t exist and can’t be created

koji_utils

Utility methods related to Koji

class libtaskotron.ext.fedora.koji_utils.KojiClient(koji_session=None)

Helper Koji methods.

Variables:session (koji.ClientSession) – Koji client session

Create a new KojiClient

Parameters:koji_session (koji.ClientSession) – an existing Koji session instance or None if you want a new default session to be created

get_nvr_rpms(nvr, dest, arches=['all'], debuginfo=False, src=False)

Retrieve the RPMs associated with a build NVR into the specified directory.

Parameters:

• nvr (str) – build NVR
• dest (str) – location where to store the RPMs
• arches – see nvr_to_urls()
• debuginfo – see nvr_to_urls()
• src – see nvr_to_urls()

Returns:

list of local filenames of the grabbed RPMs (might be empty, according to your option choices and the particular NVR)

Return type:

list of str

Raises:

• TaskotronRemoteError – if the files can’t be downloaded
• TaskotronValueError – if arches=[] and src=False, therefore there is nothing to download

get_tagged_rpms(tag, dest, arches=['all'], debuginfo=False, src=False)

Downloads all RPMs of all NVRs tagged by a specific Koji tag.

Note: This works basically the same as get_nvr_rpms(), it just downloads a lot of builds instead of a single one. For description of all shared parameters and return values, please see that method.

Parameters:tag (str) – Koji tag to be queried for available builds, e.g. f20-updates-pending

latest_by_tag(tags, pkgname)

Get the latest Koji build for the given package name in the given tag(s).

Parameters:

• tags (list) – list of tags to be searched in
• pkgname (str) – name of the package

Returns:

str containing NVR of the latest build, None if no build was found

nvr_to_urls(nvr, arches=['all'], debuginfo=False, src=True)

Get list of URLs for RPMs corresponding to a build.

Parameters:

• nvr (str) – build NVR
• arches (list of str) –

  restrict the arches of builds to provide URLs for. By default, all architectures are considered. If you want to consider just some selected arches, provide their names in a list.

  Note

  If basearch i386 is in the list, i686 arch is automatically added (since that’s the arch Koji API uses).

• debuginfo (bool) – whether to provide URLs for debuginfo RPM files or ignore them
• src (bool) – whether to include a URL for the source RPM

Return type:

list of str

Raises:

• TaskotronRemoteError – when the requested build doesn’t exist
• TaskotronValueError – if arches=[] and src=False, therefore there is nothing to query for

rpms_to_build(rpms)

Get list of koji build objects for the rpms. Order of koji objects in this list is the same as order of the respective rpm objects.

Parameters:rpms (list of str) – list of filenames as either /my/path/nvr.a.rpm or nvr.a.rpm Returns:list of Koji buildinfo dictionaries (as returned e.g. from koji.getBuild()) in the same respective order as in``rpms`` Return type:list Raises TaskotronRemoteError:  if rpm or it’s related build is not found

libtaskotron.ext.fedora.koji_utils.getNEVR(build)

Extract RPM version identifier in NEVR format from Koji build object

Parameters:build (dict) – Koji buildinfo dictionary (as returned e.g. from koji.getBuild()) Returns:NEVR string; epoch is included when non-zero, otherwise omitted Raises TaskotronValueError:  if build is of incorrect type

logger

Configure logging in libtaskotron.

There are two modes how to operate - as an external library, or as the main script runner:

• In the external library mode, we try not to change any global defaults, not touch the root logger, and not attach any handlers. The main script author should be in control of all these things.
• In the main script runner mode, we control everything - we configure the root logger, attach handlers to it, and set verbosity of this and any other library as we see fit.

libtaskotron.logger.add_filehandler(level_file=None, filelog_path=None, remove_mem_handler=False)

Add a file handler.

Parameters:

• level_file (int) – level of file logging as defined in logging. If None, a default level from config file is used.
• filelog_path (str) – path to the log file. If None, the value is loaded from config file.
• remove_mem_handler (bool) – after adding last file handler, the memory handler is no longer needed and can be removed so its buffer is not filling with messages

Returns:

created file handler

Raises IOError:

if log file can’t be opened for writing

libtaskotron.logger.init(level_stream=None, stream=True, syslog=False)

Initialize Taskotron logging.

Note: Since this touches the root logger, it should be called only when Taskotron is run as the main program (through its runner), not when it is used as a library.

Parameters:

• level_stream (int) – level of stream logging as defined in logging. If None, a default level from config file is used.
• stream (bool) – enable logging to process stream (stderr)
• syslog (bool) – enable logging to syslog

libtaskotron.logger.init_prior_config(level_stream=None)

Initialize Taskotron logging with default values which do not rely on a config file. Only stream logging is enabled here. This is used before the config file is loaded. After that a proper initialization should take place through the init() method.

Note: Since this touches the root logger, it should be called only when Taskotron is run as the main program (through its runner), not when it is used as a library.

Parameters:level_stream (int) – message level of the stream logger. The level definitions are in logging. If None, the default level is used (i.e. logging.NOTSET).

libtaskotron.logger.log = <logging.Logger object at 0x7fbd23b6ca10>

the main logger for libtaskotron library, easily accessible from all our modules

libtaskotron.logger.mem_handler = None

our current memory handler sending logged messages to memory log prior to creating file log (after its creation, content of the memory log is flushed into the file log)

libtaskotron.logger.stream_handler = None

our current stream handler sending logged messages to stderr

libtaskotron.logger.syslog_handler = None

our current syslog handler sending logged messages to syslog

main

libtaskotron.main.check_args(parser, args)

Check if passed args doesn’t have conflicts and have proper format. In case of error, this function prints error message and exits the program.

Parameters:

• parser (ArgumentParser) – parser object used to show error message and exit the program
• args (dict) – arguments previously returned by argument parser converted to dict

libtaskotron.main.process_args(args)

Processes raw input args and converts them into specific data types that can be used by tasks. This includes e.g. creating new args based on (item, item_type) pairs, or adjusting selected architecture.

Parameters:args (dict) – dictionary of raw arguments Returns:dict of args with appended specific data

minion

class libtaskotron.minion.BaseMinion(task_data, arg_data)

Base Minion class that shouldn’t be used on its own, it solely initiates inner attributes and environment. It also provides method _run() that delegates the task execution to Executor over SSH.

Create a new instance.

Parameters:

• task_data (dict) – data loaded from task formula
• arg_data (dict) – data from command line arguments

execute()

This method has to be implemented by classes that inherits from BaseMinion. It should contain environment initialization, namely set self.ssh to properly instantiated ParamikoWrapper and cleanup (if necessarily). Moreover, the order of private methods has to be: _prepare_task(), _run() and _get_output().

exitcode = None

int exitcode set by exitcode directive.

uuid_dir = None

str uuid_dir with path to artifactsdir for storing logs

class libtaskotron.minion.DisposableMinion(task_data, arg_data)

Minion class that creates a disposable client, connects to it over SSH and runs the task there.

Create a new instance.

Parameters:

• task_data (dict) – data loaded from task formula
• arg_data (dict) – data from command line arguments

execute()

Init environment and connect to disposable client for task execution.

class libtaskotron.minion.PersistentMinion(task_data, arg_data)

Minion class that connects to alredy running machine using SSH and runs the task there.

Create a new instance.

Parameters:

• task_data (dict) – data loaded from task formula
• arg_data (dict) – data from command line arguments

execute()

Init environment and connect to remote machine for task execution.

overlord

class libtaskotron.overlord.Overlord(arg_data)

Overlord class encapsulates decision whether the task is run locally or remotely (in persistent or disposable client) and orchestrates the execution.

Create new instance of Overlord.

Parameters:arg_data (dict) – data from command line arguments

exitcode = None

int exitcode contains the exitcode of task, if None, method start() was not called yet.

start()

Start the overlord, get runner and execute the task (either locally or remotely).

python_utils

A collection of convenience methods related to Python base libraries.

libtaskotron.python_utils.iterable(obj, item_type=None)

Decide whether obj is an Iterable (you can traverse through its elements - e.g. list, tuple, set, even dict), but not basestring (which satisfies most collections’ requirements, but we don’t often want to consider as a collection). You can also verify the types of items in the collection.

Parameters:

• obj (any) – any object you want to check
• item_type (type) – all items in obj must be instances of this provided type. If None, no check is performed.

Returns:

whether obj is iterable but not a string, and whether obj contains only item_type items

Return type:

bool

Raises TaskotronValueError:  

if you provide invalid parameter value

libtaskotron.python_utils.sequence(obj, item_type=None, mutable=False)

This has the same functionality and basic arguments as iterable() (read its documentation), but decides whether obj is a Sequence (ordered and indexable collection - e.g. list or tuple, but not set or dict).

Parameters:mutable (bool) – if True, the obj must be a mutable sequence (e.g. list, but not tuple)

remote_exec

Tools for remote execution primary for disposable clients

class libtaskotron.remote_exec.ParamikoWrapper(hostname, port, username, key_filename, stdio_filename=None)

Wrapper for SSH communication using paramiko library

close()

Close open connections and files.

cmd(cmd)

Execute a command.

Parameters:cmd (str) – A command to be executed Returns:returncode of the command Raises TaskotronRemoteError:  If the command has non-zero return code and it isn’t a code of exitcode directive failure. Or if the remote hasn’t sent any data for TIMEOUT seconds.

connect()

Connect to a machine over ssh. Open sftp channel and, if applicable, file that stdout/err from the machine will be saved to.

Raises TaskotronRemoteError:  when the connection does not succeed

get_dir(remote_path, local_path)

Get a directory from a remote path.

Parameters:

• remote_path (str) – A path to the remote directory
• local_path (str) – A path to the local directory

Raises TaskotronRemoteError:  

If the directory could not be downloaded

get_file(remote_path, local_path)

Get a file from a remote path.

Parameters:

• remote_path (str) – A path to the remote file
• local_path (str) – A path to the local file

Raises TaskotronRemoteError:  

If the file could not be downloaded

install_pkgs(pkgs)

Install packages via dnf.

Parameters:pkgs (list) – A list of packages to be installed Raises TaskotronRemoteError:  If the command has non-zero return code

mkdir(name, mode=777)

Create a directory on a remote machine.

Parameters:

• name (str) – A name of the directory
• mode (int) – Permissions (posix-style, decimal) of the directory, default is 777

Raises TaskotronRemoteError:  

If the directory could not be created

put_dir(local_path, remote_path, overwrite=True)

Copy a directory to a remote path.

Parameters:

• remote_path (str) – A path to the remote directory
• local_path (str) – A path to the local directory
• overwrite (bool) – Whether to overwrite remote path. Default is True.

Raises TaskotronRemoteError:  

If the directory could not be uploaded

put_file(local_path, remote_path, overwrite=True)

Copy a file to a remote path.

Parameters:

• local_path (str) – A path to the local file
• remote_path (str) – A path to the remote file
• overwrite (bool) – Whether to overwrite remote path. Default is True.

Returns:

an SFTPAttributes object containing attributes about the given file, if successful. None otherwise.

Raises TaskotronRemoteError:  

If the file could not be copied

write_file(remote_path, data, overwrite=True)

Write data to a remote file.

Parameters:

• remote_path (str) – A path to the remote file
• data (str) – Data to be written
• overwrite (bool) – Whether to overwrite remote path. Default is True.

Raises TaskotronRemoteError:  

If data could not be written

rpm_utils

Utility methods related to RPM

libtaskotron.ext.fedora.rpm_utils.cmpNEVR(nevr1, nevr2)

Compare two RPM version identifiers in NEVR format.

Parameters:

• nevr1 (str) – RPM identifier in N(E)VR format
• nevr2 (str) – RPM identifier in N(E)VR format

Returns:

-1/0/1 if nevr1 < nevr2 / nevr1 == nevr2 / nevr1 > nevr2

Return type:

int

Raises TaskotronValueError:  

if name in nevr1 doesn’t match name in nevr2

libtaskotron.ext.fedora.rpm_utils.get_dist_tag(rpmstr)

Parse disttag from an RPM package version string.

Parameters:rpmstr (str) – string to be manipulated in a format of N(E)VR (foo-1.2-3.fc20 or bar-4:1.2-3.fc20) or N(E)VRA (foo-1.2-3.fc20.x86_64 or bar-4:1.2-3.fc20.i686) Returns:string containing dist tag (fc20) Raises TaskotronValueError:  if rpmstr does not contain dist tag

libtaskotron.ext.fedora.rpm_utils.install(pkgs)

Install packages from system repositories using DNF. You need to have enough permissions to run dnf through sudo.

Parameters:pkgs (list of str) – packages to be installed, e.g. ['pidgin'], or any other specifier supported by dnf install command Raises TaskotronError:  if DNF returncode is not zero

libtaskotron.ext.fedora.rpm_utils.is_installed(pkgs)

Check if packages are installed using DNF. No additional permissions needed.

Parameters:pkgs (list of str) – packages to be checked whether they are installed, e.g. ['pidgin'], or any other specifier supported by dnf install command Returns:True if all specified packages are installed, False otherwise Return type:bool

libtaskotron.ext.fedora.rpm_utils.rpmformat(rpmstr, fmt='nvr', end_arch=False)

Parse and convert an RPM package version string into a different format. String identifiers: N - name, E - epoch, V - version, R - release, A - architecture.

Parameters:

• rpmstr (str) – string to be manipulated in a format of N(E)VR (foo-1.2-3.fc20 or bar-4:1.2-3.fc20) or N(E)VRA (foo-1.2-3.fc20.x86_64 or bar-4:1.2-3.fc20.i686)
• fmt (str) – desired format of the string to be returned. Allowed options are: nvr, nevr, nvra, nevra, n, e, v, r, a. If arch is not present in rpmstr but requested in fmt, noarch is used. Epoch is provided only when specifically requested (e.g. fmt='nevr') and being non-zero; otherwise it’s supressed (the only exception is fmt='e', where you receive 0 for zero epoch).
• end_arch (bool) – set this to True if rpmstr ends with an architecture identifier (foo-1.2-3.fc20.x86_64). It’s not possible to reliably distinguish that case automatically.

Returns:

string based on the specified format, or integer if fmt='e'

Raises TaskotronValueError:  

if fmt value is not supported

taskformula

Methods for operating with a task formula

libtaskotron.taskformula.replace_vars_in_action(action, variables)

Find all variables that are leaves (in a tree sense) in an action. Leaves are variables which can no longer be traversed, i.e. “primitive” types like str or int. Non-leaves are containers like dict or list.

For all leaves, call _replace_vars() and update their value with the function’s output.

Parameters:

• action (dict) – An action specification parsed from the task formula. See Runner.do_actions() to see what an action looks like.
• variables (dict) – names (keys) and values of variables to replace

Raises TaskotronYamlError:  

if text contains a variable that is not present in variables, or if the variable syntax is incorrect

vm

Interface to locally spawned virtual machines that are used as disposable clients for executing Taskotron tasks.

class libtaskotron.ext.disposable.vm.Image

Retrieve images from either local or remote sources

classmethod get_all_filenames(imagesdir=None)

Get list of images present on the system.

Parameters:imagesdir – absolute path to directory containing the images, path from config is used if None

classmethod get_latest(release, flavor, arch='x86_64', imagesdir=None)

Search for the most recent image available on the system.

Parameters:

• release – release of the image (e.g. 23)
• flavor – flavor of the image (e.g. minimal)
• imagesdir – absolute path to directory containing the images, path from config is used if None
• arch – architecture of the image

Returns:

testcloud.image.Image instance

Raises TaskotronImageError:  

if no such image for given release and flavor was found

classmethod get_latest_metadata(release, flavor, arch='x86_64', imagesdir=None)

Search for the most recent image available on the system.

Parameters:

• release – release of the image (e.g. 23)
• flavor – flavor of the image (e.g. minimal)
• imagesdir – absolute path to directory containing the images, path from config is used if None
• arch – arch of the image (e.g. ‘x86_64’)

Returns:

metadata of the most recent image

Return type:

dict {‘date’: str, ‘version’: str, ‘release’: str, ‘arch’: str, ‘filename’: str}

class libtaskotron.ext.disposable.vm.TestCloudMachine(uuid)

Launch virtual machines with TestCloud and prepare them for executing tasks (install packages etc.)

Parameters:uuid – unicode string uuid for the task being executed

hostname = None

hostname to use for spawned instance - based on username of current user

instancename = None

name of the testcloud instance spawned for this task

ipaddr = None

ip address of the remote machine (only set after prepared and spawned successfully)

password = None

password for user on the remote machine

prepare()

Prepare a virtual machine for running tasks.

Raises TaskotronRemoteError:  if there are any errors while preparing the virtual machine.

teardown()

Tear down the virtual machine by shutting it down and removing any trace of it from the host machine.

Raises TaskotronRemoteError:  if there is a failure while stopping or removing the virtual machine instance

username = None

username to use when connecting to the virtual machine

uuid = None

uuid of the task which spawned this instance

wait_for_port(port, timeout=60)

Wait until port is open. Repeatedly tries to socket.connect on given port.

Parameters:

• port – port to check
• timeout – timeout in seconds

Raises TaskotronInstanceError:  

when timeouted

yumrepoinfo

A wrapper object for yumrepoinfo.conf to access its information easily

class libtaskotron.ext.fedora.yumrepoinfo.YumRepoInfo(arch=None, filelist=None)

This class is a wrapper for easily accessing repoinfo.conf file.

Parameters:

• arch (str) – architecture for which to adjust repo URLs. By default it refers to the architecture of the current machine. It’s always converted to basearch.
• filelist (iterable of str) – list of config files to read information from. The first available config file is used. If None, then the default list of locations is used.

Raises TaskotronConfigError:  

if no YUM repositories data is found (empty or non-existent config file). It’s not raised if you specifically request no data to load (filelist=[]).

arches(reponame)

Get the list of all supported architectures for a repo

Parameters:reponame (str) – repository name

branched()

Get branched Fedora release (or None if it doesn’t exist).

Return type:str or None

get(reponame, key)

Get a specific key value from a repo

Parameters:

• reponame (str) – repository name
• key (str) – name of the key you want to retrieve from a repository (section)

Raises TaskotronConfigError:  

if the key can’t be retrieved (e.g. wrong key or repo name)

release_status(reponame)

Return release status of specified repo. For non-top-parent repos, return release_status of top parent repo.

Parameters:reponame (str) – repository name Returns:release status of specified repo, lowercased. One of: rawhide, branched, stable, obsolete. Return type:str

releases()

Get the list of stable (supported) Fedora releases.

Return type:list of str

repo(reponame)

Given a repo name, return the yumrepoinfo dict with keys: arches, parents, tag, url, path and name

Parameters:reponame (str) – repository name Return type:dict

repo_by_tag(tag)

Given a Koji tag, return the corresponding repo dict.

Parameters:tag (str) – a koji tag, e.g. f20-updates. Note: rawhide is not used as a Koji tag, use number identifier instead or use repo('rawhide')['tag'] to discover it first. Returns:repo dict as from repo(), or None if no such Koji tag is found

repos()

Get the list of all known repository names.

Return type:list of str

top_parent(reponame)

Go through the repo hiearchy and find the top parent for a repo

Parameters:reponame (str) – repository name Returns:the top parent reponame. If reponame doesn’t have any parent, its name is returned (it’s its own top parent) Return type:str Raises TaskotronConfigError:  if infinite parent loop detected

libtaskotron.ext.fedora.yumrepoinfo.get_yumrepoinfo(arch=None, filelist=None)

Get YumRepoInfo instance. This method is implemented using the singleton pattern - you will always receive the same instance, which will get auto-initialized on the first method call.

Parameters:

• arch (str) – architecture to return the YumRepoInfo for. It’s always converted to basearch. If None, then local machine arch is used.
• filelist – list of config files to read information from. The first available config file is used. If None, then the default list of locations is used.

Returns:

shared YumRepoInfo instance

Raises TaskotronConfigError:  

if file config parsing and handling failed