testplan.testing.multitest.entries package

Subpackages

• testplan.testing.multitest.entries.schemas package
  • Submodules
    • testplan.testing.multitest.entries.schemas.assertions module
    • testplan.testing.multitest.entries.schemas.base module
  • Module contents
• testplan.testing.multitest.entries.stdout package
  • Submodules
    • testplan.testing.multitest.entries.stdout.assertions module
    • testplan.testing.multitest.entries.stdout.base module
  • Module contents

Submodules

testplan.testing.multitest.entries.assertions module

Class based assertions, these will be serialized into native dicts via marshmallow schemas.

An assertion object will call evaluate on instantiation and will use the result of that call to set its passed attribute.

class testplan.testing.multitest.entries.assertions.Assertion(description=None, category=None, flag=None)

Bases: testplan.testing.multitest.entries.base.BaseEntry

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RawAssertion(passed, content, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

This class is used for creating explicit pass/fail entries with custom content.

Its content will be displayed preformatted, so it’s useful for integration with 3rd party testing libraries (unittest, qunit etc).

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.IsTrue(expr, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.IsFalse(expr, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.IsTrue

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Fail(description=None, category=None, flag=None, message=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FuncAssertion(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

func = None

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Equal(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.FuncAssertion

evaluate()

func()

Same as a == b.

label = '=='

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.NotEqual(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.FuncAssertion

evaluate()

func()

Same as a != b.

label = '!='

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Less(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.FuncAssertion

evaluate()

func()

Same as a < b.

label = '<'

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.LessEqual(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.FuncAssertion

evaluate()

func()

Same as a <= b.

label = '<='

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Greater(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.FuncAssertion

evaluate()

func()

Same as a > b.

label = '>'

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.GreaterEqual(first, second, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.FuncAssertion

evaluate()

func()

Same as a >= b.

label = '>='

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.IsClose(first, second, rel_tol=1e-09, abs_tol=0.0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

label = '~='

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Contain(member, container, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.NotContain(member, container, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Contain

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexAssertion(regexp, string, flags=0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexMatch(regexp, string, flags=0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.RegexAssertion

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexMatchNotExists(regexp, string, flags=0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.RegexMatch

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexSearch(regexp, string, flags=0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.RegexAssertion

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexSearchNotExists(regexp, string, flags=0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.RegexSearch

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexFindIter(regexp, string, flags=0, condition=None, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.RegexAssertion

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexMatchLine(regexp, string, flags=0, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.RegexAssertion

Match indexes are a little bit different than other assertions for this one: (line_no, begin, end)

evaluate()

get_regex_result()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.ExceptionRaised(raised_exception, expected_exceptions, pattern=None, func=None, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

TODO

evaluate()

get_match_context()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.EqualSlices(actual, expected, slices, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Assertion that checks if the given slices of two iterables match. Generates a list of SliceComparison objects as data.

evaluate()

Equal slices assertion passes if all slice comparisons pass.

generate_data(slices, actual, expected)

Build a list of SliceComparison objects, for each slice.

get_comparison_indices(slice_obj, iterable)

Generate a list of indices to be used for comparison for the given slice and iterable.

get_iterable(iterable, comparison_indices)

Generate the iterable that is being used for the current slice comparison

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.EqualExcludeSlices(actual, expected, slices, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.EqualSlices

Assertion that checks if the items that are outside slices of two iterables match.

Generates a list of SliceComparison objects as data.

evaluate()

Slice exclusion evaluation generates SliceComparison data and explicitly checks if items in the merged exclusion indices match or not.

generate_data(slices, actual, expected)

Build a list of SliceComparison objects, for each slice.

get_comparison_indices(slice_obj, iterable)

Generate a list of indices to be used for comparison for the given slice and iterable.

get_iterable(iterable, comparison_indices)

Generate the iterable that is being used for the current slice comparison

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.LineDiff(first, second, ignore_space_change=False, ignore_whitespaces=False, ignore_blank_lines=False, unified=False, context=False, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Assertion that checks if 2 blocks of textual content have difference.

If difference found, generates a list of strings as data.

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.ColumnContain(table, values, column, limit=0, report_fails_only=False, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Checks if the any of the value in values exists in the column of table.

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.TableMatch(table, expected_table, include_columns=None, exclude_columns=None, report_all=True, fail_limit=0, report_fail_only=False, strict=False, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Match two tables using compare_rows, may generate custom message if tables cannot be compared for certain reasons.

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.TableDiff(table, expected_table, include_columns=None, exclude_columns=None, report_all=True, fail_limit=0, report_fail_only=False, strict=False, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.TableMatch

Match two tables using compare_rows but only keep failing comparisons, may generate custom message if tables cannot be compared for certain reasons.

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.XMLCheck(element, xpath, tags=None, namespaces=None, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Validate XML tag texts or existence in a given xpath, supports regex patterns as tag values as well.

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.DictCheck(dictionary, has_keys=None, absent_keys=None, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Assertion that checks if a given dict contains (or does not contain) given keys.

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.DictMatch(value: Dict[KT, VT], expected: Dict[KT, VT], include_only_expected: bool = False, include_keys: List[Hashable] = None, exclude_keys: List[Hashable] = None, report_mode=<ReportOptions.ALL: 1>, description: str = None, category: str = None, actual_description: str = None, expected_description: str = None, value_cmp_func=<built-in function eq>)

Bases: testplan.testing.multitest.entries.assertions.Assertion

Match two dictionaries by comparing values under each key recursively.

evaluate()

Evaluate the dict match.

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.DictMatchAll(values, comparisons, key_weightings=None, description=None, category=None, value_cmp_func=<built-in function eq>)

Bases: testplan.testing.multitest.entries.assertions.Assertion

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FixCheck(msg, has_tags=None, absent_tags=None, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.DictCheck

Similar to DictCheck, however dict keys will have fix tag info popups on web UI

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FixMatch(value: Dict[KT, VT], expected: Dict[KT, VT], include_only_expected: bool = False, include_tags: List[Hashable] = None, exclude_tags: List[Hashable] = None, report_mode=<ReportOptions.ALL: 1>, description: str = None, category: str = None, actual_description: str = None, expected_description: str = None)

Bases: testplan.testing.multitest.entries.assertions.DictMatch

Similar to DictMatch, however dict keys will have fix tag info popups on web UI

evaluate()

Evaluate the dict match.

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FixMatchAll(values, comparisons, tag_weightings=None, description=None, category=None)

Bases: testplan.testing.multitest.entries.assertions.DictMatchAll

Similar to DictMatchAll, however dict keys will have fix tag info popups on web UI

evaluate()

meta_type = 'assertion'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

testplan.testing.multitest.entries.base module

Base classes go here.

class testplan.testing.multitest.entries.base.BaseEntry(description, category=None, flag=None)

Bases: object

Base class for all entries, stores common context like time etc.

meta_type = 'entry'

name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.base.Group(entries, description=None)

Bases: object

meta_type = 'assertion'

passed

Empty groups are truthy AKA does not contain anything that is failing.

class testplan.testing.multitest.entries.base.Summary(entries, description=None, num_passing=5, num_failing=5, key_combs_limit=10)

Bases: testplan.testing.multitest.entries.base.Group

A meta assertion that stores a subset of given entries. Groups assertion data into a nested structure by category, assertion type and pass/fail status.

If any of the entries is a Group, then its entries are expanded and the Group object is discarded.

class testplan.testing.multitest.entries.base.Log(message, description=None, flag=None)

Bases: testplan.testing.multitest.entries.base.BaseEntry

Log a str to the report.

testplan.testing.multitest.entries.summarization module

class testplan.testing.multitest.entries.summarization.SummaryRegistry

Bases: testplan.common.utils.registry.Registry

Records are done by class.__name__ Lookups are done by class_name (str)

bind(*classes)

Decorator for binding one or more classes to another.

Parameters:classes – One or more classes that will be bound to the decorated class.

bind_default(category=None)

Decorator for binding a class as category based or absolute default.

Parameters:category – (optional) If provided, the decorated class will be the default for the given category, otherwise it will be the absolute default.

default

get_category(obj)

No category defaults support for now

get_lookup_key(obj)

This method is used for generating the key when do a lookup from the registry. Object class is used by default.

get_record_key(obj)

This method is used for generating the key when we bind an object (possibly a class) via the registry.

logger

logger object

summarize(class_name, entries, limit)

testplan.testing.multitest.entries.summarization.dict_failed_keys(table)

Returns all failed keys of the dict match comparison result table.

testplan.testing.multitest.entries.summarization.summarize_dict_match(category, class_name, passed, entries, limits)

Summarized for FixMatch/DictMatch

Uses default summary logic for passing entries, further groups failing entries by failed tags/keys.

testplan.testing.multitest.entries.summarization.summarize_entries(category, class_name, passed, entries, limits)

Default summary function, just trims entries using the given limit.

Module contents

TODO.