testplan.testing.multitest.entries package

Subpackages

Submodules

testplan.testing.multitest.entries.assertions module

Class based assertions, these will be serialized into native dicts via marshmallow schemas.

An assertion object will call evaluate on instantiation and will use the result of that call to set its passed attribute.

class testplan.testing.multitest.entries.assertions.Assertion(description: str | None = None, category: str | None = None, flag: str | None = None)[source]

Bases: BaseEntry

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.ColumnContain(table: Any, values: Any, column: str, limit: int = 0, report_fails_only: bool = False, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

Checks if the any of the value in values exists in the column of table.

data: List[ColumnContainComparison]
evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Contain(member: Any, container: Any, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.DictCheck(dictionary: Dict[Any, Any], has_keys: List[Hashable] | None = None, absent_keys: List[Hashable] | None = None, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

Assertion that checks if a given dict contains (or does not contain) given keys.

absent_keys_diff: Any
evaluate() → bool[source]
has_keys_diff: Any
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.DictMatch(value: ~typing.Dict[~typing.Any, ~typing.Any], expected: ~typing.Dict[~typing.Any, ~typing.Any], include_only_expected: bool = False, include_keys: ~typing.List[~typing.Hashable] | None = None, exclude_keys: ~typing.List[~typing.Hashable] | None = None, report_mode: ~typing.Any = ReportOptions.ALL, description: str | None = None, category: str | None = None, actual_description: str | None = None, expected_description: str | None = None, value_cmp_func: ~typing.Callable[[...], ~typing.Any] = <built-in function eq>)[source]

Bases: Assertion

Match two dictionaries by comparing values under each key recursively.

comparison: List[List[Any]] | None
evaluate() → bool[source]

Evaluate the dict match.

meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.DictMatchAll(values: ~typing.List[~typing.Dict[~typing.Any, ~typing.Any]], comparisons: ~typing.Any, key_weightings: ~typing.Dict[str, int] | None = None, description: str | None = None, category: str | None = None, value_cmp_func: ~typing.Callable[[...], ~typing.Any] = <built-in function eq>)[source]

Bases: Assertion

evaluate() → bool[source]
matches: List[Dict[str, Any]] | None
meta_type = 'assertion'
property name

MyClass -> My Class

result: Any
serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Equal(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: FuncAssertion

evaluate() → bool
func(a, b, /)

Same as a == b.

label: str = '=='
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.EqualExcludeSlices(actual: Sequence[Any], expected: Sequence[Any], slices: List[slice] | Tuple[slice, ...], description: str | None = None, category: str | None = None)[source]

Bases: EqualSlices

Assertion that checks if the items that are outside slices of two iterables match.

Generates a list of SliceComparison objects as data.

data: List[SliceComparison]
evaluate() → bool[source]

Slice exclusion evaluation generates SliceComparison data and explicitly checks if items in the merged exclusion indices match or not.

generate_data(slices: List[slice] | Tuple[slice, ...], actual: Sequence[Any], expected: Sequence[Any]) → List[SliceComparison]

Build a list of SliceComparison objects, for each slice.

get_comparison_indices(slice_obj: slice, iterable: Sequence[Any]) → range | Set[int][source]

Generate a list of indices to be used for comparison for the given slice and iterable.

get_iterable(iterable: Sequence[Any], comparison_indices: range | Set[int]) → Any

Generate the iterable that is being used for the current slice comparison

included_indices: Set[int]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.EqualSlices(actual: Sequence[Any], expected: Sequence[Any], slices: List[slice] | Tuple[slice, ...], description: str | None = None, category: str | None = None)[source]

Bases: Assertion

Assertion that checks if the given slices of two iterables match. Generates a list of SliceComparison objects as data.

data: List[SliceComparison]
evaluate() → bool[source]

Equal slices assertion passes if all slice comparisons pass.

generate_data(slices: List[slice] | Tuple[slice, ...], actual: Sequence[Any], expected: Sequence[Any]) → List[SliceComparison][source]

Build a list of SliceComparison objects, for each slice.

get_comparison_indices(slice_obj: slice, iterable: Sequence[Any]) → range | Set[int][source]

Generate a list of indices to be used for comparison for the given slice and iterable.

get_iterable(iterable: Sequence[Any], comparison_indices: range | Set[int]) → Any[source]

Generate the iterable that is being used for the current slice comparison

included_indices: Set[int]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.ExceptionRaised(raised_exception: BaseException | None, expected_exceptions: Any, pattern: str | None = None, func: Callable[[...], bool] | None = None, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

TODO

evaluate() → bool[source]
exception_match: bool | None
func_match: Any
get_match_context() → Tuple[bool, Any, Any][source]
meta_type = 'assertion'
property name

MyClass -> My Class

pattern_match: Any
serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Fail(description: str | None = None, category: str | None = None, flag: str | None = None, message: Any = None)[source]

Bases: Assertion

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FixCheck(msg: Dict[Any, Any], has_tags: List[Hashable] | None = None, absent_tags: List[Hashable] | None = None, description: str | None = None, category: str | None = None)[source]

Bases: DictCheck

Similar to DictCheck, however dict keys will have fix tag info popups on web UI

absent_keys_diff: Any
evaluate() → bool
has_keys_diff: Any
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FixMatch(value: Dict[Any, Any], expected: Dict[Any, Any], include_only_expected: bool = False, include_tags: List[Hashable] | None = None, exclude_tags: List[Hashable] | None = None, report_mode: Any = ReportOptions.ALL, description: str | None = None, category: str | None = None, actual_description: str | None = None, expected_description: str | None = None)[source]

Bases: DictMatch

Similar to DictMatch, however dict keys will have fix tag info popups on web UI

comparison: List[List[Any]] | None
evaluate() → bool

Evaluate the dict match.

meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FixMatchAll(values: List[Dict[Any, Any]], comparisons: Any, tag_weightings: Dict[str, int] | None = None, description: str | None = None, category: str | None = None)[source]

Bases: DictMatchAll

Similar to DictMatchAll, however dict keys will have fix tag info popups on web UI

evaluate() → bool
matches: List[Dict[str, Any]] | None
meta_type = 'assertion'
property name

MyClass -> My Class

result: Any
serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.FuncAssertion(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

evaluate() → bool[source]
func: Callable[[...], Any] | None = None
label: str
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Greater(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: FuncAssertion

evaluate() → bool
func(a, b, /)

Same as a > b.

label: str = '>'
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.GreaterEqual(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: FuncAssertion

evaluate() → bool
func(a, b, /)

Same as a >= b.

label: str = '>='
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.IsClose(first: Any, second: Any, rel_tol: Any = 1e-09, abs_tol: Any = 0.0, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

evaluate() → bool[source]
label = '~='
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.IsFalse(expr: Any, description: str | None = None, category: str | None = None)[source]

Bases: IsTrue

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.IsTrue(expr: Any, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.Less(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: FuncAssertion

evaluate() → bool
func(a, b, /)

Same as a < b.

label: str = '<'
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.LessEqual(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: FuncAssertion

evaluate() → bool
func(a, b, /)

Same as a <= b.

label: str = '<='
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.LineDiff(first: str | List[str], second: str | List[str], ignore_space_change: bool = False, ignore_whitespaces: bool = False, ignore_blank_lines: bool = False, unified: bool | int = False, context: bool | int = False, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

Assertion that checks if 2 blocks of textual content have difference.

If difference found, generates a list of strings as data.

delta: List[str]
evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.LogfileMatch(timeout: float, results: List[tuple], failure: tuple | None, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

NOTE: this structure was designed for multiple regexes matching, NOTE: which will be implemented in the future

class ResultRep(matched: str | None, pattern: str, start_pos: str, end_pos: str)[source]

Bases: object

end_pos: str
matched: str | None
pattern: str
start_pos: str
evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.NotContain(member: Any, container: Any, description: str | None = None, category: str | None = None)[source]

Bases: Contain

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.NotEqual(first: Any, second: Any, description: str | None = None, category: str | None = None)[source]

Bases: FuncAssertion

evaluate() → bool
func(a, b, /)

Same as a != b.

label: str = '!='
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RawAssertion(passed: bool, content: Any, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

This class is used for creating explicit pass/fail entries with custom content.

Its content will be displayed preformatted, so it’s useful for integration with 3rd party testing libraries (unittest, qunit etc).

evaluate() → bool[source]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexAssertion(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

evaluate() → bool[source]
get_regex_result() → Match[Any] | None[source]
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexFindIter(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, condition: Callable[[int], bool] | None = None, description: str | None = None, category: str | None = None)[source]

Bases: RegexAssertion

condition_match: bool | None
evaluate() → bool[source]
get_regex_result() → Match[Any] | None
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexMatch(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, description: str | None = None, category: str | None = None)[source]

Bases: RegexAssertion

evaluate() → bool
get_regex_result() → Match[Any] | None[source]
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexMatchLine(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, description: str | None = None, category: str | None = None)[source]

Bases: RegexAssertion

Match indexes are a little bit different than other assertions for this one: (line_no, begin, end)

evaluate() → bool[source]
get_regex_result() → Match[Any] | None
lines: Any
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexMatchNotExists(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, description: str | None = None, category: str | None = None)[source]

Bases: RegexMatch

evaluate() → bool[source]
get_regex_result() → Match[Any] | None
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexSearch(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, description: str | None = None, category: str | None = None)[source]

Bases: RegexAssertion

evaluate() → bool
get_regex_result() → Match[Any] | None[source]
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.RegexSearchNotExists(regexp: str | bytes | Pattern[Any], string: str | bytes, flags: int = 0, description: str | None = None, category: str | None = None)[source]

Bases: RegexSearch

evaluate() → bool[source]
get_regex_result() → Match[Any] | None
match_indexes: List[Tuple[int, ...]]
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.TableDiff(table: Any, expected_table: Any, include_columns: List[str] | None = None, exclude_columns: List[str] | None = None, report_all: bool = True, fail_limit: int = 0, report_fail_only: bool = False, strict: bool = False, description: str | None = None, category: str | None = None)[source]

Bases: TableMatch

Match two tables using compare_rows but only keep failing comparisons, may generate custom message if tables cannot be compared for certain reasons.

data: List[RowComparison]
display_columns: List[str]
evaluate() → bool
message: str | None
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.TableMatch(table: Any, expected_table: Any, include_columns: List[str] | None = None, exclude_columns: List[str] | None = None, report_all: bool = True, fail_limit: int = 0, report_fail_only: bool = False, strict: bool = False, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

Match two tables using compare_rows, may generate custom message if tables cannot be compared for certain reasons.

data: List[RowComparison]
display_columns: List[str]
evaluate() → bool[source]
message: str | None
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.assertions.XMLCheck(element: Any, xpath: str, tags: List[Any] | None = None, namespaces: Dict[str, str] | None = None, description: str | None = None, category: str | None = None)[source]

Bases: Assertion

Validate XML tag texts or existence in a given xpath, supports regex patterns as tag values as well.

data: List[XMLTagComparison]
evaluate() → bool[source]
message: str | None
meta_type = 'assertion'
property name

MyClass -> My Class

serialize()

Shortcut method for serialization via schemas

testplan.testing.multitest.entries.base module

Base classes go here.

class testplan.testing.multitest.entries.base.BaseEntry(description, category=None, flag=None)[source]

Bases: object

Base class for all entries, stores common context like time etc.

meta_type = 'entry'
property name

MyClass -> My Class

serialize()[source]

Shortcut method for serialization via schemas

class testplan.testing.multitest.entries.base.Group(entries, description=None)[source]

Bases: object

meta_type = 'assertion'
property passed

Empty groups are truthy AKA does not contain anything that is failing.

class testplan.testing.multitest.entries.base.Log(message, description=None, flag=None)[source]

Bases: BaseEntry

Log a str to the report.

class testplan.testing.multitest.entries.base.Summary(entries, description=None, num_passing=5, num_failing=5, key_combs_limit=10)[source]

Bases: Group

A meta assertion that stores a subset of given entries. Groups assertion data into a nested structure by category, assertion type and pass/fail status.

If any of the entries is a Group, then its entries are expanded and the Group object is discarded.

testplan.testing.multitest.entries.summarization module

class testplan.testing.multitest.entries.summarization.SummaryRegistry[source]

Bases: Registry

Records are done by class.__name__ Lookups are done by class_name (str)

bind(*classes)

Decorator for binding one or more classes to another.

Parameters:

classes – One or more classes that will be bound to the decorated class.

bind_default(category=None)

Decorator for binding a class as category based or absolute default.

Parameters:

category – (optional) If provided, the decorated class will be the default for the given category, otherwise it will be the absolute default.

property default
get_category(obj)[source]

No category defaults support for now

get_lookup_key(obj)[source]

This method is used for generating the key when do a lookup from the registry. Object class is used by default.

get_record_key(obj)[source]

This method is used for generating the key when we bind an object (possibly a class) via the registry.

property logger: TestplanLogger

logger object

summarize(class_name, entries, limit)[source]
testplan.testing.multitest.entries.summarization.dict_failed_keys(table)[source]

Returns all failed keys of the dict match comparison result table.

testplan.testing.multitest.entries.summarization.summarize_dict_match(category, class_name, passed, entries, limits)[source]

Summarized for FixMatch/DictMatch

Uses default summary logic for passing entries, further groups failing entries by failed tags/keys.

testplan.testing.multitest.entries.summarization.summarize_entries(category, class_name, passed, entries, limits)[source]

Default summary function, just trims entries using the given limit.

Module contents

TODO.