testplan.common.report package
Submodules
testplan.common.report.base module
Base classes for building a report tree.
The idea behind this module is that we may have parallel runners for a testplan, each of which would generate a partial report.
Later on these reports would be merged together to build the final report as the testplan result.
- class testplan.common.report.base.BaseReportGroup(name, **kwargs)[source]
Bases:
Report
A report class that contains other Reports or ReportGroups in entries. Allows O(1) child report lookup via get_by_uid method.
- build_index(recursive=False)[source]
Build (refresh) indexes for this report and optionally for each child report.
This should be called explicitly if self.entries is changed.
- Parameters:
recursive (
bool
) – Flag for triggering index build on children.
- property counter: Counter
Return counts for each status, will recursively get aggregates from children and so on.
- property entry_uids
Return the UIDs of all entries in this report group.
- exception_logger
alias of
ExceptionLogger
- filter_by_tags(tag_value, all_tags=False) Self [source]
Shortcut method for filtering the report by given tags.
- filter_cases(predicate: Callable[[BaseReportGroup], bool], is_root: bool = False) Self [source]
Case-level filter with status retained
- flatten(depths=False)[source]
Depth-first traverse the report tree starting on the leftmost node (smallest index), return a list of (depth, obj) tuples or a list of reports depending on depths flag.
- Parameters:
depths – Flag for enabling/disabling depth data in result.
- Returns:
List of reports or list of (depth, report) tuples.
- property flattened_logs
Return a flattened list of the logs from each Report.
- get_by_uid(uid: str) Report [source]
Get child report via uid lookup.
- Parameters:
uid (
hashable
) – uid for the child report.
- get_by_uids(uids: List[str]) Report [source]
Get child report via a series of uid lookup.
- Parameters:
uids (
list
ofhashable
) – A uid for the child report.
- property hash: int
Generate a hash of this report object, including its entries. This hash is used to detect when changes are made under particular nodes in the report tree. Since all report entries are mutable, this hash should NOT be used to index the report entry in a set or dict - we have avoided using the magic __hash__ method for this reason. Always use the UID for indexing purposes.
- Returns:
a hash of all entries in this report group.
- Return type:
int
- merge_entries(report, strict=True)[source]
For report groups, we call merge on each child report and later merge basic attributes. With default strict=True,
MergeError
will be raise if uid does not match. However sometimes original report may not have matching child reports. For example this happens when the test target’s test cases cannot be inspected on initialization time (e.g. GTest). In this case we can merge with strict=False so that child reports are directly appended to original report’s entries.
- reset_uid(uid=None)[source]
Reset uid of test report and all of its children, it can be useful when need to generate global unique id for each report entry before saving, by default strings in standard UUID format will be applied.
- property runtime_status: RuntimeStatus
The runtime status is used for interactive running, and reports whether a particular entry is READY, WAITING, RUNNING, RESETTING, FINISHED or NOT_RUN.
A test group inherits its runtime status from its child entries.
- set_by_uid(uid, item)[source]
Set child report via uid lookup.
If an entry with a matching UID is already present, that entry is updated. Otherwise a new entry will be added.
- Parameters:
uid (
hashable
) – uid for the child report.item (
Report
) – entry to update or insert into the report.
- set_parent_uids(item)[source]
Set the parent UIDs recursively of an item and its child entries after it has been added into this report group.
- set_runtime_status_filtered(new_status: str, entries: Dict) None [source]
Alternative setter for the runtime status of an entry. Propagates only to the specified entries.
- Parameters:
new_status – new runtime status to be set
entries – tree-like structure of entries names
- class testplan.common.report.base.ExceptionLogger(*exception_classes, **kwargs)[source]
Bases:
ExceptionLoggerBase
When we run tests, we always want to return a report object, However we also want to mark the test as failed if an exception is raised (unless kwargs[‘fail’] is False).
- class testplan.common.report.base.ExceptionLoggerBase(*exception_classes, **kwargs)[source]
Bases:
object
A context manager used for suppressing & logging an exception.
- exception testplan.common.report.base.MergeError[source]
Bases:
Exception
Raised when a merge operation fails.
- class testplan.common.report.base.Report(name: str, description: str | None = None, definition_name: str | None = None, uid: str | None = None, entries: list | None = None, parent_uids: List[str] | None = None, status_override: Status | None = None, status_reason: str | None = None)[source]
Bases:
object
Base report class, support exception suppression & logging via report.logged_exceptions. Stores arbitrary objects in entries attribute.
- exception_logger
alias of
ExceptionLoggerBase
- property failed: bool
Shortcut for checking if report status should be considered failed.
- filter(*functions, **kwargs)[source]
Filtering report’s entries in place using the given functions. If any of the functions return
True
for a given entry, it will be kept.
- flattened_entries(depth)[source]
Utility function that is used by TestGroupReport.flatten.
This method should be overridden if entries have a custom hierarchy instead of a simple list.
- property hash
Return a hash of all entries in this report.
- inherit(deceased: Self) Self [source]
Inherit certain information from the old report, mainly for information preservation across interactive mode reloads.
- logged_exceptions(*exception_classes, **kwargs)[source]
Wrapper around ExceptionRecorder, passing report arg implicitly.
Basic usage:
with report.logged_exceptions(TypeError, ValueError): raise some errors here ...
- merge(report, strict=True)[source]
Child classes can override this, just make sure merge operation is idempotent, so that merging the same report onto self multiple times does not create extra data.
- Parameters:
report (
Report
) – Report that contains logs to merge.strict (
bool
) – Flag for enabling / disabling strict merge ops.
- property passed: bool
Shortcut for getting if report status should be considered passed.
- reset_uid(uid=None)[source]
Reset uid of the report, it can be useful when need to generate a global unique id instead of the current one.
- property runtime_status: RuntimeStatus
Used for interactive mode, the runtime status of a testcase will be one of
RuntimeStatus
.
- property unknown: bool
Shortcut for checking if report status is unknown.
- property unstable: bool
Shortcut for checking if report status should be considered unstable.
- class testplan.common.report.base.ReportCategories[source]
Bases:
object
Some possible categories of report nodes, grouped as easy-to-use constants.
Test
is extensible, so shouldReportCategories
be.- BOOST_SUITE = 'boost-suite'
- BOOST_TEST = 'boost-test'
- CPPUNIT = 'cppunit'
- CPPUNIT_SUITE = 'cppunit-suite'
- ERROR = 'error'
- GTEST = 'gtest'
- GTEST_SUITE = 'gtest-suite'
- HOBBESTEST = 'hobbestest'
- HOBBESTEST_SUITE = 'hobbestest-suite'
- JUNIT = 'junit'
- MULTITEST = 'multitest'
- PARAMETRIZATION = 'parametrization'
- PYTEST = 'pytest'
- PYUNIT = 'pyunit'
- QUNIT = 'qunit'
- SYNTHESIZED = 'synthesized'
- TASK_RERUN = 'task_rerun'
- TESTCASE = 'testcase'
- TESTGROUP = 'testgroup'
- TESTPLAN = 'testplan'
- TESTSUITE = 'testsuite'
- UNITTEST = 'unittest'
- class testplan.common.report.base.RuntimeStatus(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)[source]
Bases:
Enum
Constants for test runtime status - for interactive mode
total order encoded in value, serialized to “lower-case string” of name
- FINISHED = 6
- NONE = 7
- NOT_RUN = 5
- READY = 4
- RESETTING = 2
- RUNNING = 1
- WAITING = 3
- precede(other: Self) bool
Return self<value.
- classmethod precedent(stats: List[Self]) Self [source]
Return the precedent status from a list of statuses, using the ordering of statuses in rule.
Note that the client can send RESETTING signal to reset the test report to its initial status, but client will not receive a temporary report containing RESETTING status, instead WAITING status is used and after reset the report goes to READY status.
- Parameters:
stats – List of statuses of which we want to get the precedent.
- exception testplan.common.report.base.SkipTestcaseException[source]
Bases:
Exception
Raised from an explicit call to result.skip.
- class testplan.common.report.base.Status(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)[source]
Bases:
Enum
Constants for test result and utilities for propagating status upward.
partial order encoded by value, serialized to “lower-case string” of name
tens of value encoding status category, tenths of value only for differentiating enum members
- BOTTOM = -1
- ERROR = 9
- FAILED = 19
- INCOMPLETE = 18.1
- NONE = 59
- PASSED = 39
- SKIPPED = 48.1
- UNKNOWN = 29
- UNSTABLE = 49
- XFAIL = 48.2
- XPASS = 48.3
- XPASS_STRICT = 18.2
testplan.common.report.log module
We’d like to use python’s logging interface when we log messages through Report objects. The most lightweight way to do it is to use a logging.LoggingAdapter which is a thin wrapper around a logging.Logger.
We can then use a global mapping and a custom handler to append log messages to the report object’s logs list.
testplan.common.report.schemas module
Base schemas for report serialization.
- class testplan.common.report.schemas.BaseReportGroupSchema(*, only: Sequence[str] | AbstractSet[str] | None = None, exclude: Sequence[str] | AbstractSet[str] = (), many: bool | None = None, context: dict | None = None, load_only: Sequence[str] | AbstractSet[str] = (), dump_only: Sequence[str] | AbstractSet[str] = (), partial: bool | Sequence[str] | AbstractSet[str] | None = None, unknown: str | None = None)[source]
Bases:
ReportSchema
Schema for
base.BaseReportGroup
.- opts: Any = <marshmallow.schema.SchemaOpts object>
- source_class
alias of
BaseReportGroup
- class testplan.common.report.schemas.ReportLogSchema(*, only: Sequence[str] | AbstractSet[str] | None = None, exclude: Sequence[str] | AbstractSet[str] = (), many: bool | None = None, context: dict | None = None, load_only: Sequence[str] | AbstractSet[str] = (), dump_only: Sequence[str] | AbstractSet[str] = (), partial: bool | Sequence[str] | AbstractSet[str] | None = None, unknown: str | None = None)[source]
Bases:
Schema
Schema for log record data created by report stdout.
- opts: Any = <marshmallow.schema.SchemaOpts object>
- class testplan.common.report.schemas.ReportSchema(*, only: Sequence[str] | AbstractSet[str] | None = None, exclude: Sequence[str] | AbstractSet[str] = (), many: bool | None = None, context: dict | None = None, load_only: Sequence[str] | AbstractSet[str] = (), dump_only: Sequence[str] | AbstractSet[str] = (), partial: bool | Sequence[str] | AbstractSet[str] | None = None, unknown: str | None = None)[source]
Bases:
TreeNodeSchema
Schema for
base.Report
.- opts: Any = <marshmallow.schema.SchemaOpts object>