testplan package¶
Subpackages¶
- testplan.common package
- Subpackages
- testplan.common.config package
- testplan.common.entity package
- testplan.common.exporters package
- testplan.common.report package
- testplan.common.serialization package
- testplan.common.utils package
- Subpackages
- Submodules
- testplan.common.utils.callable module
- testplan.common.utils.comparison module
- testplan.common.utils.context module
- testplan.common.utils.convert module
- testplan.common.utils.difflib module
- testplan.common.utils.exceptions module
- testplan.common.utils.helper module
- testplan.common.utils.interface module
- testplan.common.utils.logger module
- testplan.common.utils.match module
- testplan.common.utils.networking module
- testplan.common.utils.parser module
- testplan.common.utils.path module
- testplan.common.utils.process module
- testplan.common.utils.registry module
- testplan.common.utils.remote module
- testplan.common.utils.reporting module
- testplan.common.utils.strings module
- testplan.common.utils.table module
- testplan.common.utils.testing module
- testplan.common.utils.thread module
- testplan.common.utils.timing module
- testplan.common.utils.validation module
- Module contents
- testplan.common.remote package
- Module contents
- Subpackages
- testplan.exporters package
- Subpackages
- testplan.exporters.testing package
- Subpackages
- testplan.exporters.testing.json package
- testplan.exporters.testing.pdf package
- testplan.exporters.testing.xml package
- testplan.exporters.testing.http package
- testplan.exporters.testing.webserver package
- Submodules
- Module contents
- Subpackages
- testplan.exporters.testing package
- Module contents
- Subpackages
- testplan.testing package
- Subpackages
- testplan.testing.multitest package
- Subpackages
- testplan.testing.multitest.driver package
- testplan.testing.multitest.entries package
- Submodules
- Module contents
- Subpackages
- testplan.testing.cpp package
- testplan.testing.bdd package
- testplan.testing.multitest package
- Submodules
- Module contents
- Subpackages
- testplan.report package
- testplan.runners package
- testplan.runnable package
testplan.base module¶
Testplan base module.
-
class
testplan.base.
Testplan
(name: str, description: Optional[str] = None, parse_cmdline: bool = True, parser: Type[testplan.parser.TestplanParser] = <class 'testplan.parser.TestplanParser'>, interactive_port: Optional[int] = None, abort_signals: Optional[List[int]] = None, logger_level: int = 25, file_log_level: int = 10, runpath: Union[str, Callable] = <function default_runpath>, path_cleanup: bool = True, all_tasks_local: bool = False, shuffle: Optional[List[str]] = None, shuffle_seed: float = 8567.0, exporters: Optional[List[T]] = None, stdout_style: testplan.report.testing.styles.Style = Style(passing='testcase', failing='assertion-detail'), report_dir: str = '/home/docs/checkouts/readthedocs.org/user_builds/testplan/checkouts/latest/doc/en', xml_dir: Optional[str] = None, json_path: Optional[str] = None, http_url: Optional[str] = None, pdf_path: Optional[str] = None, pdf_style: testplan.report.testing.styles.Style = Style(passing='test', failing='test'), report_tags: Optional[List[T]] = None, report_tags_all: Optional[List[T]] = None, resource_monitor: bool = False, merge_scheduled_parts: bool = False, browse: bool = False, ui_port: Optional[int] = None, web_server_startup_timeout: int = 10, test_filter: Type[testplan.testing.filtering.BaseFilter] = <testplan.testing.filtering.Filter object>, test_sorter: Type[testplan.testing.ordering.BaseSorter] = <testplan.testing.ordering.NoopSorter object>, test_lister: Optional[testplan.testing.listing.MetadataBasedLister] = None, test_lister_output: Optional[os.PathLike] = None, verbose: bool = False, debug: bool = False, timeout: int = 14400, interactive_handler: Type[testplan.runnable.interactive.base.TestRunnerIHandler] = <class 'testplan.runnable.interactive.base.TestRunnerIHandler'>, extra_deps: Optional[List[Union[str, module]]] = None, label: Optional[str] = None, auto_part_runtime_limit: int = 1800, plan_runtime_target: int = 1800, **options)[source]¶ Bases:
testplan.common.entity.base.RunnableManager
A collection of tests and tests executors with the ability to selectively execute a subset or a shuffled set of those tests.
It manages a
TestRunner
to execute the tests and also accepts allRunnableManagerConfig
andTestRunnerConfig
options.Since it’s a manager of a TestRunner object, it exposes all
TestRunner
, attributes and methods likeadd_resource()
,add()
, andschedule()
.Parameters: - name – Name of test plan.
- description – Description of test plan.
- parse_cmdline – Parse command line arguments.
- parser – Command line parser.
- interactive_port – Enable interactive execution mode on a port.
- abort_signals – Signals to catch and trigger abort. By default, SIGINT and SIGTERM will trigger Testplan to abort.
- logger_level – Logger level for stdout.
- runpath – Input runpath.
- path_cleanup – Clean previous runpath entries.
- all_tasks_local – Schedule all tasks in local pool.
- shuffle – Shuffle strategy.
- shuffle_seed – Shuffle seed.
- exporters – Exporters for reports creation.
- stdout_style – Styling output options.
- report_dir – Report directory.
- xml_dir – XML output directory.
- json_path – JSON output path <PATH>/*.json.
- http_url – HTTP url to post JSON report.
- pdf_path – PDF output path <PATH>/*.pdf.
- pdf_style – PDF creation styling options.
- report_tags – Matches tests marked with any of the given tags.
- report_tags_all – Match tests marked with all of the given tags.
- resource_monitor – Enable resource monitor.
- merge_scheduled_parts – Merge reports of scheduled MultiTest parts.
- browse – Open web browser to display the test report.
- ui_port – Port of web server for displaying test report.
- web_server_startup_timeout – Timeout for starting web server.
- test_filter – Tests filtering class.
- test_sorter – Tests sorting class.
- test_lister – Tests listing class.
- test_lister_output – listing results goes to this file, if None goes to stdout
- verbose – Enable or disable verbose mode.
- debug – Enable or disable debug mode.
- timeout – Timeout value in seconds to kill Testplan and all child processes, default to 14400s(4h), set to 0 to disable.
- interactive_handler – Handler for interactive mode execution.
- extra_deps – Extra module dependencies for interactive reload, or paths of these modules.
- label – Label the test report with the given name, useful to categorize or classify similar reports .
Param: file_log_level: Logger level for file.
-
CONFIG
¶ alias of
TestplanConfig
-
STATUS
¶
-
abort
()¶ Default abort policy. First abort all dependencies and then itself.
-
abort_dependencies
()¶ Dependencies to be aborted first.
-
aborted
¶ Returns if entity was aborted.
-
aborting
()¶ Suppressing not implemented debug log by parent class.
-
active
¶ Expose the runnable active attribute.
-
args
¶ Parsed arguments.
-
cfg
¶ Expose the runnable configuration object.
-
context_input
(exclude: list = None) → Dict[str, Any]¶ All attr of self in a dict for context resolution
-
define_runpath
()¶ Define runpath directory based on parent object and configuration.
-
enrich_options
(options)[source]¶ Enrich the options using parsed command line arguments. The command line arguments will override any explicit programmatic declaration for a given keyword.
-
classmethod
filter_locals
(local_vars)¶ Filter out init params of None value, they will take default value defined in its ConfigOption object; also filter out special vars that are not init params from local_vars.
Parameters: local_vars –
-
logger
¶ logger object
-
classmethod
main_wrapper
(name, description=None, parse_cmdline=True, parser=<class 'testplan.parser.TestplanParser'>, interactive_port=None, abort_signals=None, logger_level=25, file_log_level=10, runpath=<function default_runpath>, path_cleanup=True, all_tasks_local=False, shuffle=None, shuffle_seed=590.0, exporters=None, stdout_style=Style(passing='testcase', failing='assertion-detail'), report_dir='/home/docs/checkouts/readthedocs.org/user_builds/testplan/checkouts/latest/doc/en', xml_dir=None, json_path=None, http_url=None, pdf_path=None, pdf_style=Style(passing='test', failing='test'), report_tags=None, report_tags_all=None, resource_monitor=False, merge_scheduled_parts=False, browse=False, ui_port=None, web_server_startup_timeout=10, test_filter=<testplan.testing.filtering.Filter object>, test_sorter=<testplan.testing.ordering.NoopSorter object>, test_lister=None, test_lister_output=None, verbose=False, debug=False, timeout=14400, interactive_handler=<class 'testplan.runnable.interactive.base.TestRunnerIHandler'>, extra_deps=None, label=None, auto_part_runtime_limit=1800, plan_runtime_target=1800, **options)[source]¶ Decorator that will be used for wrapping main methods in test scripts.
It accepts all arguments of a
Testplan
entity.
-
make_runpath_dirs
()¶ Creates runpath related directories.
-
parser
¶ Returns a new command line parser.
-
pause
()¶ Pauses entity execution.
-
pausing
()¶ Pause the runnable execution.
-
processed_args
¶ Processed parsed arguments.
-
resume
()¶ Resumes entity execution.
-
resuming
()¶ Resume the runnable execution.
-
run
()[source]¶ Runs the tests added and returns the result object. Also handles usr1 and usr2 signals.
Returns: Result containing tests and execution steps results. Return type: TestplanResult
-
runnable
¶ Runnable instance.
-
runpath
¶ Expose the runnable runpath.
-
scratch
¶ Path to be used for temp files by entity.
-
status
¶ Expose the runnable status.
-
uid
()¶ Unique identifier of self.
-
wait
(target_status, timeout=None)¶ Wait until objects status becomes target status.
Parameters: - target_status (
str
) – expected status - timeout (
int
orNoneType
) – timeout in seconds
- target_status (
-
class
testplan.base.
TestplanConfig
(**options)[source]¶ Bases:
testplan.common.entity.base.RunnableManagerConfig
,testplan.runnable.base.TestRunnerConfig
Configuration object for
Testplan
entity.-
classmethod
build_schema
()¶ Build a validation schema using the config options defined in this class and its parent classes.
-
denormalize
()¶ Create new config object that inherits all explicit attributes from its parents as well.
-
get_local
(name, default=None)¶ Returns a local config setting (not from container)
-
ignore_extra_keys
= True¶
-
parent
¶ Returns the parent configuration.
-
set_local
(name, value)¶ set without any check
-
classmethod
-
class
testplan.base.
TestplanMock
(*args, **kwargs)[source]¶ Bases:
testplan.base.Testplan
A mock Testplan class for testing purpose. It is recommended to use mockplan fixture defined in conftest.py if you can. Only use this when necessary, e.g. you need to override default parameters.
-
CONFIG
¶ alias of
TestplanConfig
-
STATUS
¶
-
abort
()¶ Default abort policy. First abort all dependencies and then itself.
-
abort_dependencies
()¶ Dependencies to be aborted first.
-
aborted
¶ Returns if entity was aborted.
-
aborting
()¶ Suppressing not implemented debug log by parent class.
-
active
¶ Expose the runnable active attribute.
-
args
¶ Parsed arguments.
-
cfg
¶ Expose the runnable configuration object.
-
context_input
(exclude: list = None) → Dict[str, Any]¶ All attr of self in a dict for context resolution
-
define_runpath
()¶ Define runpath directory based on parent object and configuration.
-
enrich_options
(options)¶ Enrich the options using parsed command line arguments. The command line arguments will override any explicit programmatic declaration for a given keyword.
-
classmethod
filter_locals
(local_vars)¶ Filter out init params of None value, they will take default value defined in its ConfigOption object; also filter out special vars that are not init params from local_vars.
Parameters: local_vars –
-
logger
¶ logger object
-
classmethod
main_wrapper
(name, description=None, parse_cmdline=True, parser=<class 'testplan.parser.TestplanParser'>, interactive_port=None, abort_signals=None, logger_level=25, file_log_level=10, runpath=<function default_runpath>, path_cleanup=True, all_tasks_local=False, shuffle=None, shuffle_seed=590.0, exporters=None, stdout_style=Style(passing='testcase', failing='assertion-detail'), report_dir='/home/docs/checkouts/readthedocs.org/user_builds/testplan/checkouts/latest/doc/en', xml_dir=None, json_path=None, http_url=None, pdf_path=None, pdf_style=Style(passing='test', failing='test'), report_tags=None, report_tags_all=None, resource_monitor=False, merge_scheduled_parts=False, browse=False, ui_port=None, web_server_startup_timeout=10, test_filter=<testplan.testing.filtering.Filter object>, test_sorter=<testplan.testing.ordering.NoopSorter object>, test_lister=None, test_lister_output=None, verbose=False, debug=False, timeout=14400, interactive_handler=<class 'testplan.runnable.interactive.base.TestRunnerIHandler'>, extra_deps=None, label=None, auto_part_runtime_limit=1800, plan_runtime_target=1800, **options)¶ Decorator that will be used for wrapping main methods in test scripts.
It accepts all arguments of a
Testplan
entity.
-
make_runpath_dirs
()¶ Creates runpath related directories.
-
parser
¶ Returns a new command line parser.
-
pause
()¶ Pauses entity execution.
-
pausing
()¶ Pause the runnable execution.
-
processed_args
¶ Processed parsed arguments.
-
resume
()¶ Resumes entity execution.
-
resuming
()¶ Resume the runnable execution.
-
run
()¶ Runs the tests added and returns the result object. Also handles usr1 and usr2 signals.
Returns: Result containing tests and execution steps results. Return type: TestplanResult
-
runnable
¶ Runnable instance.
-
runpath
¶ Expose the runnable runpath.
-
scratch
¶ Path to be used for temp files by entity.
-
status
¶ Expose the runnable status.
-
uid
()¶ Unique identifier of self.
-
wait
(target_status, timeout=None)¶ Wait until objects status becomes target status.
Parameters: - target_status (
str
) – expected status - timeout (
int
orNoneType
) – timeout in seconds
- target_status (
-
-
class
testplan.base.
TestplanResult
[source]¶ Bases:
testplan.runnable.base.TestRunnerResult
Result object of a
Testplan
runnable manager
entity.-
exit_code
¶ System exit code based on successful run.
-
report
¶ Tests report.
-
success
¶ Run was successful.
-
testplan.defaults module¶
All default values that will be shared between config objects go here.
testplan.environment module¶
Module containing environments related classes.
-
class
testplan.environment.
EnvironmentCreator
(uid, resources=None)[source]¶ Bases:
object
Environment creator interface.
-
class
testplan.environment.
Environments
(uid='environments', **options)[source]¶ Bases:
testplan.common.entity.base.Resource
Environments holder resource.
Also inherits all
Resource
options.-
envs
¶ Returns all added environments.
-
-
class
testplan.environment.
LocalEnvironment
(uid, resources=None)[source]¶ Bases:
testplan.environment.EnvironmentCreator
Creator class of a local environment.
testplan.parser module¶
Classes that parse command-line arguments used to control testplan behaviour. This module encodes the argument and option names, types, and behaviours.
-
class
testplan.parser.
HelpParser
(prog=None, usage=None, description=None, epilog=None, parents=[], formatter_class=<class 'argparse.HelpFormatter'>, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True, allow_abbrev=True)[source]¶ Bases:
argparse.ArgumentParser
Extends
ArgumentParser
in order to print the help message upon failure.
-
class
testplan.parser.
LogLevelAction
(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]¶ Bases:
argparse.Action
Custom parser action to convert from a string log level to its int value, e.g. “DEBUG” -> 10. The level can also be specified as “NONE”, which will be stored internally as None.
-
LEVELS
= {'CRITICAL': 50, 'DEBUG': 10, 'ERROR': 40, 'INFO': 20, 'NONE': None, 'USER_INFO': 25, 'WARNING': 30}¶
-
-
class
testplan.parser.
TestplanParser
(name: str, default_options: Dict[KT, VT])[source]¶ Bases:
object
Wrapper around argparse.ArgumentParser, adds extra step for processing arguments, useful when there are cross-dependencies between them.
-
add_arguments
(parser)[source]¶ Virtual method to be overridden by custom parsers.
Parameters: parser – parser instance
-
generate_parser
() → testplan.parser.HelpParser[source]¶ Generates an argparse.ArgumentParser instance.
-
process_args
(namespace: argparse.Namespace) → Dict[KT, VT][source]¶ Overrides this method to add extra argument processing logic.
Can be used for interdependent argument processing.
Testplan uses the result dictionary to initialize the configuration.
Parameters: namespace – namespace of parsed arguments Returns: initial configuration
-
Module contents¶
Module containing Testplan main class.