testplan.runnable package

Subpackages

• testplan.runnable.interactive package
  • Submodules
    • testplan.runnable.interactive.base
      • TestRunnerIHandler
        • TestRunnerIHandler.CONFIG
        • TestRunnerIHandler.STATUS
        • TestRunnerIHandler.abort_dependencies()
        • TestRunnerIHandler.aborting()
        • TestRunnerIHandler.all_tests()
        • TestRunnerIHandler.all_tests_operation()
        • TestRunnerIHandler.environment_resource_context()
        • TestRunnerIHandler.environment_resource_operation()
        • TestRunnerIHandler.environment_resource_start()
        • TestRunnerIHandler.environment_resource_stop()
        • TestRunnerIHandler.exit_code
        • TestRunnerIHandler.exporters
        • TestRunnerIHandler.get_driver_info_report()
        • TestRunnerIHandler.get_environment()
        • TestRunnerIHandler.get_environment_context()
        • TestRunnerIHandler.get_environment_resource()
        • TestRunnerIHandler.http_handler_info
        • TestRunnerIHandler.reload()
        • TestRunnerIHandler.reload_report()
        • TestRunnerIHandler.report
        • TestRunnerIHandler.reset_all_tests()
        • TestRunnerIHandler.reset_test()
        • TestRunnerIHandler.run()
        • TestRunnerIHandler.run_all_tests()
        • TestRunnerIHandler.run_test()
        • TestRunnerIHandler.run_test_case()
        • TestRunnerIHandler.run_test_case_param()
        • TestRunnerIHandler.run_test_suite()
        • TestRunnerIHandler.setup()
        • TestRunnerIHandler.start_environment()
        • TestRunnerIHandler.start_resource()
        • TestRunnerIHandler.start_test_resources()
        • TestRunnerIHandler.start_tests()
        • TestRunnerIHandler.stop_environment()
        • TestRunnerIHandler.stop_resource()
        • TestRunnerIHandler.stop_test_resources()
        • TestRunnerIHandler.stop_tests()
        • TestRunnerIHandler.target
        • TestRunnerIHandler.teardown()
        • TestRunnerIHandler.test()
        • TestRunnerIHandler.test_case_report()
        • TestRunnerIHandler.test_report()
        • TestRunnerIHandler.test_resource()
        • TestRunnerIHandler.test_resource_operation()
        • TestRunnerIHandler.test_resource_start()
        • TestRunnerIHandler.test_resource_stop()
      • TestRunnerIHandlerConfig
        • TestRunnerIHandlerConfig.get_options()
    • testplan.runnable.interactive.http
      • OutOfOrderError
      • TestRunnerHTTPHandler
        • TestRunnerHTTPHandler.CONFIG
        • TestRunnerHTTPHandler.aborting()
        • TestRunnerHTTPHandler.bind_addr
        • TestRunnerHTTPHandler.run()
        • TestRunnerHTTPHandler.setup()
      • TestRunnerHTTPHandlerConfig
        • TestRunnerHTTPHandlerConfig.get_options()
      • decode_uri_component()
      • generate_interactive_api()
    • testplan.runnable.interactive.reloader
      • ModuleReloader
        • ModuleReloader.reload()
    • testplan.runnable.interactive.resource_loader
      • ResourceLoader
        • ResourceLoader.load()
  • Module contents

Submodules

testplan.runnable.base module

Tests runner module.

class testplan.runnable.base.TaskInformation(target: testplan.testing.base.Test | testplan.runners.pools.tasks.base.Task | Callable, materialized_test: testplan.testing.base.Test, uid: str, task_arguments: dict, num_of_parts: int)

Bases: object

materialized_test: Test

num_of_parts: int

target: Test | Task | Callable

task_arguments: dict

uid: str

class testplan.runnable.base.TestRunner(**options)

Bases: Runnable

Adds tests to test executor resources and invoke report exporter objects to create the TestRunnerResult.

Parameters:

• name (str) – Name of test runner.

• description (str) – Description of test runner.

• logger_level (int) – Logger level for stdout.

• runpath (str or callable) – Input runpath.

• path_cleanup (bool) – Clean previous runpath entries.

• all_tasks_local (bool) – Schedule all tasks in local pool

• shuffle (list of str) – Shuffle strategy.

• shuffle_seed (float) – Shuffle seed.

• exporters (list) – Exporters for reports creation.

• stdout_style (Style) – Styling output options.

• report_dir (str) – Report directory.

• xml_dir (str) – XML output directory.

• pdf_path (str) – PDF output path <PATH>/*.pdf.

• json_path (str) – JSON output path <PATH>/*.json.

• pdf_style (Style) – PDF creation styling options.

• http_url (str) – Web url for posting test report.

• report_tags (list) – Matches tests marked with any of the given tags.

• report_tags_all (list) – Match tests marked with all of the given tags.

• merge_scheduled_parts (bool) – Merge report of scheduled MultiTest parts.

• browse (bool or NoneType) – Open web browser to display the test report.

• ui_port (int or NoneType) – Port of web server for displaying test report.

• web_server_startup_timeout (int) – Timeout for starting web server.

• test_filter (Subclass of BaseFilter) – Tests filtering class.

• test_sorter (Subclass of BaseSorter) – Tests sorting class.

• test_lister (Subclass of BaseLister) – Tests listing class.

• verbose (bool) – Enable or disable verbose mode.

• debug (bool) – Enable or disable debug mode.

• timeout (NoneType or int (greater than 0).) – Timeout value for test execution.

• abort_wait_timeout (int) – Timeout for test runner abort.

• interactive_handler (Subclass of :py:class: TestRunnerIHandler <testplan.runnable.interactive.base.TestRunnerIHandler>) – Handler for interactive mode execution.

• extra_deps (list of module or str) – Extra module dependencies for interactive reload, or paths of these modules.

• label (str or NoneType) – Label the test report with the given name, useful to categorize or classify similar reports .

• runtime_data (dict) – Historical runtime data which will be used for Multitest auto-part and weight-based Task smart-scheduling

• auto_part_runtime_limit (int or float or literal “auto”) – The runtime limitation for auto-part task

• plan_runtime_target (int or float or literal “auto”) – The testplan total runtime limitation for smart schedule

Param:

file_log_level: Logger level for file.

Also inherits all Runnable options.

CONFIG

alias of TestRunnerConfig

RESULT

alias of TestRunnerResult

STATUS

alias of TestRunnerStatus

abort_dependencies()

Yield all dependencies to be aborted before self abort.

aborting()

Stop the web server if it is running.

add(target: Test | Task | Callable, resource: str | None = None) → str | None

Adds a runnable test entity, or a Task, or a callable that returns a test entity to a Executor resource.

Parameters:

• target (Runnable or Task or callable) – Test target.

• resource (str or NoneType) – Name of the target executor, which is usually a Pool, default value None indicates using local executor.

Returns:

Assigned uid for test.

Return type:

str or `NoneType

add_environment(env: EnvironmentCreator, resource: Environments | None = None)

Adds an environment to the target resource holder.

Parameters:

• env (Subclass of EnvironmentCreator) – Environment creator instance.

• resource (Subclass of Environments) – Target environments holder resource.

Returns:

Environment uid.

Return type:

str

add_exporters(exporters: List[BaseExporter])

Add a list of report exporters for outputting test report.

Parameters:

exporters (list of Executor) – Test exporters to be added.

add_main_batch_steps()

Runnable steps to be executed while resources are running.

add_post_resource_steps()

Runnable steps to be executed after resources stopped.

add_pre_resource_steps()

Runnable steps to be executed before resources started.

add_remote_service(remote_service: RemoteService)

Adds a remote service RemoteService object to test runner.

Parameters:

• remote_service – RemoteService object

• remote_service – RemoteService

add_resource(resource: Resource, uid: str | None = None) → str

Adds a test executor resource in the test runner environment.

Parameters:

• resource – Test executor to be added.

• uid – Optional input resource uid. We now force its equality with resource’s own uid.

Returns:

Resource uid assigned.

auto_part(tasks: List[Task]) → List[Task]

Automatically partitions tasks into smaller parts based on runtime limits. This method takes a list of tasks and partitions them into smaller tasks if their runtime exceeds the configured auto_part_runtime_limit. The partitioning is determined by analyzing runtime data and calculating appropriate parts and weights for each task.

If the auto_part_runtime_limit is set to “auto”, the method derives the runtime limit based on historical runtime data or defaults to a predefined maximum value if no runtime data is available.

Parameters:

tasks (List[Task]) – List of tasks to be partitioned.

Returns:

List of partitioned tasks.

Return type:

List[Task]

calculate_pool_size() → None

Calculate the right size of the pool based on the weight (runtime) of the tasks, so that runtime of all tasks meets the plan_runtime_target.

calculate_pool_size_by_tasks(tasks: Collection[Task]) → int

Calculate the right size of the pool based on the weight (runtime) of the tasks, so that runtime of all tasks meets the plan_runtime_target.

disable_reset_report_uid()

Do not generate unique strings in uuid4 format as report uid

discard_pending_tasks(exec_selector: Expr, report_status: Status = Status.INCOMPLETE, report_reason: str = '')

discover(path: str = '.', name_pattern: str | Pattern = '.*\\.py$') → List[Task]

Discover task targets under path in the modules that matches name pattern, and return the created Task object.

Parameters:

• path – the root path to start a recursive walk and discover, default is current directory.

• name_pattern – a regex pattern to match the file name.

Returns:

A list of Task objects

property exporters

Return a list of report exporters.

get_default_exporters()

Instantiate certain exporters if related cmdline argument (e.g. –pdf) or programmatic arguments (e.g. pdf_path) is passed but there are not any exporter declarations.

get_test_metadata()

static get_testplan_version()

make_runpath_dirs()

Creates runpath related directories.

property report: TestReport

Tests report.

run()

Executes the defined steps and populates the result object.

schedule(task: Task | None = None, resource: str | None = None, **options) → str | None

Schedules a serializable Task in a task runner Pool executor resource.

Parameters:

• task (Task) – Input task, if it is None, a new Task will be constructed using the options parameter.

• resource (str or NoneType) – Name of the target executor, which is usually a Pool, default value None indicates using local executor.

• options (dict) – Task input options.

Return uid:

Assigned uid for task.

Return type:

str or NoneType

schedule_all(path: str = '.', name_pattern: str | Pattern = '.*\\.py$', resource: str | None = None)

Discover task targets under path in the modules that matches name pattern, create task objects from them and schedule them to resource (usually pool) for execution.

Parameters:

• path (str) – the root path to start a recursive walk and discover, default is current directory.

• name_pattern (str) – a regex pattern to match the file name.

• resource (str or NoneType) – Name of the target executor, which is usually a Pool, default value None indicates using local executor.

skip_step(step)

Callable to determine if step should be skipped.

uid()

Entity uid.

class testplan.runnable.base.TestRunnerConfig(**options)

Bases: RunnableConfig

Configuration object for TestRunner runnable object.

classmethod get_options()

Runnable specific config options.

ignore_extra_keys = True

class testplan.runnable.base.TestRunnerResult

Bases: RunnableResult

Result object of a TestRunner runnable object.

property success

Run was successful.

class testplan.runnable.base.TestRunnerStatus

Bases: RunnableStatus

Status of a TestRunner runnable object.

testplan.runnable.base.check_local_server(browse)

Early exit if local server (interactive extra) is not installed when user asks for displaying report using local server feature.

testplan.runnable.base.collate_for_merging(es: List[TestGroupReport | TestCaseReport]) → List[List[TestGroupReport | TestCaseReport]]

Group report entries into buckets, where synthesized ones in the same bucket containing the previous non-synthesized one.

testplan.runnable.base.get_exporters(values)

Validation function for exporter declarations.

Parameters:

values – Single or a list of exporter declaration(s).

Returns:

List of initialized exporter objects.

testplan.runnable.base.result_for_failed_task(original_result)

Create a new result entry for invalid result retrieved from a resource.

testplan.runnable.base.validate_lines(d: dict) → bool

Module contents