Analysis workflows¶

The dyff.audit.analysis module contains tools for implementing analysis Methods and running them on platform data. The Dyff platform uses the functions in dyff.audit.analysis.runners to run the Method code, and you can use these same functions via the DyffLocalPlatform to develop and debug analysis code locally.

Within a Method implementation, you use AnalysisContext to access inputs and metadata for the current analysis, display formatted conclusions in Jupyter notebooks, and output artifacts such as Scores to be stored by Dyff. See the SafetyCase tutorial for usage examples.

Analysis context¶

AnalysisContext is Dyff’s mechanism for making input data available to user- authored analysis Methods.

When the Method is implemented in a framework such as Jupyter that does not support “arguments”, the implementation accesses its inputs by instantiating an AnalysisContext. The AnalysisContext gets its configuration information from environment variables. The runners for analyses implemented in other ways also use AnalysisContext under the hood.

When running an analysis on the Dyff Platform, the platform provides the analysis_config_file and the local_storage_root arguments via the environment variables DYFF_AUDIT_ANALYSIS_CONFIG_FILE and DYFF_AUDIT_LOCAL_STORAGE_ROOT.

Note

If you are creating an AnalysisContext instance in code that will run on the Dyff Platform, you must call the constructor with no arguments, e.g., ctx = AnalysisContext().

Parameters:

analysis_config_file – The path to a YAML-format specification of an Analysis. If not specified, it is read from the DYFF_AUDIT_ANALYSIS_CONFIG_FILE environment variable.
local_storage_root – The root directory for local storage of entity data. If not specified, it is read from the DYFF_AUDIT_LOCAL_STORAGE_ROOT environment variable.
analysis – You can also specify the analysis as an Analysis instance. If you do, you must also specify the id. This is mainly useful for debugging.
id – The ID of the analysis, which is needed when instantiating from an Analysis instance, because Analysis doesn’t have an .id field.
allow_override_from_environment – If True, environment variables will override values in the config file. By default, the config file has precedence.

Conclusion(*, text: str, indicator: Literal['Information', 'Question', 'Hazard'] = 'Information') → None¶

Display a Conclusion widget at the current position in the Jupyter notebook.

Parameters:

text – The text to display.
indicator – The icon to display.

Score(*, quantity: float, text: str, output: str | None = None, display: bool = True, format: str | None = None, unit: str | None = None) → None¶

Display a Score widget at the current position in the Jupyter notebook.

If output is given, the score will be saved in the Dyff datastore under the specified name. The name must match the name of a score declared in the Method spec. In this case, format and unit take the value specific in the spec, and overriding this value is an error.

Parameters:

quantity – The measured value of the score.
text – A text description of what the score means.
output – If given, it must match the name of a score declared in the Method spec. The score quantity will be saved in the Dyff datastore under that name.
display – If False, do not display the score widget in the Jupyter notebook (but still output the score if output is given).
format – A Python format string used to render the quantity as a string. It must use the key quantity, and it may use the key unit, e.g. "{quantity} {unit}".
unit – A string representation of the unit of measurement, e.g., "MJ/kg", "%", etc. Prefer SI units when applicable.

TitleCard(*, headline: str, author: str, summary_phrase: str, summary_text: str, system_title: str | None = None, system_summary: str | None = None, usecase_title: str | None = None, usecase_summary: str | None = None) → None¶

Display a TitleCard widget at the current position in the Jupyter notebook.

Normally, this should be the first output in the notebook.

Parameters:

headline – The headline text for the notebook.
author – Description of the author(s) of the notebook.
summary_phrase – A “sub-heading” for the summary information.
summary_text – A text summary of the notebook.
system_title – The “title” of the system-under-test. When running on the Dyff platform, this defaults to the title given in the system’s documentation.
system_summary – A “summary” of the system-under-test. When running on the Dyff platform, this defaults to the summary given in the system’s documentation.
usecase_title –
The “title” of the Method being run. When running on the Dyff platform, this defaults to the title given in the Method’s documentation.

Deprecated since version 0.10.2: “usecase” is a deprecated alias for “Method”; this name will change in a future release.
usecase_summary –
A “summary” of the Method being run. When running on the Dyff platform, this defaults to the summary given in the Methods’s documentation.

Deprecated since version 0.10.2: “usecase” is a deprecated alias for “Method”; this name will change in a future release.

get_argument(keyword: str) → str¶

Get the value of an argument passed to the analysis.

Parameters:: keyword – The keyword specified for the argument in the Method spec

open_input_dataset(keyword: str) → Dataset¶

Open a dataset provided as input to the analysis.

Parameters:: keyword – The keyword specified for the input in the Method spec

property analysis: Analysis¶: The spec for the current analysis.

property arguments: dict[str, str]¶: The arguments passed to the analysis.

property id: str¶: The ID of the current analysis.

property inputs: list[str]¶: The names of all of the input datasets passed to the analysis.

property local_storage_root: Path¶: The root path where subdirectories containing artifacts for individual Dyff resources will be created on the local file system.

property output_path: Path¶: The path where output artifacts are stored on the local file system.

property system: SystemInformation | None¶

Information about the system under test.

Currently, this is populated only for the SafetyCase workflow.

property usecase: UseCaseInformation | None¶

Information about the use case being tested.

Currently, this is populated only for the SafetyCase workflow.

Deprecated since version 0.10.2: “usecase” is a deprecated alias for “Method”; this name will change in a future release.

Runners¶

dyff.audit.analysis.runners.run_analysis(method: MethodBase, *, storage_root: Path, config_file: Path) → None¶

Run an analysis workflow locally.

The analysis workflow consists of running a Method on specified inputs to generate output artifacts such as SafetyCases and Measurements.

The analysis is run in a sub-process. Input artifacts must be accessible at storage_root / entity_id for each input entity. Output artifacts will be created in storage_root / analysis_id. Logs will be saved in storage_root / analysis_id / .dyff / logs.txt.

Parameters:

method – The specification of the method to run.
storage_root – The root directory for local storage of input and output artifacts.
config_file –
A YAML file containing a Kubernetes Analysis resource (kind analyses.dyff.io/v1alpha1).

Note

The format of this file is expected to change in a future release.

dyff.audit.analysis.runners.run_report(report: Report, *, storage_root: Path)¶

Run a Report workflow locally.

Deprecated since version 0.8.0: Report functionality has been refactored into the Method/Measurement/Analysis apparatus. Creation of new Reports is disabled.

The workflow is run in a sub-process. Input artifacts must be accessible at storage_root / entity_id for each input entity. Output artifacts will be created in storage_root / report_id. Logs will be saved in storage_root / report_id / .dyff / logs.txt.

Parameters:

report – The specification of the Report to run.
storage_root – The root directory for local storage of input and output artifacts.