dyff.client¶

The dyff.client package provides a Python client for interacting with the Dyff REST API.

Installation¶

Install the dyff-client package from PyPI:

python3 -m pip install dyff-client

Configuration¶

Here are well-known API locations:

http://api.dyff.local/v0 - A self-hosted Dyff on a local machine.
https://api.dyff.io/v0 - The Dyff instance hosted by the Dyff maintainers.

Reference¶

class dyff.client.Client(*, api_key: str, endpoint: str | None = None, verify_ssl_certificates: bool = True, insecure: bool = False)¶

The Python client for the Dyff Platform API.

API operations are grouped by the resource type that they manipulate. For example, all operations on Evaluation resources are accessed like client.evaluations.create().

The Python API functions may have somewhat different behavior from the corresponding API endpoints, and the Python client also adds several higher-level API functions that are implemented with multiple endpoint calls.

Parameters:

api_key (str) – An API token to use for authentication.
endpoint (str) – The URL where the Dyff Platform API is hosted. Defaults to the UL DSRI-hosted Dyff instance.
verify_ssl_certificates (bool) –
You can disable certificate verification for testing; you should do this only if you have also changed endpoint to point to a trusted local server.

Deprecated since version 0.2.2: Use insecure instead
insecure (bool) – Disable certificate verification for testing. you should do this only if you have also changed endpoint to point to a trusted local server.

property datasets: DatasetsOperations¶: Operations on Dataset entities.

property evaluations: EvaluationsOperations¶: Operations on Evaluation entities.

property inferenceservices: InferenceservicesOperations¶: Operations on InferenceService entities.

property inferencesessions: InferencesessionsOperations¶: Operations on InferenceSession entities.

property measurements: MeasurementsOperations¶: Operations on Measurement entities.

property methods: MethodsOperations¶: Operations on Method entities.

property models: ModelsOperations¶: Operations on Model entities.

property modules: ModulesOperations¶: Operations on Module entities.

property raw: DyffV0API¶: The “raw” API client, which can be used to send JSON requests directly.

property reports: ReportsOperations¶: Operations on Report entities.

property safetycases: SafetycasesOperations¶: Operations on SafetyCase entities.

class dyff.client.client.InferenceSessionClient¶

A client used for making inference requests to a running InferenceSession.

Note

Do not instantiate this class. Create an instance using inferencesessions.client()

verify_ssl_certifcates is deprecated, use insecure instead.

infer(body: Any) → Any¶

Make an inference request.

The input and output are arbitrary JSON objects. The required format depends on the endpoint and input/output adapters specified when creating the inference client.

Parameters:: body (Any) – A JSON object containing the inference input.
Returns:: A JSON object containing the inference output.

class dyff.client.client.DatasetsOperations¶

Operations on Dataset entities.

Note

Do not instantiate this class. Access it through the .datasets attribute of Client.

verify_ssl_certifcates is deprecated, use insecure instead.

create(dataset_request: DatasetCreateRequest) → Dataset¶

Create a Dataset.

Note

This operation may incur compute costs.

Parameters:: dataset_request (DatasetCreateRequest) – The dataset request specification.
Returns:: A full Dataset entity with .id and other properties set.

create_arrow_dataset(dataset_directory: Path | str, *, account: str, name: str) → Dataset¶

Create a Dataset resource describing an existing Arrow dataset.

Internally, constructs a DatasetCreateRequest using information obtained from the Arrow dataset, then calls create() with the constructed request.

Typical usage:

dataset = client.datasets.create_arrow_dataset(dataset_directory, ...)
client.datasets.upload_arrow_dataset(dataset, dataset_directory)

Parameters:

dataset_directory (str) – The root directory of the Arrow dataset.
account (str) – The account that will own the Dataset resource.
name (str) – The name of the Dataset resource.

Returns:

The complete Dataset resource.

Return type:

Dataset

delete(dataset_id: str) → Status¶

Mark a Dataset for deletion.

Parameters:: dataset_id (str) – The dataset key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status

documentation(dataset_id: str) → Documentation¶

Get the documentation associated with a Dataset.

Get the documentation associated with a Dataset. Raises a 404 error if no entity exists with that key.

Parameters:: dataset_id (str) – The ID of the Dataset.
Returns:: The documentation associated with the Dataset.
Return type:: Documentation :raises ~azure.core.exceptions.HttpResponseError:

downlinks(dataset_id: str) → list[ArtifactURL]¶

Get a list of signed GET URLs from which Dataset artifacts can be downloaded.

Parameters:: dataset_id (str) – The ID of the Dataset.
Returns:: List of signed GET URLs.
Return type:: list[ArtifactURL] :raises ~azure.core.exceptions.HttpResponseError:

download(dataset_id: str, destination: Path | str) → None¶

Download all of the files in a dataset to a local directory.

The destination directory must be an empty directory that exists.

Parameters:

dataset_id (str) – The ID of the Dataset.
destination (Path | str) – The destination directory. Must exist and be empty.

Raises:

HttpResponseError – If datasets.downlinks(dataset_id) raises
ValueError – If arguments are invalid

edit_documentation(dataset_id: str, edit_request: DocumentationEditRequest) → Documentation¶

Edit the documentation associated with a Dataset.

Edit the documentation associated with a Dataset. Raises a 404 error if no entity exists with that key. Returns the modified Documentation.

Parameters:

dataset_id (str) – The ID of the Dataset.
edit_request (DocumentationEditRequest) – Object containing the edits to make.

Returns:

The modified documentation.

Return type:

Documentation :raises ~azure.core.exceptions.HttpResponseError:

get(dataset_id: str) → Dataset¶

Get a Dataset by its key.

Parameters:: dataset_id (str) – The dataset key
Returns:: The Dataset with the given key.

label(dataset_id: str, labels: dict[str, str | None]) → None¶

Label the specified Dataset with key-value pairs (stored in the .labels field of the resource).

Providing None for the value deletes the label.

See Label for a description of the constraints on label keys and values.

Parameters:

dataset_id (str) – The ID of the Dataset to label.
labels (dict[str, Optional[str]]) – The label keys and values.

Get all Datasets matching a query. The query is a set of equality constraints specified as key-value pairs.

Parameters:

query (str | dict[str, Any] | list[dict[str, Any]])
id (str)
account (str)
status (str)
reason (str)
labels (dict[str, str])
name (str) – Default value is None.

Returns:

list of Dataset resources satisfying the query.

Return type:

list[Dataset]

Raises:

HttpResponseError –

upload_arrow_dataset(dataset: Dataset, dataset_directory: Path | str) → None¶

Uploads the data files in an existing Arrow dataset for which a Dataset resource has already been created.

Typical usage:

dataset = client.datasets.create_arrow_dataset(dataset_directory, ...)
client.datasets.upload_arrow_dataset(dataset, dataset_directory)

Parameters:

dataset (Dataset) – The Dataset resource for the Arrow dataset.
dataset_directory (str) – The root directory of the Arrow dataset.

class dyff.client.client.EvaluationsOperations¶

Operations on Evaluation entities.

Note

Do not instantiate this class. Access it through the .evaluations attribute of Client.

create(evaluation_request: EvaluationCreateRequest) → Evaluation¶

Create an Evaluation.

Note

This operation will incur compute costs.

Parameters:: evaluation_request (EvaluationCreateRequest) – The evaluation request specification.
Returns:: A full Evaluation entity with .id and other properties set.

delete(evaluation_id: str) → Status¶

Mark an Evaluation for deletion.

Parameters:: evaluation_id (str) – The evaluation key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status

downlinks(evaluation_id: str) → list[ArtifactURL]¶

Get a list of signed GET URLs from which Evaluation artifacts can be downloaded.

Parameters:: evaluation_id (str) – The ID of the Evaluation.
Returns:: List of signed GET URLs.
Return type:: list[ArtifactURL] :raises ~azure.core.exceptions.HttpResponseError:

download(evaluation_id: str, destination: Path | str) → None¶

Download all of the files in an evaluation to a local directory.

The destination directory must be an empty directory that exists.

Parameters:

evaluation_id (str) – The ID of the Evaluation.
destination (Path | str) – The destination directory. Must exist and be empty.

Raises:

HttpResponseError – If evaluations.downlinks(evaluation_id) raises
ValueError – If arguments are invalid

get(evaluation_id: str) → Evaluation¶

Get an Evaluation by its key.

Parameters:: evaluation_id (str) – The evaluation id
Returns:: The Evaluation with the given key.

label(evaluation_id: str, labels: dict[str, str | None]) → None¶

Label the specified Evaluation with key-value pairs (stored in the .labels field of the resource).

Providing None for the value deletes the label.

See Label for a description of the constraints on label keys and values.

Parameters:

evaluation_id (str) – The ID of the Evaluation to label.
labels (dict[str, Optional[str]]) – The label keys and values.

Get all Evaluations matching a query. The query is a set of equality constraints specified as key-value pairs.

Parameters:

query (str | dict[str, Any] | list[dict[str, Any]])
id (str)
account (str)
status (str)
reason (str)
labels (dict[str, str])
dataset (str)
inferenceService (str)
inferenceServiceName (str)
model (str)
modelName (str)

Returns:

list of Evaluation resources satisfying the query.

Return type:

list[Evaluation]

Raises:

HttpResponseError –

class dyff.client.client.InferenceservicesOperations¶

Operations on InferenceService entities.

Note

Do not instantiate this class. Access it through the .inferenceservices attribute of Client.

create(inference_service_request: InferenceServiceCreateRequest) → InferenceService¶

Create an InferenceService.

Note

This operation may incur compute costs.

Parameters:: inference_service_request (InferenceServiceCreateRequest) – The inference service request specification.
Returns:: A full InferenceService entity with .id and other properties set.

delete(service_id: str) → Status¶

Mark an InferenceService for deletion.

Parameters:: service_id (str) – The inference service key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status

documentation(service_id: str) → Documentation¶

Get the documentation associated with an InferenceService.

Get the documentation associated with a InferenceService. Raises a 404 error if no entity exists with that key.

Parameters:: service_id (str) – The ID of the InferenceService.
Returns:: The documentation associated with the InferenceService.
Return type:: Documentation :raises ~azure.core.exceptions.HttpResponseError:

edit_documentation(service_id: str, edit_request: DocumentationEditRequest) → Documentation¶

Edit the documentation associated with an InferenceService.

Edit the documentation associated with an InferenceService. Raises a 404 error if no entity exists with that key. Returns the modified Documentation.

Parameters:

service_id (str) – The ID of the InferenceService.
edit_request (DocumentationEditRequest) – Object containing the edits to make.

Returns:

The modified documentation.

Return type:

Documentation :raises ~azure.core.exceptions.HttpResponseError:

get(service_id: str) → InferenceService¶

Get an InferenceService by its key.

Parameters:: service_id (str) – The inference service id
Returns:: The InferenceService with the given key.

label(service_id: str, labels: dict[str, str | None]) → None¶

Label the specified InferenceService with key-value pairs (stored in the .labels field of the resource).

Providing None for the value deletes the label.

See Label for a description of the constraints on label keys and values.

Parameters:

service_id (str) – The ID of the InferenceService to label.
labels (dict[str, Optional[str]]) – The label keys and values.

Get all InferenceServices matching a query. The query is a set of equality constraints specified as key-value pairs.

Parameters:

query (str | dict[str, Any] | list[dict[str, Any]])
id (str)
account (str)
status (str)
reason (str)
labels (dict[str, str])
name (str)
model (str)
modelName (str)

Returns:

list of InferenceService resources satisfying the query.

Return type:

list[InferenceService]

Raises:

HttpResponseError –

class dyff.client.client.InferencesessionsOperations¶

Operations on Inferencesession entities.

Note

Do not instantiate this class. Access it through the .inferencesessions attribute of Client.

verify_ssl_certifcates is deprecated, use insecure instead.

client(session_id: str, token: str, *, interface: InferenceInterface | None = None, endpoint: str | None = None, input_adapter: Adapter | None = None, output_adapter: Adapter | None = None) → InferenceSessionClient¶

Create an InferenceSessionClient that interacts with the given inference session. The token should be one returned either from Client.inferencesessions.create() or from Client.inferencesessions.token(session_id).

The inference endpoint in the session must also be specified, either directly through the endpoint argument or by specifying an interface. Specifying interface will also use the input and output adapters from the interface. You can also specify these separately in the input_adapter and output_adapter. The non-interface arguments override the corresponding values in interface if both are specified.

Parameters:

session_id (str) – The inference session to connect to
token (str) – An access token with permission to run inference against the session.
interface (Optional[InferenceInterface]) – The interface to the session. Either interface or endpoint must be specified.
endpoint (str) – The inference endpoint in the session to call. Either endpoint or interface must be specified.
input_adapter (Optional[Adapter]) – Optional input adapter, applied to the input before sending it to the session. Will override the input adapter from interface if both are specified.
output_adapter (Optional[Adapter]) – Optional output adapter, applied to the output of the session before returning to the client. Will override the output adapter from interface if both are specified.

Returns:

An InferenceSessionClient that makes inference calls to the specified session.

create(inference_session_request: InferenceSessionCreateRequest) → InferenceSessionAndToken¶

Create an InferenceSession.

Note

This operation will incur compute costs.

Parameters:

account_id (str) – The account that will own the evaluation.
inference_session_request (InferenceSessionCreateRequest) – The inference service request specification.

Returns:

A full InferenceSession entity with .id and other properties set.

delete(session_id: str) → Status¶

Mark an InferenceSession for deletion.

Parameters:: session_id (str) – The inference session key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status

get(session_id: str) → InferenceSession¶

Get an InferenceSession by its key.

Parameters:: session_id (str) – The inference session id
Returns:: The InferenceSession with the given key.

label(session_id: str, labels: dict[str, str | None]) → None¶

Label the specified InferenceSession with key-value pairs (stored in the .labels field of the resource).

Providing None for the value deletes the label.

See Label for a description of the constraints on label keys and values.

Parameters:

session_id (str) – The ID of the InferenceSession to label.
labels (dict[str, Optional[str]]) – The label keys and values.

Get all InferenceSessions matching a query. The query is a set of equality constraints specified as key-value pairs.

Parameters:

query (str | dict[str, Any] | list[dict[str, Any]])
id (str)
account (str)
status (str)
reason (str)
labels (dict[str, str])
name (str)
model (str)
modelName (str)

Returns:

list of InferenceSession resources satisfying the query.

Return type:

list[InferenceSession]

Raises:

HttpResponseError –

ready(session_id: str) → bool¶

Return True if the session is ready to receive inference input.

The readiness probe is expected to fail with status codes 404 or 503, as these will occur at times during normal session start-up. The ready() method returns False in these cases. Any other status codes will raise an HttpResponseError.

Parameters:: session_id (str) – The ID of the session.
Raises:: HttpResponseError –

terminate(session_id: str) → Status¶

Terminate a session.

Parameters:: session_id (str) – The inference session key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status
Raises:: HttpResponseError –

token(session_id: str, *, expires: datetime | None = None) → str¶

Create a session token.

The session token is a short-lived token that allows the bearer to make inferences with the session (via an InferenceSessionClient) and to call ready(), get(), and terminate() on the session.

Parameters:

session_id (str) – The ID of the session.
expires (Optional[datetime]) – The expiration time of the token. Must be < the expiration time of the session. Default: expiration time of the session.

Raises:

HttpResponseError –

class dyff.client.client.ModelsOperations¶

Operations on Model entities.

Note

Do not instantiate this class. Access it through the .models attribute of Client.

create(model_request: ModelCreateRequest) → Model¶

Create a Model.

Note

This operation will incur compute costs.

Parameters:: model_request – The model specification.
Returns:: A full Model entity with .id and other properties set.

delete(model_id: str) → Status¶

Mark a Model for deletion.

Parameters:: model_id (str) – The model key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status

documentation(model_id: str) → Documentation¶

Get the documentation associated with a Model.

Get the documentation associated with a Model. Raises a 404 error if no entity exists with that key.

Parameters:: model_id (str) – The ID of the Model.
Returns:: The documentation associated with the Model.
Return type:: Documentation :raises ~azure.core.exceptions.HttpResponseError:

edit_documentation(model_id: str, edit_request: DocumentationEditRequest) → Documentation¶

Edit the documentation associated with a Model.

Edit the documentation associated with a Method. Raises a 404 error if no entity exists with that key. Returns the modified Documentation.

Parameters:

model_id (str) – The ID of the Model.
edit_request (DocumentationEditRequest) – Object containing the edits to make.

Returns:

The modified documentation.

Return type:

Documentation :raises ~azure.core.exceptions.HttpResponseError:

get(model_id: str) → Model¶

Get a Model by its key.

Parameters:: model_id (str) – The inference service id
Returns:: The Model with the given key.

label(model_id: str, labels: dict[str, str | None]) → None¶

Label the specified Model with key-value pairs (stored in the .labels field of the resource).

Providing None for the value deletes the label.

See Label for a description of the constraints on label keys and values.

Parameters:

model_id (str) – The ID of the Model to label.
labels (dict[str, Optional[str]]) – The label keys and values.

Get all Models matching a query. The query is a set of equality constraints specified as key-value pairs.

Parameters:

query (str | dict[str, Any] | list[dict[str, Any]])
id (str)
account (str)
status (str)
reason (str)
labels (dict[str, str])
name (str)

Returns:

list of Model resources satisfying the query.

Return type:

list[Model]

Raises:

HttpResponseError –

class dyff.client.client.ReportsOperations¶

Operations on Report entities.

Note

Do not instantiate this class. Access it through the .reports attribute of Client.

create(report_request: ReportCreateRequest) → Report¶

Create a Report.

Note

This operation will incur compute costs.

Parameters:: report_request – The report specification.
Returns:: A full Report entity with .id and other properties set.

delete(report_id: str) → Status¶

Mark a Report for deletion.

Parameters:: report_id (str) – The report key
Returns:: The resulting status of the entity
Return type:: dyff.schema.platform.Status

downlinks(report_id: str) → list[ArtifactURL]¶

Get a list of signed GET URLs from which Report artifacts can be downloaded.

Parameters:: report_id (str) – The ID of the Report.
Returns:: List of signed GET URLs.
Return type:: list[ArtifactURL] :raises ~azure.core.exceptions.HttpResponseError:

download(report_id: str, destination: Path | str) → None¶

Download all of the files in a report to a local directory.

The destination directory must be an empty directory that exists.

Parameters:

report_id (str) – The ID of the Report.
destination (Path | str) – The destination directory. Must exist and be empty.

Raises:

HttpResponseError – If reports.downlinks(report_id) raises
ValueError – If arguments are invalid

download_logs(report_id, destination: Path | str) → None¶

Download the logs file from the report.

The destination must be a file path that does not exist, and its parent directory must exist.

Parameters:

report_id (str) – The ID of the Report.
destination (Path | str) – The destination file. Must not exist, and its parent directory must exist.

Raises:

HttpResponseError – If raw.reports.logs(report_id) raises
ValueError – If arguments are invalid

get(report_id: str) → Report¶

Get a Report by its key.

Parameters:: report_id (str) – The report id
Returns:: The Report with the given key.

label(report_id: str, labels: dict[str, str | None]) → None¶

Label the specified Report with key-value pairs (stored in the .labels field of the resource).

Providing None for the value deletes the label.

See Label for a description of the constraints on label keys and values.

Parameters:

report_id (str) – The ID of the Report to label.
labels (dict[str, Optional[str]]) – The label keys and values.

logs(report_id) → Iterable[str]¶

Stream the logs from the report as a sequence of lines.

Parameters:

report_id (str) – The ID of the Report.

Returns:

An Iterable over the lines in the logs file. The response is streamed, and may time out if it is not consumed quickly enough.

Return type:

Iterable[str]

Raises:

HttpResponseError – If raw.reports.logs(report_id) raises, or if the streaming response times out or encounters some other error.
ValueError – If arguments are invalid

Get all Reports matching a query. The query is a set of equality constraints specified as key-value pairs.

Parameters:

query (str | dict[str, Any] | list[dict[str, Any]])
id (str)
account (str)
status (str)
reason (str)
labels (dict[str, str])
report (str)
dataset (str)
evaluation (str)
inferenceService (str)
model (str)

Returns:

list of Report resources satisfying the query.

Return type:

list[Report]

Raises:

HttpResponseError –