Data Types

Result bases and the containers that hold them. Custom features return a FeatureResult, custom metrics a MetricResult; at run time those are collected into the containers below.

FeatureResult

Bases: ArbitraryData

Specific container for feature results.

MetricResult

Bases: ArbitraryData

Specific container for metric results.

ArbitraryData

Bases: BaseModel

Pydantic model for arbitrary data.

This model allows for any extra fields to be added to the data, making it flexible for various use cases.

FeatureResultContainer

Bases: BaseModel

Feature results container.

`get_all(feature_cls: FeatureClass[TFeatureResult]) -> Mapping[FeatureName, TFeatureResult]`

Get all results for a class.

`get_all_with_config(feature_cls: FeatureClass[TFeatureResult]) -> Mapping[FeatureName, tuple[TFeatureResult, BaseFeature[TFeatureResult]]]`

Get all results for a given feature class.

Parameters:

feature_cls (FeatureClass) –

The feature class to retrieve results for.

Returns:

Mapping[FeatureName, FeatureComputed] –

A mapping of feature names to their corresponding results and configurations.

Raises:

FeatureResultWrongClassError –

If the provided feature class is not allowed in this FeatureResults instance.

`get(feature_cls: FeatureClass[TFeatureResult], feature_name: FeatureName) -> TFeatureResult`

Get a single result for a given feature class and name.

Parameters:

feature_cls (FeatureClass) –

The feature class to retrieve the result for.
feature_name (FeatureName) –

The name of the feature to retrieve the result for.

Returns:

FeatureResult –

The result for the specified feature (without configuration).

Raises:

FeatureResultWrongClassError –

If the provided feature class is not allowed.
FeatureResulUnknownNameError –

If the provided feature name is not found for the given class.

`get_with_config(feature_cls: FeatureClass[TFeatureResult], feature_name: FeatureName) -> tuple[TFeatureResult, BaseFeature[TFeatureResult]]`

Get a single result for a given feature class and name.

Parameters:

feature_cls (FeatureClass) –

The feature class to retrieve the result for.
feature_name (FeatureName) –

The name of the feature to retrieve the result for.

Returns:

FeatureComputed –

The result and configuration for the specified feature.

Raises:

FeatureResultWrongClassError –

If the provided feature class is not allowed.
FeatureResulUnknownNameError –

If the provided feature name is not found for the given class.

`first(feature_cls: FeatureClass[TFeatureResult]) -> TFeatureResult`

Retrieve the first result for a given feature class.

Parameters:

feature_cls (FeatureClass[TFeatureResult]) –

The class for which the first result should be retrieved.

Returns:

FeatureResult –

The first feature result for the given class (without configuration).

Raises:

FeatureResultWrongClassError –

If the provided feature class is not allowed.

`first_with_config(feature_cls: FeatureClass[TFeatureResult]) -> tuple[TFeatureResult, BaseFeature[TFeatureResult]]`

Retrieve the first result with its configuration for a given feature class.

Parameters:

feature_cls (FeatureClass[TFeatureResult]) –

The class for which the first result should be retrieved.

Returns:

FeatureComputed –

A tuple containing the first feature result and its configuration for the given class.

Raises:

FeatureResultWrongClassError –

If the provided feature class is not allowed.

MetricResultContainer

Bases: BaseModel

Metric results container.

`get_all_with_config(metric_cls: MetricClass[TMetricResult]) -> Mapping[MetricName, tuple[TMetricResult, BaseMetric[TMetricResult]]]`

Get all results for a given metric class with their configurations.

Parameters:

metric_cls (MetricClass) –

The metric class to retrieve results for.

Returns:

Mapping[MetricName, MetricComputed] –

A mapping of metric names to their corresponding results and configurations.

Raises:

MetricResultWrongClassError –

If the provided metric class is not allowed in this MetricResults instance.

`get_all(metric_cls: MetricClass[TMetricResult]) -> Mapping[MetricName, TMetricResult]`

Get all results for a given metric class.

Parameters:

metric_cls (MetricClass) –

The metric class to retrieve results for.

Returns:

Mapping[MetricName, MetricResult] –

A mapping of metric names to their corresponding results (without configurations).

Raises:

MetricResultWrongClassError –

If the provided metric class is not allowed in this MetricResults instance.

`get(metric_cls: MetricClass[TMetricResult], metric_name: MetricName) -> TMetricResult`

Get a single result for a given metric class and name.

Parameters:

metric_cls (MetricClass) –

The metric class to retrieve the result for.
metric_name (MetricName) –

The name of the metric to retrieve the result for.

Returns:

MetricResult –

The result for the specified metric (without configuration).

Raises:

MetricResultWrongClassError –

If the provided metric class is not allowed.
MetricResulUnknownNameError –

If the provided metric name is not found for the given class.

`get_with_config(metric_cls: MetricClass[TMetricResult], metric_name: MetricName) -> tuple[TMetricResult, BaseMetric[TMetricResult]]`

Get a single result with its configuration for a given metric class and name.

Parameters:

metric_cls (MetricClass) –

The metric class to retrieve the result for.
metric_name (MetricName) –

The name of the metric to retrieve the result for.

Returns:

MetricComputed –

The result and configuration for the specified metric.

Raises:

MetricResultWrongClassError –

If the provided metric class is not allowed.
MetricResulUnknownNameError –

If the provided metric name is not found for the given class.

`first(metric_cls: MetricClass[TMetricResult]) -> TMetricResult`

Retrieve the first result for a given metric class.

Parameters:

metric_cls (MetricClass[TMetricResult]) –

The class for which the first result should be retrieved.

Returns:

MetricResult –

The first metric result for the given class (without configuration).

Raises:

MetricResultWrongClassError –

If the provided metric class is not allowed.

`first_with_config(metric_cls: MetricClass[TMetricResult]) -> tuple[TMetricResult, BaseMetric[TMetricResult]]`

Retrieve the first result with its configuration for a given metric class.

Parameters:

metric_cls (MetricClass[TMetricResult]) –

The class for which the first result should be retrieved.

Returns:

MetricComputed –

A tuple containing the first metric result and its configuration for the given class.

Raises:

MetricResultWrongClassError –

If the provided metric class is not allowed.

BenchmarkResultContainer

Bases: BaseModel

Container for benchmark outputs grouped by model and algorithm.

`get_all_metrics() -> Generator[tuple[ModelName, AlgorithmName, MetricResultContainer]]`

Yield all metric result groups across models and algorithms.

Yields:

tuple[ModelName, AlgorithmName, MetricResults] –

A tuple containing the model name, algorithm name, and corresponding metric results.

`get_all_metrics_of_type(metric_cls: MetricClass[TMetricResult]) -> Generator[tuple[ModelName, AlgorithmName, TMetricResult]]`

Yield all metric results matching a specific metric class.

Parameters:

metric_cls (MetricClass) –

The metric class used to filter metric results.

Yields:

tuple[ModelName, AlgorithmName, MetricResult] –

A tuple containing the model name, algorithm name, and each non-null metric result of the requested type.