Plots

Overview

Plots turn the metrics and features computed during a benchmark into charts. They run as the last stage of the pipeline, after every algorithm, feature, and metric has finished, so a plot has access to the full set of results.

A plot subclasses BasePlot, declares the metrics or features it draws by passing their classes to the @plot decorator, and implements run(self, benchmark_results). Both BasePlot and @plot come from luna_bench.custom.

Built-in Plots

Import these from luna_bench.plots and add them like any other component:

from luna_bench.plots import AverageRuntimePlot, AverageApproximationRatioPlot

bench.add_plot("avg-runtime", AverageRuntimePlot())
bench.add_plot("avg-approx", AverageApproximationRatioPlot())

Each built-in plot already knows which metric or feature it needs, so adding it without the matching metric raises an error when the benchmark runs.

Performance Plots

Analysis Plots

Property Plots

Quick charts from a DataFrame

You don't have to register a @plot to explore results. bench.results_to_dataframe() returns a pandas DataFrame with one row per (algorithm, model) pair, so you can chart it directly with seaborn or matplotlib — handy for getting insights fast before committing to a reusable plot.

import seaborn as sns
import matplotlib.pyplot as plt

df = bench.results_to_dataframe()
# Columns: "algorithm", "model", plus one "<name>/<field>" column per feature and metric,
# e.g. "runtime/runtime_seconds", "approx-ratio/approximation_ratio", "num-vars/var_number".

sns.barplot(df, x="algorithm", y="runtime/runtime_seconds")
plt.show()

Each metric and feature contributes its result fields as "<name>/<field>" columns, where <name> is the ID you passed to add_metric or add_feature.

Plot Base Classes

Custom plots subclass BasePlot and implement run. For common chart types you can instead subclass a helper from luna_bench.plots.generics, which adds a create(...) method so you don't write matplotlib code by hand:

Writing Custom Plots

Pass the metric or feature classes the plot draws to @plot, then read them from the BenchmarkResultContainer in run. You can write a class or a plain function:

from luna_bench.custom import plot, BenchmarkResultContainer
from luna_bench.metrics import ApproximationRatio


@plot(ApproximationRatio)
def my_approx_plot(benchmark_results: BenchmarkResultContainer) -> None:
    for _model, algo, result in benchmark_results.get_all_metrics_of_type(ApproximationRatio):
        print(algo, result.approximation_ratio)

from luna_bench.custom import plot, BasePlot, BenchmarkResultContainer
from luna_bench.metrics import ApproximationRatio


@plot(ApproximationRatio)
class MyApproxPlot(BasePlot):
    def run(self, benchmark_results: BenchmarkResultContainer) -> None:
        for model_name, algorithm_name, result in benchmark_results.get_all_metrics_of_type(
            ApproximationRatio
        ):
            print(model_name, algorithm_name, result.approximation_ratio)
            # draw with matplotlib / seaborn here

get_all_metrics_of_type(MetricClass) yields (model_name, algorithm_name, result) for every (model, algorithm) pair, with result typed as that metric's result. Use get_all_metrics() to walk every metric at once.

Using a chart helper

Subclass a helper instead of BasePlot to get a create(...) method for the chart:

from luna_bench.custom import plot, BenchmarkResultContainer
from luna_bench.metrics import ApproximationRatio
from luna_bench.plots.generics.bar_plot import BarPlot


@plot(ApproximationRatio)
class AvgApproxBar(BarPlot):
    def run(self, benchmark_results: BenchmarkResultContainer) -> None:
        rows = [
            {"algorithm": algo, "approximation_ratio": result.approximation_ratio}
            for _model, algo, result in benchmark_results.get_all_metrics_of_type(ApproximationRatio)
        ]
        self.create(
            rows=rows,
            x="algorithm",
            y="approximation_ratio",
            xlabel="Algorithm",
            ylabel="Approximation Ratio",
            title="Average Approximation Ratio per Solver",
        )

The built-in plots under luna_bench/plots/ are the best reference for the helper APIs.

Adding a plot to a benchmark

bench.add_plot("my-plot", MyApproxPlot())

Error Handling

A failing plot does not stop the rest. If a plot's required data is missing or its run method raises, Luna Bench logs a warning and moves on to the next plot, so run_plots() (and run()) finish even when some plots are skipped. Check the logs to see which plots were skipped and why.