Advanced Customization of the Benchmark Pipeline

In this tutorial, we're exploring advanced customization options within LunaBench, specifically tailored for those working with custom optimization problems. This guide is designed for users requiring precise control over their benchmarking setup. From fine-tuning solver configurations to running multiple algorithms in parallel and customizing output directories, this tutorial provides a comprehensive walkthrough for enhancing your benchmarking processes with LunaBench.

Before we begin, ensure you have installed the LunaBencch Python SDK.

Upon completing this tutorial, you will be able to:

1. Preprocess problem instances for more efficient optimization.
2. Configure algorithms with additional arguments for tailored solver performance.
3. Utilize multiprocessing to run multiple algorithms concurrently, optimizing computational resources.
4. Customize output directories to fit your workflow preferences.
5. Launch the benchmark pipeline with all your configurations in place, streamlining the process of solving your custom datasets.

1. Creating a Dataset of QUBOs

To start, we'll begin with assembling a straightforward dataset. For the purposes of this tutorial, our focus will be on crafting a basic dataset comprised of QUBOs.

# Define the problem type as qubo
problem_name = "qubo"

# Define the dataset
dataset = {
    "qubo_00": {
        problem_name: [
            [2, 1],
            [1, 2]
        ]
    },
    "qubo_01": {
        problem_name: [
            [-2, 3],
            [3, -2]
        ]
    },
    "qubo_02": {
        problem_name: [
            [2, 1, 0],
            [1, 2, 1],
            [0, 1, 2]
        ]
    },
}

2. Preprocessing Problem Instances

Before diving into solving optimization problems, it's beneficial to preprocess these problems to streamline and simplify them. This step can improve result quality or decrease the amount of computational time and resources needed. It can help to identify and eliminate unnecessary constraints and variables, and to simplify the overall structure of the problem, making the optimization process more straightforward and efficient.

LunaBench provides functionalities for preprocessing problem instances. You can specify preprocessing steps in a manner similar to how solving algorithms are designated, using either a list of algorithm names or a dictionary configuration. For details on the preprocessing algorithms that LunaBench currently supports, please refer to the LunaBench GitHub page.

# Specify the preprocessing algorithms
preprocessing_algorithms = ["normalize_qubo"]

3. Configuring the Algorithms

The effectiveness of solver algorithms can differ greatly depending on their configurations. LunaBench emphasizes the importance of customization, enabling users to adjust solver settings to suit particular needs. This customization is facilitated by adding extra arguments for the chosen solver, which are structured in a dictionary format. In this setup, the dictionary keys represent the names of the algorithms, and the values are dictionaries themselves, containing specific configurations for each algorithm.

For example, you have the option to set the optimizer for the hybrid VQE (Variational Quantum Eigensolver) algorithm to match your exact requirements, among other customizable settings.

# specify the algorithms with their configurations
solve_algorithms = {
    "sa": {"n_iter": 100, "n_temp_iter": 1000, "temp_start": 100},
    "vqe": {"optimizer": "COBYLA", "maxiter": 100},
}


# define the number of runs for each algorithm
n_runs = 5

4. Running multiple algorithms in parallel

Running just one algorithm at a time for each problem instance might not make the most of your hardware's computational power. To address this, LunaBench offers the ability to run multiple algorithms simultaneously using multiprocessing. This feature lets you set the maximum number of processes that can run at the same time, enhancing efficiency and potentially reducing overall computational time.

While multiprocessing can significantly speed up the solving process by utilizing your hardware more effectively, it's worth noting that there might be additional overhead. This overhead could, in some scenarios, offset the benefits of parallel processing. It's recommended to employ this feature when you're confident that your hardware setup is optimized for multiprocessing, ensuring you achieve the desired performance improvement.

# Specify the maximum number of parallel running algorithms
process_count = 2

5. Customizing the Output Directory

By default, LunaBench generates an output directory named luna_results, organizing each dataset and its corresponding results with unique identifiers. Recognizing that users might have different preferences for managing their workflow, LunaBench allows for the customization of output directories.

To tailor this to your workflow, you can specify the output path, dataset name, and result ID. This customization ensures that you can easily locate your solutions at {$output_path}/{$dataset_name}/solve/{$result_id}/solutions.csv.

# Define the output path variables
output_path = "results_tutorial"
dataset_name = "qubo_example"
result_id = "run_00"

6. Launching the Benchmark Pipeline

Once you've fine-tuned all the settings and configurations to your preference, solving the dataset follows the standard process. All your custom configurations can be seamlessly integrated into the benchmark pipeline as parameters.

from lunabench import solve_dataset

# Solve the dataset with all customization options
solved, result_id = solve_dataset(
    problem_name=problem_name,
    dataset_name=dataset_name,
    dataset=dataset,
    solve_algorithms=solve_algorithms,
    n_runs=n_runs,
    pre_algorithms=preprocessing_algorithms,
    process_count=process_count,
    result_id=result_id,
    save_path=output_path,
)