Rust Benchmarking

What is Benchmarking in Rust?

Benchmarking in Rust involves measuring the time taken by specific sections of code to execute. This helps in:

1. Identifying Slow Code: Find parts of your program that need optimization.
2. Comparing Implementations: Test different approaches to see which one performs better.
3. Improving Application Speed: Write faster, more efficient code.

Setting Up Benchmarking in Rust

By default, benchmarking in Rust is supported using the bencher crate, but it requires using nightly Rust. Here’s how you can set it up:

1. Install Rust Nightly

Switch to the nightly version of Rust:

rustup install nightly
rustup default nightly

2. Enable Benchmarks in Cargo.toml

Add the following to your Cargo.toml file:

[dev-dependencies]
bencher = "0.1"

Enable the test benchmark feature:

[[bench]]
name = "my_benchmark"
harness = false

Writing a Benchmark

Benchmarks are placed in the benches directory at the root of your project. Each file in this directory contains benchmarking functions.

Example: Benchmarking a Function

File: src/lib.rs

pub fn compute_sum(n: u64) -> u64 {
    (1..=n).sum()
}

File: benches/my_benchmark.rs

extern crate bencher;

use bencher::Bencher;
use my_project::compute_sum; // Replace `my_project` with your crate name

fn benchmark_compute_sum(b: &mut Bencher) {
    b.iter(|| {
        compute_sum(1_000); // Benchmark the compute_sum function with input 1,000
    });
}

benchmark_group!(benches, benchmark_compute_sum);
benchmark_main!(benches);

Running Benchmarks

Run the benchmarks using:

cargo bench

Output Example

running 1 test
test benchmark_compute_sum ... bench:      1,234 ns/iter (+/- 56)

Explanation:

• 1,234 ns/iter: The function took an average of 1,234 nanoseconds per iteration.
• +/- 56: Indicates the margin of error.

Benchmarking Tips

1. Avoid External Factors: Run benchmarks on a dedicated machine to avoid interference from other processes.
2. Use Large Inputs: Small inputs might not show meaningful differences in performance.
3. Run Multiple Iterations: Benchmarking tools like bencher automatically run multiple iterations to get accurate results.

Using Criterion for Advanced Benchmarking

The Criterion crate provides a more flexible and stable way to perform benchmarking in Rust. Unlike bencher, it works on the stable version of Rust.

1. Add Criterion to Cargo.toml

[dev-dependencies]
criterion = "0.4"

2. Write a Benchmark

File: benches/criterion_benchmark.rs

use criterion::{black_box, Criterion, criterion_group, criterion_main};
use my_project::compute_sum; // Replace `my_project` with your crate name

fn criterion_benchmark(c: &mut Criterion) {
    c.bench_function("compute_sum 1_000", |b| {
        b.iter(|| compute_sum(black_box(1_000)));
    });
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

3. Run Criterion Benchmarks

Run the benchmarks using:

cargo bench

Output Example:

compute_sum 1_000
                        time:   [1.234 ns 1.567 ns 1.890 ns]

Explanation:

• time: Shows the average time taken for the benchmark.

Comparing Multiple Implementations

Benchmarks are useful for comparing different implementations of the same functionality.

Example: Comparing Two Functions

File: src/lib.rs

pub fn compute_sum_loop(n: u64) -> u64 {
    let mut sum = 0;
    for i in 1..=n {
        sum += i;
    }
    sum
}

pub fn compute_sum_builtin(n: u64) -> u64 {
    (1..=n).sum()
}

File: benches/compare_benchmark.rs

use criterion::{Criterion, criterion_group, criterion_main};
use my_project::{compute_sum_loop, compute_sum_builtin};

fn compare_sums(c: &mut Criterion) {
    c.bench_function("compute_sum_loop 1_000", |b| {
        b.iter(|| compute_sum_loop(1_000));
    });

    c.bench_function("compute_sum_builtin 1_000", |b| {
        b.iter(|| compute_sum_builtin(1_000));
    });
}

criterion_group!(benches, compare_sums);
criterion_main!(benches);

When to Use Benchmarking?

1. Performance-Critical Applications: Programs where speed is crucial (e.g., game engines, data processing).
2. Comparing Implementations: Test which function or algorithm is faster.
3. Optimizing Code: Identify and improve bottlenecks.