Benchmark Library Documentation

This guide explains how to maintain the generated benchmark library documentation. For user-facing benchmark discovery and command examples, see Benchmark Catalog.

Fastpath treats the benchmark library as an ordered set of library paths. User configured benchmark libraries are searched before the default Fastpath benchmark library. When the same suite/name benchmark id is present in more than one library, the first definition found is used for documentation generation.

Documentation Inputs

Benchmark documentation has two inputs:

  • Benchmark YAML files from the configured Fastpath benchmark libraries.

  • Documentation metadata from documentation.yaml files at the root of those benchmark libraries.

Fastpath combines those inputs into one RST page per benchmark id under documentation/user-guide/benchmarkcatalog/. The generated file name is based on the suite/name benchmark id, with / replaced by -. For example, lmbench/lat-mem-rd is generated as lmbench-lat-mem-rd.rst.

The same library ordering applies to documentation metadata. If duplicate documentation entries are found for a benchmark id, the first entry found is used.

Add Benchmark Documentation

When adding or updating a benchmark, add documentation metadata for the benchmark id in benchmarks/documentation.yaml. The key must use the suite/name form used by the benchmark YAML.

The metadata should describe what the benchmark measures, the Fastpath parameters exposed by the benchmark YAML, and the result classes produced by the benchmark container. The fastpath_parameters names must match the merged benchmark YAML params keys, including inherited parameters from included base YAML files. Parameter descriptions remain manually maintained in documentation.yaml.

Add benchmark_profile metadata for each benchmark so the generated catalog page carries the same high-level classification for every benchmark:

  • type should be Micro for focused kernel-path or primitive benchmarks, and Macro for broader workloads where the result includes application, runtime, framework, or system-level behavior.

  • kernel_subsystem should name Linux kernel subsystems, not only the user-space application. For example, a database workload can mention filesystem I/O, memory management, networking, and scheduler behavior when those areas are the likely kernel sources of sensitivity.

  • coverage should say whether the metric is directly measured from the named kernel-facing operation or inferred from a larger workload metric.

  • topology should describe the Fastpath role layout. Use Single role: executer for benchmarks that do not declare explicit roles; executer is the default Fastpath role. For multi-role benchmarks, list the declared roles and note when multi-node placement is preferred.

Generation Rules

During documentation generation, Fastpath scans the configured benchmark libraries in lookup order, loads runnable benchmark YAML files with suite and name fields, merges any matching documentation metadata, and renders one RST page per benchmark id.

If documentation metadata is missing, Fastpath still renders a page with a documentation status note and the metadata available from the benchmark YAML. This keeps the catalog complete while making missing documentation visible.

Generate Benchmark RST

Any change to a benchmark YAML file, benchmark parameters, result classes, or benchmark documentation metadata must be reflected in the generated RST page in Fastpath’s benchmark catalog documentation.

Use fastpath benchmark show with --output to regenerate the RST for one benchmark:

fastpath benchmark show \
  --id lmbench/lat-mem-rd \
  --output documentation/user-guide/benchmarkcatalog/lmbench-lat-mem-rd.rst

After generating a new benchmark page, add it to the benchmark library toctree in documentation/user-guide/benchmarkcatalog/index.rst.

Check Documentation Sync

Run the benchmark documentation sync check after updating benchmark YAML, documentation metadata, or generated benchmark RST files:

python3 documentation/check_bmlib_docsync.py

The check verifies that the generated benchmark RST files match the current benchmark YAML files and documentation metadata. It reports missing, extra, or unlisted benchmark pages, and validates that documented Fastpath parameter names match the merged benchmark YAML parameter interface.

Build Documentation

Build the full Sphinx documentation after updating generated pages:

sphinx-build -b html -a -W documentation public

See Compile Documentation Locally for the complete local documentation build setup.