guidellm.cli

GuideLLM command-line interface entry point.

Primary CLI application providing benchmark execution, dataset preprocessing, and mock server functionality for language model evaluation. Organizes commands into three main groups: benchmark operations for performance testing, preprocessing utilities for data transformation, and mock server capabilities for development and testing. Supports multiple backends, output formats, and flexible configuration through CLI options and environment variables.

Example: :: # Run a benchmark against a model guidellm run --backend kind=openai_http,target=http://localhost:8000 \ --profile kind=sweep \ --data kind=synthetic_text,prompt_tokens=256,output_tokens=128

# Export a saved benchmark report
guidellm export benchmarks.json --output kind=json,path=report.json

# Preprocess a dataset
guidellm preprocess dataset kind=json_file,path=input.json \
    output.json --tokenizer kind=huggingface_auto,model=gpt2 \
    --strategy kind=ignore,prompt_tokens=128,output_tokens=64

# Start a mock server for testing
guidellm mock-server --host 0.0.0.0 --port 8080

43
44
45
46

@click.group()
@click.version_option(package_name="guidellm", message="guidellm version: %(version)s")
def cli():
    """GuideLLM CLI for benchmarking, preprocessing, and testing language models."""