Skip to content

guidellm.schemas.benchmark.profiles.profile

Profile argument schemas for multi-strategy benchmark execution.

Defines the base argument model for profile configuration, including warmup and cooldown phase settings. Uses Pydantic class registry for polymorphic deserialization of profile-specific argument types.

ProfileArgs

Bases: PydanticClassRegistryMixin['ProfileArgs'], ABC

Base class for profile creation arguments.

This class serves as a base for defining argument models used in the creation of profile instances. It inherits from PydanticClassRegistryMixin to enable automatic registration of subclasses, allowing for flexible and extensible profile configurations.

Attributes:

Name Type Description
schema_discriminator str

Field name for polymorphic deserialization

Source code in src/guidellm/schemas/benchmark/profiles/profile.py
class ProfileArgs(PydanticClassRegistryMixin["ProfileArgs"], ABC):
    """Base class for profile creation arguments.

    This class serves as a base for defining argument models used in the creation
    of profile instances. It inherits from PydanticClassRegistryMixin to enable
    automatic registration of subclasses, allowing for flexible and extensible
    profile configurations.

    :cvar schema_discriminator: Field name for polymorphic deserialization
    """

    model_config = standard_model_config()

    schema_discriminator: ClassVar[str] = "kind"

    @classmethod
    def __pydantic_schema_base_type__(cls) -> type[ProfileArgs]:
        """
        Return base type for polymorphic validation hierarchy.

        :return: Base ProfileArgs class for schema validation
        """
        if cls.__name__ == "ProfileArgs":
            return cls

        return ProfileArgs

    kind: str = Field(
        description="Profile type discriminator",
        examples=["concurrent", "synchronous"],
    )
    rampup_duration: NonNegativeFloat = Field(
        default=0.0,
        description=("Duration in seconds to ramp up the targeted scheduling rate"),
    )
    warmup: TransientPhaseConfig = Field(
        default_factory=TransientPhaseConfig,
        description="Warmup phase to exclude initial transient period",
        examples=[0.0, 1.0, {"mode": "percent", "percent": 2.0}],
    )
    cooldown: TransientPhaseConfig = Field(
        default_factory=TransientPhaseConfig,
        description="Cooldown phase to exclude final transient period",
        examples=[0.0, 1.0, {"mode": "duration", "value": 2.0}],
    )

    def validate_metrics(self, metrics: Any) -> None:
        """
        Check the metrics configuration supports this profile.

        Called once the whole benchmark configuration has validated, so a
        profile that needs a particular metric configured can say so before the
        run starts rather than failing partway through. Defaults to accepting
        any configuration.

        :param metrics: Validated metrics arguments for the run
        :raises ValueError: If the metrics configuration cannot support this
            profile
        """

    @field_validator("warmup", "cooldown", mode="before")
    @classmethod
    def _coerce_transient_phase(cls, v: Any) -> Any:
        if isinstance(v, str):
            with contextlib.suppress(json.JSONDecodeError, ValueError):
                v = json.loads(v)
        if isinstance(v, int | float | None):
            return TransientPhaseConfig.create_from_value(v)
        return v

__pydantic_schema_base_type__() classmethod

Return base type for polymorphic validation hierarchy.

Returns:

Type Description
type[ProfileArgs]

Base ProfileArgs class for schema validation

Source code in src/guidellm/schemas/benchmark/profiles/profile.py
@classmethod
def __pydantic_schema_base_type__(cls) -> type[ProfileArgs]:
    """
    Return base type for polymorphic validation hierarchy.

    :return: Base ProfileArgs class for schema validation
    """
    if cls.__name__ == "ProfileArgs":
        return cls

    return ProfileArgs

validate_metrics(metrics)

Check the metrics configuration supports this profile.

Called once the whole benchmark configuration has validated, so a profile that needs a particular metric configured can say so before the run starts rather than failing partway through. Defaults to accepting any configuration.

Parameters:

Name Type Description Default
metrics Any

Validated metrics arguments for the run

required

Raises:

Type Description
ValueError

If the metrics configuration cannot support this profile

Source code in src/guidellm/schemas/benchmark/profiles/profile.py
def validate_metrics(self, metrics: Any) -> None:
    """
    Check the metrics configuration supports this profile.

    Called once the whole benchmark configuration has validated, so a
    profile that needs a particular metric configured can say so before the
    run starts rather than failing partway through. Defaults to accepting
    any configuration.

    :param metrics: Validated metrics arguments for the run
    :raises ValueError: If the metrics configuration cannot support this
        profile
    """