sagemaker.serve.ai_inference_recommender.result

Parsing of benchmark output artifacts from S3.

Classes

BenchmarkMetric(name, unit, avg, min, max, ...)	A single benchmark metric with its statistical aggregates.
BenchmarkMetrics(request_throughput, ...)	Typed access to the well-known AIPerf metrics.
BenchmarkResult(metrics, s3_output_location, ...)	Parsed result of a completed benchmark job.

class sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric(name: str, unit: str | None = None, avg: float | None = None, min: float | None = None, max: float | None = None, p50: float | None = None, p90: float | None = None, p95: float | None = None, p99: float | None = None, stddev: float | None = None, raw: ~typing.Dict[str, ~typing.Any] = <factory>)

Bases: object

A single benchmark metric with its statistical aggregates.

avg: float | None = None

classmethod from_dict(name: str, data: Dict[str, Any]) → BenchmarkMetric

max: float | None = None

min: float | None = None

name: str

p50: float | None = None

p90: float | None = None

p95: float | None = None

p99: float | None = None

raw: Dict[str, Any]

stddev: float | None = None

unit: str | None = None

class sagemaker.serve.ai_inference_recommender.result.BenchmarkMetrics(request_throughput: ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric | None = None, request_latency: ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric | None = None, time_to_first_token: ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric | None = None, inter_token_latency: ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric | None = None, output_token_throughput: ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric | None = None, all_metrics: ~typing.Dict[str, ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetric] = <factory>)

Bases: object

Typed access to the well-known AIPerf metrics.

Use .get(name) to look up any metric by its raw key. print()-ing this object renders every metric in a table; print(result) (the parent BenchmarkResult) shows just the well-known metrics.

all_metrics: Dict[str, BenchmarkMetric]

classmethod from_profile_json(profile: Dict[str, Any]) → BenchmarkMetrics

get(name: str) → BenchmarkMetric | None

inter_token_latency: BenchmarkMetric | None = None

output_token_throughput: BenchmarkMetric | None = None

request_latency: BenchmarkMetric | None = None

request_throughput: BenchmarkMetric | None = None

time_to_first_token: BenchmarkMetric | None = None

class sagemaker.serve.ai_inference_recommender.result.BenchmarkResult(metrics: ~sagemaker.serve.ai_inference_recommender.result.BenchmarkMetrics, s3_output_location: str, endpoint: str | None = None, workload_config: str | None = None, tool_version: str | None = None, profile: ~typing.Dict[str, ~typing.Any] = <factory>)

Bases: object

Parsed result of a completed benchmark job.

endpoint: str | None = None

classmethod from_job(job, *, session: boto3.session.Session | None = None) → BenchmarkResult

Download and parse the benchmark output for a completed AIBenchmarkJob.

Populates endpoint, workload_config, and tool_version from the job’s BenchmarkTarget and WorkloadConfigIdentifier plus the AIPerf profile metadata so the parsed result is self-describing.

Parameters:

• job – An AIBenchmarkJob (or BenchmarkJob re-export) that has reached a terminal state.

• session – Optional boto3 session. Defaults to the ambient session.

Returns:

A parsed BenchmarkResult.

Raises:

RuntimeError – if the job has no S3 output location set.

classmethod from_s3(s3_output_location: str, *, session: boto3.session.Session | None = None, endpoint: str | None = None, workload_config: str | None = None) → BenchmarkResult

Download and parse the benchmark output artifact from S3.

Parameters:

• s3_output_location – s3://bucket/prefix/ location written by the benchmark job.

• session – Optional boto3 session. Defaults to the ambient session.

• endpoint – Optional endpoint identifier to attach to the result. Threaded through by from_job().

• workload_config – Optional workload-config identifier to attach. Threaded through by from_job().

Returns:

A parsed BenchmarkResult.

metrics: BenchmarkMetrics

profile: Dict[str, Any]

s3_output_location: str

tool_version: str | None = None

workload_config: str | None = None