Session¶
Placeholder docstring
- class sagemaker.session.LogState¶
Bases:
object
Placeholder docstring
- STARTING = 1¶
- WAIT_IN_PROGRESS = 2¶
- TAILING = 3¶
- JOB_COMPLETE = 4¶
- COMPLETE = 5¶
- class sagemaker.session.Session(boto_session=None, sagemaker_client=None, sagemaker_runtime_client=None, sagemaker_featurestore_runtime_client=None, default_bucket=None, settings=None, sagemaker_metrics_client=None, sagemaker_config=None, default_bucket_prefix=None)¶
Bases:
object
Manage interactions with the Amazon SageMaker APIs and any other AWS services needed.
This class provides convenient methods for manipulating entities and resources that Amazon SageMaker uses, such as training jobs, endpoints, and input datasets in S3. AWS service calls are delegated to an underlying Boto3 session, which by default is initialized using the AWS configuration chain. When you make an Amazon SageMaker API call that accesses an S3 bucket location and one is not specified, the
Session
creates a default bucket based on a naming convention which includes the current AWS account ID.Initialize a SageMaker
Session
.- Parameters
boto_session (boto3.session.Session) – The underlying Boto3 session which AWS service calls are delegated to (default: None). If not provided, one is created with default AWS configuration chain.
sagemaker_client (boto3.SageMaker.Client) – Client which makes Amazon SageMaker service calls other than
InvokeEndpoint
(default: None). Estimators created using thisSession
use this client. If not provided, one will be created using this instance’sboto_session
.sagemaker_runtime_client (boto3.SageMakerRuntime.Client) – Client which makes
InvokeEndpoint
calls to Amazon SageMaker (default: None). Predictors created using thisSession
use this client. If not provided, one will be created using this instance’sboto_session
.sagemaker_featurestore_runtime_client (boto3.SageMakerFeatureStoreRuntime.Client) – Client which makes SageMaker FeatureStore record related calls to Amazon SageMaker (default: None). If not provided, one will be created using this instance’s
boto_session
.default_bucket (str) – The default Amazon S3 bucket to be used by this session. This will be created the next time an Amazon S3 bucket is needed (by calling
default_bucket()
). If not provided, it will be fetched from the sagemaker_config. If not configured there either, a default bucket will be created based on the following format: “sagemaker-{region}-{aws-account-id}”. Example: “sagemaker-my-custom-bucket”.settings (sagemaker.session_settings.SessionSettings) – Optional. Set of optional parameters to apply to the session.
sagemaker_metrics_client (boto3.SageMakerMetrics.Client) – Client which makes SageMaker Metrics related calls to Amazon SageMaker (default: None). If not provided, one will be created using this instance’s
boto_session
.sagemaker_config (dict) – A dictionary containing default values for the SageMaker Python SDK. (default: None). The dictionary must adhere to the schema defined at ~sagemaker.config.config_schema.SAGEMAKER_PYTHON_SDK_CONFIG_SCHEMA. If sagemaker_config is not provided and configuration files exist (at the default paths for admins and users, or paths set through the environment variables SAGEMAKER_ADMIN_CONFIG_OVERRIDE and SAGEMAKER_USER_CONFIG_OVERRIDE), a new dictionary will be generated from those configuration files. Alternatively, this dictionary can be generated by calling
load_sagemaker_config()
and then be provided to the Session.default_bucket_prefix (str) – The default prefix to use for S3 Object Keys. (default: None). If provided and where applicable, it will be used by the SDK to construct default S3 URIs, in the format: s3://{default_bucket}/{default_bucket_prefix}/<rest of object key> This parameter can also be specified via {sagemaker_config} instead of here. If not provided here or within {sagemaker_config}, default S3 URIs will have the format: s3://{default_bucket}/<rest of object key>
- property boto_region_name¶
Placeholder docstring
- upload_data(path, bucket=None, key_prefix='data', callback=None, extra_args=None)¶
Upload local file or directory to S3.
If a single file is specified for upload, the resulting S3 object key is
{key_prefix}/{filename}
(filename does not include the local path, if any specified). If a directory is specified for upload, the API uploads all content, recursively, preserving relative structure of subdirectories. The resulting object key names are:{key_prefix}/{relative_subdirectory_path}/filename
.- Parameters
path (str) – Path (absolute or relative) of local file or directory to upload.
bucket (str) – Name of the S3 Bucket to upload to (default: None). If not specified, the default bucket of the
Session
is used (if default bucket does not exist, theSession
creates it).key_prefix (str) – Optional S3 object key name prefix (default: ‘data’). S3 uses the prefix to create a directory structure for the bucket content that it display in the S3 console.
extra_args (dict) – Optional extra arguments that may be passed to the upload operation. Similar to ExtraArgs parameter in S3 upload_file function. Please refer to the ExtraArgs parameter documentation here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html#the-extraargs-parameter
- Returns
- The S3 URI of the uploaded file(s). If a file is specified in the path argument,
the URI format is:
s3://{bucket name}/{key_prefix}/{original_file_name}
. If a directory is specified in the path argument, the URI format iss3://{bucket name}/{key_prefix}
.
- Return type
- upload_string_as_file_body(body, bucket, key, kms_key=None)¶
Upload a string as a file body.
- Parameters
body (str) – String representing the body of the file.
bucket (str) – Name of the S3 Bucket to upload to (default: None). If not specified, the default bucket of the
Session
is used (if default bucket does not exist, theSession
creates it).key (str) – S3 object key. This is the s3 path to the file.
kms_key (str) – The KMS key to use for encrypting the file.
- Returns
- The S3 URI of the uploaded file.
The URI format is:
s3://{bucket name}/{key}
.
- Return type
- download_data(path, bucket, key_prefix='', extra_args=None)¶
Download file or directory from S3.
- Parameters
path (str) – Local path where the file or directory should be downloaded to.
bucket (str) – Name of the S3 Bucket to download from.
key_prefix (str) – Optional S3 object key name prefix.
extra_args (dict) – Optional extra arguments that may be passed to the download operation. Please refer to the ExtraArgs parameter in the boto3 documentation here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-example-download-file.html
- Returns
List of local paths of downloaded files
- Return type
- read_s3_file(bucket, key_prefix)¶
Read a single file from S3.
- list_s3_files(bucket, key_prefix)¶
Lists the S3 files given an S3 bucket and key.
- default_bucket()¶
Return the name of the default bucket to use in relevant Amazon SageMaker interactions.
This function will create the s3 bucket if it does not exist.
- Returns
- The name of the default bucket. If the name was not explicitly specified through
the Session or sagemaker_config, the bucket will take the form:
sagemaker-{region}-{AWS account ID}
.
- Return type
- expected_bucket_owner_id_bucket_check(bucket_name, s3, expected_bucket_owner_id)¶
Checks if the bucket belongs to a particular owner and throws a Client Error if it is not
- general_bucket_check_if_user_has_permission(bucket_name, s3, bucket, region, bucket_creation_date_none)¶
Checks if the person running has the permissions to the bucket
If there is any other error that comes up with calling head bucket, it is raised up here If there is no bucket , it will create one
- create_bucket_for_not_exist_error(bucket_name, region, s3)¶
Creates the S3 bucket in the given region
- train(input_mode, input_config, role=None, job_name=None, output_config=None, resource_config=None, vpc_config=None, hyperparameters=None, stop_condition=None, tags=None, metric_definitions=None, enable_network_isolation=None, image_uri=None, training_image_config=None, infra_check_config=None, container_entry_point=None, container_arguments=None, algorithm_arn=None, encrypt_inter_container_traffic=None, use_spot_instances=False, checkpoint_s3_uri=None, checkpoint_local_path=None, experiment_config=None, debugger_rule_configs=None, debugger_hook_config=None, tensorboard_output_config=None, enable_sagemaker_metrics=None, profiler_rule_configs=None, profiler_config=None, environment=None, retry_strategy=None, remote_debug_config=None, session_chaining_config=None)¶
Create an Amazon SageMaker training job.
- Parameters
input_mode (str) – The input mode that the algorithm supports. Valid modes: * ‘File’ - Amazon SageMaker copies the training dataset from the S3 location to a directory in the Docker container. * ‘Pipe’ - Amazon SageMaker streams data directly from S3 to the container via a Unix-named pipe. * ‘FastFile’ - Amazon SageMaker streams data from S3 on demand instead of downloading the entire dataset before training begins.
input_config (list) – A list of Channel objects. Each channel is a named input source. Please refer to the format details described: https://botocore.readthedocs.io/en/latest/reference/services/sagemaker.html#SageMaker.Client.create_training_job
role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. You must grant sufficient permissions to this role.
job_name (str) – Name of the training job being created.
output_config (dict) – The S3 URI where you want to store the training results and optional KMS key ID.
resource_config (dict) – Contains values for ResourceConfig: * instance_count (int): Number of EC2 instances to use for training. The key in resource_config is ‘InstanceCount’. * instance_type (str): Type of EC2 instance to use for training, for example, ‘ml.c4.xlarge’. The key in resource_config is ‘InstanceType’.
vpc_config (dict) – Contains values for VpcConfig: * subnets (list[str]): List of subnet ids. The key in vpc_config is ‘Subnets’. * security_group_ids (list[str]): List of security group ids. The key in vpc_config is ‘SecurityGroupIds’.
hyperparameters (dict) – Hyperparameters for model training. The hyperparameters are made accessible as a dict[str, str] to the training code on SageMaker. For convenience, this accepts other types for keys and values, but
str()
will be called to convert them before training.stop_condition (dict) – Defines when training shall finish. Contains entries that can be understood by the service like
MaxRuntimeInSeconds
.tags (Optional[Tags]) – Tags for labeling a training job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
metric_definitions (list[dict]) – A list of dictionaries that defines the metric(s) used to evaluate the training jobs. Each dictionary contains two keys: ‘Name’ for the name of the metric, and ‘Regex’ for the regular expression used to extract the metric from the logs.
enable_network_isolation (bool) – Whether to request for the training job to run with network isolation or not.
image_uri (str) – Docker image containing training code.
training_image_config (dict) –
Training image configuration. Optionally, the dict can contain ‘TrainingRepositoryAccessMode’ and ‘TrainingRepositoryCredentialsProviderArn’ (under ‘TrainingRepositoryAuthConfig’). For example,
training_image_config = { "TrainingRepositoryAccessMode": "Vpc", "TrainingRepositoryAuthConfig": { "TrainingRepositoryCredentialsProviderArn": "arn:aws:lambda:us-west-2:1234567890:function:test" }, }
If TrainingRepositoryAccessMode is set to Vpc, the training image is accessed through a private Docker registry in customer Vpc. If it’s set to Platform or None, the training image is accessed through ECR. If TrainingRepositoryCredentialsProviderArn is provided, the credentials to authenticate to the private Docker registry will be retrieved from this AWS Lambda function. (default:
None
). When it’s set to None, SageMaker will not do authentication before pulling the image in the private Docker registry.container_entry_point (List[str]) – Optional. The entrypoint script for a Docker container used to run a training job. This script takes precedence over the default train processing instructions.
container_arguments (List[str]) – Optional. The arguments for a container used to run a training job.
algorithm_arn (str) – Algorithm Arn from Marketplace.
encrypt_inter_container_traffic (bool) – Specifies whether traffic between training containers is encrypted for the training job (default:
False
).use_spot_instances (bool) – whether to use spot instances for training.
checkpoint_s3_uri (str) – The S3 URI in which to persist checkpoints that the algorithm persists (if any) during training. (default:
None
).checkpoint_local_path (str) – The local path that the algorithm writes its checkpoints to. SageMaker will persist all files under this path to checkpoint_s3_uri continually during training. On job startup the reverse happens - data from the s3 location is downloaded to this path before the algorithm is started. If the path is unset then SageMaker assumes the checkpoints will be provided under /opt/ml/checkpoints/. (default:
None
).experiment_config (dict[str, str]) – Experiment management configuration. Optionally, the dict can contain four keys: ‘ExperimentName’, ‘TrialName’, ‘TrialComponentDisplayName’ and ‘RunName’. The behavior of setting these keys is as follows: * If ExperimentName is supplied but TrialName is not a Trial will be automatically created and the job’s Trial Component associated with the Trial. * If TrialName is supplied and the Trial already exists the job’s Trial Component will be associated with the Trial. * If both ExperimentName and TrialName are not supplied the trial component will be unassociated. * TrialComponentDisplayName is used for display in Studio. * RunName is used to record an experiment run.
enable_sagemaker_metrics (bool) – enable SageMaker Metrics Time Series. For more information see: https://docs.aws.amazon.com/sagemaker/latest/dg/API_AlgorithmSpecification.html #SageMaker-Type -AlgorithmSpecification-EnableSageMakerMetricsTimeSeries (default:
None
).profiler_rule_configs (list[dict]) – A list of profiler rule configurations.src/sagemaker/lineage/artifact.py:285
profiler_config (dict) – Configuration for how profiling information is emitted with SageMaker Profiler. (default:
None
).remote_debug_config (dict) –
Configuration for RemoteDebug. (default:
None
) The dict can contain ‘EnableRemoteDebug’(bool). For example,remote_debug_config = { "EnableRemoteDebug": True, }
session_chaining_config (dict) –
Configuration for SessionChaining. (default:
None
) The dict can contain ‘EnableSessionTagChaining’(bool). For example,session_chaining_config = { "EnableSessionTagChaining": True, }
environment (dict[str, str]) – Environment variables to be set for use during training job (default:
None
)retry_strategy (dict) – Defines RetryStrategy for InternalServerFailures. * max_retry_attsmpts (int): Number of times a job should be retried. The key in RetryStrategy is ‘MaxRetryAttempts’.
infra_check_config (dict) –
Infra check configuration. Optionally, the dict can contain ‘EnableInfraCheck’(bool). For example,
infra_check_config = { "EnableInfraCheck": True, }
- Returns
ARN of the training job, if it is created.
- Return type
- Raises
- botocore.exceptions.ClientError – If Sagemaker throws an exception while creating
training job. –
- ValueError – If both image_uri and algorithm are provided, or if neither is provided.
- update_training_job(job_name, profiler_rule_configs=None, profiler_config=None, resource_config=None, remote_debug_config=None)¶
Calls the UpdateTrainingJob API for the given job name and returns the response.
- Parameters
job_name (str) – Name of the training job being updated.
profiler_rule_configs (list) – List of profiler rule configurations. (default:
None
).profiler_config (dict) – Configuration for how profiling information is emitted with SageMaker Profiler. (default:
None
).resource_config (dict) – Configuration of the resources for the training job. You can update the keep-alive period if the warm pool status is Available. No other fields can be updated. (default:
None
).remote_debug_config (dict) –
Configuration for RemoteDebug. (default:
None
) The dict can contain ‘EnableRemoteDebug’(bool). For example,remote_debug_config = { "EnableRemoteDebug": True, }
- Returns
ARN of training job
- Return type
- Raises
- botocore.exceptions.ClientError – If Sagemaker throws an error while updating training
job. –
- botocore.exceptions.ParamValidationError – If any request parameters are in an invalid
format. –
- process(inputs, output_config, job_name, resources, stopping_condition, app_specification, environment=None, network_config=None, role_arn=None, tags=None, experiment_config=None)¶
Create an Amazon SageMaker processing job.
- Parameters
inputs ([dict]) – List of up to 10 ProcessingInput dictionaries.
output_config (dict) – A config dictionary, which contains a list of up to 10 ProcessingOutput dictionaries, as well as an optional KMS key ID.
job_name (str) – The name of the processing job. The name must be unique within an AWS Region in an AWS account. Names should have minimum length of 1 and maximum length of 63 characters.
resources (dict) – Encapsulates the resources, including ML instances and storage, to use for the processing job.
stopping_condition (dict[str,int]) – Specifies a limit to how long the processing job can run, in seconds.
app_specification (dict[str,str]) – Configures the processing job to run the given image. Details are in the processing container specification.
environment (dict) – Environment variables to start the processing container with.
network_config (dict) – Specifies networking options, such as network traffic encryption between processing containers, whether to allow inbound and outbound network calls to and from processing containers, and VPC subnets and security groups to use for VPC-enabled processing jobs.
role_arn (str) – The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
tags (Optional[Tags]) – A list of dictionaries containing key-value pairs.
experiment_config (dict[str, str]) – Experiment management configuration. Optionally, the dict can contain three keys: ‘ExperimentName’, ‘TrialName’, and ‘TrialComponentDisplayName’. The behavior of setting these keys is as follows: * If ExperimentName is supplied but TrialName is not a Trial will be automatically created and the job’s Trial Component associated with the Trial. * If TrialName is supplied and the Trial already exists the job’s Trial Component will be associated with the Trial. * If both ExperimentName and TrialName are not supplied the trial component will be unassociated. * TrialComponentDisplayName is used for display in Studio.
- Raises
- botocore.exceptions.ClientError – If Sagemaker throws an error while creating
processing job. –
- create_monitoring_schedule(monitoring_schedule_name, schedule_expression, statistics_s3_uri, constraints_s3_uri, monitoring_inputs, monitoring_output_config, instance_count, instance_type, volume_size_in_gb, volume_kms_key=None, image_uri=None, entrypoint=None, arguments=None, record_preprocessor_source_uri=None, post_analytics_processor_source_uri=None, max_runtime_in_seconds=None, environment=None, network_config=None, role_arn=None, tags=None, data_analysis_start_time=None, data_analysis_end_time=None)¶
Create an Amazon SageMaker monitoring schedule.
- Parameters
monitoring_schedule_name (str) – The name of the monitoring schedule. The name must be unique within an AWS Region in an AWS account. Names should have a minimum length of 1 and a maximum length of 63 characters.
schedule_expression (str) – The cron expression that dictates the monitoring execution schedule.
statistics_s3_uri (str) – The S3 uri of the statistics file to use.
constraints_s3_uri (str) – The S3 uri of the constraints file to use.
monitoring_inputs ([dict]) – List of MonitoringInput dictionaries.
monitoring_output_config (dict) – A config dictionary, which contains a list of MonitoringOutput dictionaries, as well as an optional KMS key ID.
instance_count (int) – The number of instances to run.
instance_type (str) – The type of instance to run.
volume_size_in_gb (int) – Size of the volume in GB.
volume_kms_key (str) – KMS key to use when encrypting the volume.
image_uri (str) – The image uri to use for monitoring executions.
entrypoint (str) – The entrypoint to the monitoring execution image.
arguments (str) – The arguments to pass to the monitoring execution image.
record_preprocessor_source_uri (str or None) – The S3 uri that points to the script that pre-processes the dataset (only applicable to first-party images).
post_analytics_processor_source_uri (str or None) – The S3 uri that points to the script that post-processes the dataset (only applicable to first-party images).
max_runtime_in_seconds (int) – Specifies a limit to how long the processing job can run, in seconds.
environment (dict) – Environment variables to start the monitoring execution container with.
network_config (dict) – Specifies networking options, such as network traffic encryption between processing containers, whether to allow inbound and outbound network calls to and from processing containers, and VPC subnets and security groups to use for VPC-enabled processing jobs.
role_arn (str) – The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
tags (Optional[Tags]) – A list of dictionaries containing key-value pairs.
data_analysis_start_time (str) – Start time for the data analysis window for the one time monitoring schedule (NOW), e.g. “-PT1H”
data_analysis_end_time (str) – End time for the data analysis window for the one time monitoring schedule (NOW), e.g. “-PT1H”
- update_monitoring_schedule(monitoring_schedule_name, schedule_expression=None, statistics_s3_uri=None, constraints_s3_uri=None, monitoring_inputs=None, monitoring_output_config=None, instance_count=None, instance_type=None, volume_size_in_gb=None, volume_kms_key=None, image_uri=None, entrypoint=None, arguments=None, record_preprocessor_source_uri=None, post_analytics_processor_source_uri=None, max_runtime_in_seconds=None, environment=None, network_config=None, role_arn=None, data_analysis_start_time=None, data_analysis_end_time=None)¶
Update an Amazon SageMaker monitoring schedule.
- Parameters
monitoring_schedule_name (str) – The name of the monitoring schedule. The name must be unique within an AWS Region in an AWS account. Names should have a minimum length of 1 and a maximum length of 63 characters.
schedule_expression (str) – The cron expression that dictates the monitoring execution schedule.
statistics_s3_uri (str) – The S3 uri of the statistics file to use.
constraints_s3_uri (str) – The S3 uri of the constraints file to use.
monitoring_inputs ([dict]) – List of MonitoringInput dictionaries.
monitoring_output_config (dict) – A config dictionary, which contains a list of MonitoringOutput dictionaries, as well as an optional KMS key ID.
instance_count (int) – The number of instances to run.
instance_type (str) – The type of instance to run.
volume_size_in_gb (int) – Size of the volume in GB.
volume_kms_key (str) – KMS key to use when encrypting the volume.
image_uri (str) – The image uri to use for monitoring executions.
entrypoint (str) – The entrypoint to the monitoring execution image.
arguments (str) – The arguments to pass to the monitoring execution image.
record_preprocessor_source_uri (str or None) – The S3 uri that points to the script that pre-processes the dataset (only applicable to first-party images).
post_analytics_processor_source_uri (str or None) – The S3 uri that points to the script that post-processes the dataset (only applicable to first-party images).
max_runtime_in_seconds (int) – Specifies a limit to how long the processing job can run, in seconds.
environment (dict) – Environment variables to start the monitoring execution container with.
network_config (dict) – Specifies networking options, such as network traffic encryption between processing containers, whether to allow inbound and outbound network calls to and from processing containers, and VPC subnets and security groups to use for VPC-enabled processing jobs.
role_arn (str) – The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
tags ([dict[str,str]]) – A list of dictionaries containing key-value pairs.
data_analysis_start_time (str) – Start time for the data analysis window for the one time monitoring schedule (NOW), e.g. “-PT1H”
data_analysis_end_time (str) – End time for the data analysis window for the one time monitoring schedule (NOW), e.g. “-PT1H”
- start_monitoring_schedule(monitoring_schedule_name)¶
Starts a monitoring schedule.
- Parameters
monitoring_schedule_name (str) – The name of the Amazon SageMaker Monitoring Schedule to start.
- stop_monitoring_schedule(monitoring_schedule_name)¶
Stops a monitoring schedule.
- Parameters
monitoring_schedule_name (str) – The name of the Amazon SageMaker Monitoring Schedule to stop.
- delete_monitoring_schedule(monitoring_schedule_name)¶
Deletes a monitoring schedule.
- Parameters
monitoring_schedule_name (str) – The name of the Amazon SageMaker Monitoring Schedule to delete.
- describe_monitoring_schedule(monitoring_schedule_name)¶
Calls the DescribeMonitoringSchedule API for given name and returns the response.
- list_monitoring_executions(monitoring_schedule_name, sort_by='ScheduledTime', sort_order='Descending', max_results=100)¶
Lists the monitoring executions associated with the given monitoring_schedule_name.
- Parameters
monitoring_schedule_name (str) – The monitoring_schedule_name for which to retrieve the monitoring executions.
sort_by (str) – The field to sort by. Can be one of: “CreationTime”, “ScheduledTime”, “Status”. Default: “ScheduledTime”.
sort_order (str) – The sort order. Can be one of: “Ascending”, “Descending”. Default: “Descending”.
max_results (int) – The maximum number of results to return. Must be between 1 and 100.
- Returns
Dictionary of monitoring schedule executions.
- Return type
- list_monitoring_schedules(endpoint_name=None, sort_by='CreationTime', sort_order='Descending', max_results=100)¶
Lists the monitoring executions associated with the given monitoring_schedule_name.
- Parameters
endpoint_name (str) – The name of the endpoint to filter on. If not provided, does not filter on it. Default: None.
sort_by (str) – The field to sort by. Can be one of: “Name”, “CreationTime”, “Status”. Default: “CreationTime”.
sort_order (str) – The sort order. Can be one of: “Ascending”, “Descending”. Default: “Descending”.
max_results (int) – The maximum number of results to return. Must be between 1 and 100.
- Returns
Dictionary of monitoring schedule executions.
- Return type
- update_monitoring_alert(monitoring_schedule_name, monitoring_alert_name, data_points_to_alert, evaluation_period)¶
Update the monitoring alerts associated with the given schedule_name and alert_name
- Parameters
- Returns
A dict represents the update alert response.
- Return type
- list_monitoring_alerts(monitoring_schedule_name, next_token=None, max_results=10)¶
Lists the monitoring alerts associated with the given monitoring_schedule_name.
- Parameters
- Returns
list of monitoring alerts.
- Return type
- list_monitoring_alert_history(monitoring_schedule_name=None, monitoring_alert_name=None, sort_by='CreationTime', sort_order='Descending', next_token=None, max_results=10, creation_time_before=None, creation_time_after=None, status_equals=None)¶
Lists the alert history associated with the given schedule_name and alert_name.
- Parameters
monitoring_schedule_name (Optional[str]) – The name of the monitoring_schedule_name to filter on. If not provided, does not filter on it. Default: None.
monitoring_alert_name (Optional[str]) – The name of the monitoring_alert_name to filter on. If not provided, does not filter on it. Default: None.
sort_by (Optional[str]) – sort_by (str): The field to sort by. Can be one of: “Name”, “CreationTime” Default: “CreationTime”.
sort_order (Optional[str]) – The sort order. Can be one of: “Ascending”, “Descending”. Default: “Descending”.
next_token (Optional[str]) – The pagination token. Default: None
max_results (Optional[int]) – The maximum number of results to return. Must be between 1 and 100. Default: 10.
creation_time_before (Optional[str]) – A filter to filter alert history before a time
creation_time_after (Optional[str]) – A filter to filter alert history after a time Default: None.
status_equals (Optional[str]) – A filter to filter alert history by status Default: None.
- Returns
list of monitoring alert history.
- Return type
- was_processing_job_successful(job_name)¶
Calls the DescribeProcessingJob API for the given job name.
It returns True if job was successful.
- describe_processing_job(job_name)¶
Calls the DescribeProcessingJob API for the given job name and returns the response.
- stop_processing_job(job_name)¶
Calls the StopProcessingJob API for the given job name.
- Parameters
job_name (str) – The name of the processing job to stop.
- stop_training_job(job_name)¶
Calls the StopTrainingJob API for the given job name.
- Parameters
job_name (str) – The name of the training job to stop.
- describe_training_job(job_name)¶
Calls the DescribeTrainingJob API for the given job name and returns the response.
- describe_training_plan(training_plan_name)¶
Calls the DescribeTrainingPlan API for the given training plan and returns the response.
- list_training_plans(filters=None, requested_start_time_after=None, requested_start_time_before=None, start_time_after=None, start_time_before=None, sort_order=None, sort_by=None, max_results=None, next_token=None)¶
Calls the ListrTrainingPlan API for the given filters and returns the response.
- Parameters
filters (dict) – A dictionary of key-value pairs used to filter the training plans. Default to None.
requested_start_time_after (datetime) – A timestamp that filters the results to only include training plans with a requested start time after this timestamp.
requested_start_time_before (datetime) – A timestamp that filters the results to only include training plans with a requested start time before this timestamp.
start_time_after (datetime) – A timestamp that filters the results to only include training plans with an actual start time after this timestamp.
start_time_before (datetime) – A timestamp that filters the results to only include training plans with an actual start time before this timestamp.
sort_order (str) – The order that the training plans will be listed in result. Default to None.
sort_by (str) – The value that the training plans will be sorted by. Default to None.
max_results (int) – The number of candidates will be listed in results, between 1 and 100. Default to None. If None, will return all the training_plans.
next_token (str) – The pagination token. Default to None.
- Returns
- A dictionary containing the following keys:
”TrainingPlanSummaries”: A list of dictionaries, where each dictionary represents a training plan.
”NextToken”: A token to retrieve the next set of results, if there are more than the maximum number of results returned.
- Return type
- auto_ml(input_config, output_config, auto_ml_job_config, role=None, job_name=None, problem_type=None, job_objective=None, generate_candidate_definitions_only=False, tags=None, model_deploy_config=None)¶
Create an Amazon SageMaker AutoML job.
- Parameters
input_config (list[dict]) – A list of Channel objects. Each channel contains “DataSource” and “TargetAttributeName”, “CompressionType” and “SampleWeightAttributeName” are optional fields.
output_config (dict) – The S3 URI where you want to store the training results and optional KMS key ID.
auto_ml_job_config (dict) – A dict of AutoMLJob config, containing “StoppingCondition”, “SecurityConfig”, optionally contains “VolumeKmsKeyId”.
role (str) – The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
job_name (str) – A string that can be used to identify an AutoMLJob. Each AutoMLJob should have a unique job name.
problem_type (str) – The type of problem of this AutoMLJob. Valid values are “Regression”, “BinaryClassification”, “MultiClassClassification”. If None, SageMaker AutoMLJob will infer the problem type automatically.
job_objective (dict) – AutoMLJob objective, contains “AutoMLJobObjectiveType” (optional), “MetricName” and “Value”.
generate_candidate_definitions_only (bool) – Indicates whether to only generate candidate definitions. If True, AutoML.list_candidates() cannot be called. Default: False.
tags (Optional[Tags]) – A list of dictionaries containing key-value pairs.
model_deploy_config (dict) – Specifies how to generate the endpoint name for an automatic one-click Autopilot model deployment. Contains “AutoGenerateEndpointName” and “EndpointName”
- describe_auto_ml_job(job_name)¶
Calls the DescribeAutoMLJob API for the given job name and returns the response.
- list_candidates(job_name, status_equals=None, candidate_name=None, candidate_arn=None, sort_order=None, sort_by=None, max_results=None)¶
Returns the list of candidates of an AutoML job for a given name.
- Parameters
job_name (str) – The name of the AutoML job. If None, will use object’s latest_auto_ml_job name.
status_equals (str) – Filter the result with candidate status, values could be “Completed”, “InProgress”, “Failed”, “Stopped”, “Stopping”
candidate_name (str) – The name of a specified candidate to list. Default to None.
candidate_arn (str) – The Arn of a specified candidate to list. Default to None.
sort_order (str) – The order that the candidates will be listed in result. Default to None.
sort_by (str) – The value that the candidates will be sorted by. Default to None.
max_results (int) – The number of candidates will be listed in results, between 1 to 100. Default to None. If None, will return all the candidates.
- Returns
A list of dictionaries with candidates information
- Return type
- wait_for_auto_ml_job(job, poll=5)¶
Wait for an Amazon SageMaker AutoML job to complete.
- Parameters
- Returns
Return value from the
DescribeAutoMLJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the auto ml job fails with CapacityError.
exceptions.UnexpectedStatusException – If the auto ml job fails.
- wait_for_optimization_job(job, poll=5)¶
Wait for an Amazon SageMaker Optimization job to complete.
- Parameters
- Returns
Return value from the
DescribeOptimizationJob
API.- Return type
(dict)
- Raises
exceptions.ResourceNotFound – If optimization job fails with CapacityError.
exceptions.UnexpectedStatusException – If optimization job fails.
- logs_for_auto_ml_job(job_name, wait=False, poll=10)¶
Display logs for a given AutoML job, optionally tailing them until job is complete.
If the output is a tty or a Jupyter cell, it will be color-coded based on which instance the log entry is from.
- Parameters
- Raises
exceptions.CapacityError – If waiting and auto ml job fails with CapacityError.
exceptions.UnexpectedStatusException – If waiting and auto ml job fails.
- create_auto_ml_v2(input_config, job_name, problem_config, output_config, job_objective=None, model_deploy_config=None, data_split_config=None, role=None, security_config=None, tags=None)¶
Create an Amazon SageMaker AutoMLV2 job.
- Parameters
input_config (list[dict]) – A list of AutoMLDataChannel objects. Each channel contains “DataSource” and other optional fields.
job_name (str) – A string that can be used to identify an AutoMLJob. Each AutoMLJob should have a unique job name.
problem_config (object) –
A collection of settings specific to the problem type used to configure an AutoML job V2. There must be one and only one config of the following type. Supported problem types are:
Image Classification (sagemaker.automl.automlv2.ImageClassificationJobConfig),
Tabular (sagemaker.automl.automlv2.TabularJobConfig),
Text Classification (sagemaker.automl.automlv2.TextClassificationJobConfig),
Text Generation (TextGenerationJobConfig),
- Time Series Forecasting (
sagemaker.automl.automlv2.TimeSeriesForecastingJobConfig).
output_config (dict) – The S3 URI where you want to store the training results and optional KMS key ID.
job_objective (dict) – AutoMLJob objective, contains “AutoMLJobObjectiveType” (optional), “MetricName” and “Value”.
model_deploy_config (dict) – Specifies how to generate the endpoint name for an automatic one-click Autopilot model deployment. Contains “AutoGenerateEndpointName” and “EndpointName”
data_split_config (dict) – This structure specifies how to split the data into train and validation datasets.
role (str) – The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
security_config (dict) – The security configuration for traffic encryption or Amazon VPC settings.
tags (Optional[Tags]) – A list of dictionaries containing key-value pairs.
- describe_auto_ml_job_v2(job_name)¶
Calls the DescribeAutoMLJobV2 API for the given job name and returns the response.
- compile_model(input_model_config, output_model_config, role=None, job_name=None, stop_condition=None, tags=None)¶
Create an Amazon SageMaker Neo compilation job.
- Parameters
input_model_config (dict) – the trained model and the Amazon S3 location where it is stored.
output_model_config (dict) – Identifies the Amazon S3 location where you want Amazon SageMaker Neo to save the results of compilation job
role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker Neo compilation jobs use this role to access model artifacts. You must grant sufficient permissions to this role.
job_name (str) – Name of the compilation job being created.
stop_condition (dict) – Defines when compilation job shall finish. Contains entries that can be understood by the service like
MaxRuntimeInSeconds
.tags (Optional[Tags]) – List of tags for labeling a compile model job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
- Returns
ARN of the compile model job, if it is created.
- Return type
- package_model_for_edge(output_model_config, role=None, job_name=None, compilation_job_name=None, model_name=None, model_version=None, resource_key=None, tags=None)¶
Create an Amazon SageMaker Edge packaging job.
- Parameters
output_model_config (dict) – Identifies the Amazon S3 location where you want Amazon SageMaker Edge to save the results of edge packaging job
role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker Edge edge packaging jobs use this role to access model artifacts. You must grant sufficient permissions to this role.
job_name (str) – Name of the edge packaging job being created.
compilation_job_name (str) – Name of the compilation job being created.
resource_key (str) – KMS key to encrypt the disk used to package the job
tags (Optional[Tags]) – List of tags for labeling a compile model job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
- tune(job_name, strategy, objective_type, objective_metric_name, max_jobs, max_parallel_jobs, parameter_ranges, static_hyperparameters, input_mode, metric_definitions, role, input_config, output_config, resource_config, stop_condition, tags, warm_start_config, max_runtime_in_seconds=None, strategy_config=None, completion_criteria_config=None, enable_network_isolation=False, image_uri=None, algorithm_arn=None, early_stopping_type='Off', encrypt_inter_container_traffic=False, vpc_config=None, use_spot_instances=False, checkpoint_s3_uri=None, checkpoint_local_path=None, random_seed=None, environment=None, hpo_resource_config=None, autotune=False, auto_parameters=None)¶
Create an Amazon SageMaker hyperparameter tuning job.
- Parameters
job_name (str) – Name of the tuning job being created.
strategy (str) – Strategy to be used for hyperparameter estimations.
strategy_config (dict) – A configuration for the hyperparameter tuning job optimisation strategy.
objective_type (str) – The type of the objective metric for evaluating training jobs. This value can be either ‘Minimize’ or ‘Maximize’.
objective_metric_name (str) – Name of the metric for evaluating training jobs.
max_jobs (int) – Maximum total number of training jobs to start for the hyperparameter tuning job.
max_parallel_jobs (int) – Maximum number of parallel training jobs to start.
parameter_ranges (dict) – Dictionary of parameter ranges. These parameter ranges can be one of three types: Continuous, Integer, or Categorical.
static_hyperparameters (dict) – Hyperparameters for model training. These hyperparameters remain unchanged across all of the training jobs for the hyperparameter tuning job. The hyperparameters are made accessible as a dictionary for the training code on SageMaker.
image_uri (str) – Docker image URI containing training code.
algorithm_arn (str) – Resource ARN for training algorithm created on or subscribed from AWS Marketplace (default: None).
input_mode (str) – The input mode that the algorithm supports. Valid modes: * ‘File’ - Amazon SageMaker copies the training dataset from the S3 location to a directory in the Docker container. * ‘Pipe’ - Amazon SageMaker streams data directly from S3 to the container via a Unix-named pipe. * ‘FastFile’ - Amazon SageMaker streams data from S3 on demand instead of downloading the entire dataset before training begins.
metric_definitions (list[dict]) – A list of dictionaries that defines the metric(s) used to evaluate the training jobs. Each dictionary contains two keys: ‘Name’ for the name of the metric, and ‘Regex’ for the regular expression used to extract the metric from the logs. This should be defined only for jobs that don’t use an Amazon algorithm.
role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. You must grant sufficient permissions to this role.
input_config (list) – A list of Channel objects. Each channel is a named input source. Please refer to the format details described: https://botocore.readthedocs.io/en/latest/reference/services/sagemaker.html#SageMaker.Client.create_training_job
output_config (dict) – The S3 URI where you want to store the training results and optional KMS key ID.
resource_config (dict) – Contains values for ResourceConfig: * instance_count (int): Number of EC2 instances to use for training. The key in resource_config is ‘InstanceCount’. * instance_type (str): Type of EC2 instance to use for training, for example, ‘ml.c4.xlarge’. The key in resource_config is ‘InstanceType’.
stop_condition (dict) – When training should finish, e.g.
MaxRuntimeInSeconds
.tags (list[dict]) – List of tags for labeling the tuning job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
warm_start_config (dict) – Configuration defining the type of warm start and other required configurations.
max_runtime_in_seconds (int or PipelineVariable) – The maximum time in seconds that a training job launched by a hyperparameter tuning job can run.
completion_criteria_config (sagemaker.tuner.TuningJobCompletionCriteriaConfig) – A configuration for the completion criteria.
early_stopping_type (str) – Specifies whether early stopping is enabled for the job. Can be either ‘Auto’ or ‘Off’. If set to ‘Off’, early stopping will not be attempted. If set to ‘Auto’, early stopping of some training jobs may happen, but is not guaranteed to.
enable_network_isolation (bool) – Specifies whether to isolate the training container (default:
False
).encrypt_inter_container_traffic (bool) – Specifies whether traffic between training containers is encrypted for the training jobs started for this hyperparameter tuning job (default:
False
).vpc_config (dict) – Contains values for VpcConfig (default: None): * subnets (list[str]): List of subnet ids. The key in vpc_config is ‘Subnets’. * security_group_ids (list[str]): List of security group ids. The key in vpc_config is ‘SecurityGroupIds’.
use_spot_instances (bool) – whether to use spot instances for training.
checkpoint_s3_uri (str) – The S3 URI in which to persist checkpoints that the algorithm persists (if any) during training. (default:
None
).checkpoint_local_path (str) – The local path that the algorithm writes its checkpoints to. SageMaker will persist all files under this path to checkpoint_s3_uri continually during training. On job startup the reverse happens - data from the s3 location is downloaded to this path before the algorithm is started. If the path is unset then SageMaker assumes the checkpoints will be provided under /opt/ml/checkpoints/. (default:
None
).random_seed (int) – An initial value used to initialize a pseudo-random number generator. Setting a random seed will make the hyperparameter tuning search strategies to produce more consistent configurations for the same tuning job. (default:
None
).environment (dict[str, str]) – Environment variables to be set for use during training jobs (default:
None
)hpo_resource_config (dict) – The configuration for the hyperparameter tuning resources, including the compute instances and storage volumes, used for training jobs launched by the tuning job, where you must specify either instance_configs or instance_count + instance_type + volume_size: * instance_count (int): Number of EC2 instances to use for training. The key in resource_config is ‘InstanceCount’. * instance_type (str): Type of EC2 instance to use for training, for example, ‘ml.c4.xlarge’. The key in resource_config is ‘InstanceType’. * volume_size (int or PipelineVariable): The volume size in GB of the data to be processed for hyperparameter optimisation * instance_configs (List[InstanceConfig]): A list containing the configuration(s) for one or more resources for processing hyperparameter jobs. These resources include compute instances and storage volumes to use in model training jobs. * volume_kms_key_id: The AWS Key Management Service (AWS KMS) key that Amazon SageMaker uses to encrypt data on the storage volume attached to the ML compute instance(s) that run the training job.
autotune (bool) – Whether the parameter ranges or other unset settings of a tuning job should be chosen automatically (default: False).
auto_parameters (dict[str, str]) – Dictionary of auto parameters. The keys are names of auto parameters and values are example values of auto parameters (default:
None
).
- create_tuning_job(job_name, tuning_config, training_config=None, training_config_list=None, warm_start_config=None, tags=None, autotune=False)¶
Create an Amazon SageMaker hyperparameter tuning job.
This method supports creating tuning jobs with single or multiple training algorithms (estimators), while the
tune()
method above only supports creating tuning jobs with single training algorithm.- Parameters
job_name (str) – Name of the tuning job being created.
tuning_config (dict) – Configuration to launch the tuning job.
training_config (dict) – Configuration to launch training jobs under the tuning job using a single algorithm.
training_config_list (list[dict]) – A list of configurations to launch training jobs under the tuning job using one or multiple algorithms. Either training_config or training_config_list should be provided, but not both.
warm_start_config (dict) – Configuration defining the type of warm start and other required configurations.
tags (Optional[Tags]) – List of tags for labeling the tuning job. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
autotune (bool) – Whether the parameter ranges or other unset settings of a tuning job should be chosen automatically.
- describe_tuning_job(job_name)¶
Calls DescribeHyperParameterTuningJob API for the given job name, returns the response.
- stop_tuning_job(name)¶
Stop the Amazon SageMaker hyperparameter tuning job with the specified name.
- Parameters
name (str) – Name of the Amazon SageMaker hyperparameter tuning job.
- Raises
ClientError – If an error occurs while trying to stop the hyperparameter tuning job.
- transform(job_name, model_name, strategy, max_concurrent_transforms, max_payload, input_config, output_config, resource_config, experiment_config, env=None, tags=None, data_processing=None, model_client_config=None, batch_data_capture_config=None)¶
Create an Amazon SageMaker transform job.
- Parameters
job_name (str) – Name of the transform job being created.
model_name (str) – Name of the SageMaker model being used for the transform job.
strategy (str) – The strategy used to decide how to batch records in a single request. Possible values are ‘MultiRecord’ and ‘SingleRecord’.
max_concurrent_transforms (int) – The maximum number of HTTP requests to be made to each individual transform container at one time.
max_payload (int) – Maximum size of the payload in a single HTTP request to the container in MB.
env (dict) – Environment variables to be set for use during the transform job.
input_config (dict) – A dictionary describing the input data (and its location) for the job.
output_config (dict) – A dictionary describing the output location for the job.
resource_config (dict) – A dictionary describing the resources to complete the job.
experiment_config (dict[str, str]) – Experiment management configuration. Optionally, the dict can contain three keys: ‘ExperimentName’, ‘TrialName’, and ‘TrialComponentDisplayName’. The behavior of setting these keys is as follows: * If ExperimentName is supplied but TrialName is not a Trial will be automatically created and the job’s Trial Component associated with the Trial. * If TrialName is supplied and the Trial already exists the job’s Trial Component will be associated with the Trial. * If both ExperimentName and TrialName are not supplied the trial component will be unassociated. * TrialComponentDisplayName is used for display in Studio.
tags (Optional[Tags]) – List of tags for labeling a transform job.
data_processing (dict) – A dictionary describing config for combining the input data and transformed data. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
model_client_config (dict) – A dictionary describing the model configuration for the job. Dictionary contains two optional keys, ‘InvocationsTimeoutInSeconds’, and ‘InvocationsMaxRetries’.
batch_data_capture_config (BatchDataCaptureConfig) – Configuration object which specifies the configurations related to the batch data capture for the transform job
- create_model(name, role=None, container_defs=None, vpc_config=None, enable_network_isolation=None, primary_container=None, tags=None)¶
Create an Amazon SageMaker
Model
.Specify the S3 location of the model artifacts and Docker image containing the inference code. Amazon SageMaker uses this information to deploy the model in Amazon SageMaker. This method can also be used to create a Model for an Inference Pipeline if you pass the list of container definitions through the containers parameter.
- Parameters
name (str) – Name of the Amazon SageMaker
Model
to create.role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. You must grant sufficient permissions to this role.
container_defs (list[dict[str, str]] or [dict[str, str]]) – A single container definition or a list of container definitions which will be invoked sequentially while performing the prediction. If the list contains only one container, then it’ll be passed to SageMaker Hosting as the
PrimaryContainer
and otherwise, it’ll be passed asContainers
.You can also specify the return value ofsagemaker.get_container_def()
orsagemaker.pipeline_container_def()
, which will used to create more advanced container configurations, including model containers which need artifacts from S3.vpc_config (dict[str, list[str]]) – The VpcConfig set on the model (default: None) * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
enable_network_isolation (bool) – Whether the model requires network isolation or not.
primary_container (str or dict[str, str]) – Docker image which defines the inference code. You can also specify the return value of
sagemaker.container_def()
, which is used to create more advanced container configurations, including model containers which need artifacts from S3. This field is deprecated, please use container_defs instead.tags (Optional[Tags]) – Optional. The list of tags to add to the model.
Example
>>> tags = [{'Key': 'tagname', 'Value': 'tagvalue'}] For more information about tags, see https://boto3.amazonaws.com/v1/documentation /api/latest/reference/services/sagemaker.html#SageMaker.Client.add_tags
- Returns
Name of the Amazon SageMaker
Model
created.- Return type
- create_model_from_job(training_job_name, name=None, role=None, image_uri=None, model_data_url=None, env=None, enable_network_isolation=None, vpc_config_override='VPC_CONFIG_DEFAULT', tags=None)¶
Create an Amazon SageMaker
Model
from a SageMaker Training Job.- Parameters
training_job_name (str) – The Amazon SageMaker Training Job name.
name (str) – The name of the SageMaker
Model
to create (default: None). If not specified, the training job name is used.role (str) – The
ExecutionRoleArn
IAM Role ARN for theModel
, specified either by an IAM role name or role ARN. If None, theRoleArn
from the SageMaker Training Job will be used.image_uri (str) – The Docker image URI (default: None). If None, it defaults to the training image URI from
training_job_name
.model_data_url (str) – S3 location of the model data (default: None). If None, defaults to the
ModelS3Artifacts
oftraining_job_name
.env (dict[string,string]) – Model environment variables (default: {}).
enable_network_isolation (bool) – Whether the model requires network isolation or not.
vpc_config_override (dict[str, list[str]]) – Optional override for VpcConfig set on the model. Default: use VpcConfig from training job. * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
tags (Optional[Tags]) – Optional. The list of tags to add to the model. For more, see https://docs.aws.amazon.com/sagemaker/latest/dg/API_Tag.html.
- Returns
The name of the created
Model
.- Return type
- create_model_package_from_algorithm(name, description, algorithm_arn, model_data)¶
Create a SageMaker Model Package from the results of training with an Algorithm Package.
- create_model_package_from_containers(containers=None, content_types=None, response_types=None, inference_instances=None, transform_instances=None, model_package_name=None, model_package_group_name=None, model_metrics=None, metadata_properties=None, marketplace_cert=False, approval_status='PendingManualApproval', description=None, drift_check_baselines=None, customer_metadata_properties=None, validation_specification=None, domain=None, sample_payload_url=None, task=None, skip_model_validation='None', source_uri=None, model_card=None, model_life_cycle=None)¶
Get request dictionary for CreateModelPackage API.
- Parameters
containers (list) – A list of inference containers that can be used for inference specifications of Model Package (default: None).
content_types (list) – The supported MIME types for the input data (default: None).
response_types (list) – The supported MIME types for the output data (default: None).
inference_instances (list) – A list of the instance types that are used to generate inferences in real-time (default: None).
transform_instances (list) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed (default: None).
model_package_name (str) – Model Package name, exclusive to model_package_group_name, using model_package_name makes the Model Package un-versioned (default: None).
model_package_group_name (str) – Model Package Group name, exclusive to model_package_name, using model_package_group_name makes the Model Package versioned (default: None).
model_metrics (ModelMetrics) – ModelMetrics object (default: None).
metadata_properties (MetadataProperties) – MetadataProperties object (default: None)
marketplace_cert (bool) – A boolean value indicating if the Model Package is certified for AWS Marketplace (default: False).
approval_status (str) – Model Approval Status, values can be “Approved”, “Rejected”, or “PendingManualApproval” (default: “PendingManualApproval”).
description (str) – Model Package description (default: None).
drift_check_baselines (DriftCheckBaselines) – DriftCheckBaselines object (default: None).
customer_metadata_properties (dict[str, str]) – A dictionary of key-value paired metadata properties (default: None).
domain (str) – Domain values can be “COMPUTER_VISION”, “NATURAL_LANGUAGE_PROCESSING”, “MACHINE_LEARNING” (default: None).
sample_payload_url (str) – The S3 path where the sample payload is stored (default: None).
task (str) – Task values which are supported by Inference Recommender are “FILL_MASK”, “IMAGE_CLASSIFICATION”, “OBJECT_DETECTION”, “TEXT_GENERATION”, “IMAGE_SEGMENTATION”, “CLASSIFICATION”, “REGRESSION”, “OTHER” (default: None).
skip_model_validation (str) – Indicates if you want to skip model validation. Values can be “All” or “None” (default: None).
source_uri (str) – The URI of the source for the model package (default: None).
model_card (ModeCard or ModelPackageModelCard) – document contains qualitative and quantitative information about a model (default: None).
model_life_cycle (ModelLifeCycle) – ModelLifeCycle object (default: None).
- wait_for_model_package(model_package_name, poll=5)¶
Wait for an Amazon SageMaker endpoint deployment to complete.
- Parameters
- Returns
Return value from the
DescribeEndpoint
API.- Return type
- Raises
exceptions.CapacityError – If the Model Package job fails with CapacityError.
exceptions.UnexpectedStatusException – If waiting and the Model Package job fails.
- describe_model(name)¶
Calls the DescribeModel API for the given model name.
- create_endpoint_config(name, model_name, initial_instance_count, instance_type, accelerator_type=None, tags=None, kms_key=None, data_capture_config_dict=None, volume_size=None, model_data_download_timeout=None, container_startup_health_check_timeout=None, explainer_config_dict=None)¶
Create an Amazon SageMaker endpoint configuration.
The endpoint configuration identifies the Amazon SageMaker model (created using the
CreateModel
API) and the hardware configuration on which to deploy the model. Provide this endpoint configuration to theCreateEndpoint
API, which then launches the hardware and deploys the model.- Parameters
name (str) – Name of the Amazon SageMaker endpoint configuration to create.
model_name (str) – Name of the Amazon SageMaker
Model
.initial_instance_count (int) – Minimum number of EC2 instances to launch. The actual number of active instances for an endpoint at any given time varies due to autoscaling.
instance_type (str) – Type of EC2 instance to launch, for example, ‘ml.c4.xlarge’.
accelerator_type (str) – Type of Elastic Inference accelerator to attach to the instance. For example, ‘ml.eia1.medium’. For more information: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html
tags (Optional[Tags]) – Optional. The list of tags to add to the endpoint config.
kms_key (str) – The KMS key that is used to encrypt the data on the storage volume attached to the instance hosting the endpoint.
data_capture_config_dict (dict) – Specifies configuration related to Endpoint data capture for use with Amazon SageMaker Model Monitoring. Default: None.
volume_size (int) – The size, in GB, of the ML storage volume attached to individual inference instance associated with the production variant. Currenly only Amazon EBS gp2 storage volumes are supported.
model_data_download_timeout (int) – The timeout value, in seconds, to download and extract model data from Amazon S3 to the individual inference instance associated with this production variant.
container_startup_health_check_timeout (int) – The timeout value, in seconds, for your inference container to pass health check by SageMaker Hosting. For more information about health check see: https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code .html#your-algorithms -inference-algo-ping-requests
explainer_config_dict (dict) – Specifies configuration to enable explainers. Default: None.
Example
>>> tags = [{'Key': 'tagname', 'Value': 'tagvalue'}] For more information about tags, see https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker .html#SageMaker .Client.add_tags
- Returns
Name of the endpoint point configuration created.
- Return type
- create_endpoint_config_from_existing(existing_config_name, new_config_name, new_tags=None, new_kms_key=None, new_data_capture_config_dict=None, new_production_variants=None, new_explainer_config_dict=None, endpoint_type=EndpointType.MODEL_BASED)¶
Create an Amazon SageMaker endpoint configuration from an existing one.
It also updates any values that were passed in. The endpoint configuration identifies the Amazon SageMaker model (created using the
CreateModel
API) and the hardware configuration on which to deploy the model. Provide this endpoint configuration to theCreateEndpoint
API, which then launches the hardware and deploys the model.- Parameters
new_config_name (str) – Name of the Amazon SageMaker endpoint configuration to create.
existing_config_name (str) – Name of the existing Amazon SageMaker endpoint configuration.
new_tags (Optional[Tags]) – Optional. The list of tags to add to the endpoint config. If not specified, the tags of the existing endpoint configuration are used. If any of the existing tags are reserved AWS ones (i.e. begin with “aws”), they are not carried over to the new endpoint configuration.
new_kms_key (str) – The KMS key that is used to encrypt the data on the storage volume attached to the instance hosting the endpoint (default: None). If not specified, the KMS key of the existing endpoint configuration is used.
new_data_capture_config_dict (dict) – Specifies configuration related to Endpoint data capture for use with Amazon SageMaker Model Monitoring (default: None). If not specified, the data capture configuration of the existing endpoint configuration is used.
new_production_variants (list[dict]) – The configuration for which model(s) to host and the resources to deploy for hosting the model(s). If not specified, the
ProductionVariants
of the existing endpoint configuration is used.new_explainer_config_dict (dict) – Specifies configuration to enable explainers. (default: None). If not specified, the explainer configuration of the existing endpoint configuration is used.
endpoint_type (EndpointType) – The Endpoint type to determine whether model will be deployed as Amazon SageMaker Inference Component to the endpoint
- Returns
Name of the endpoint point configuration created.
- Return type
- create_endpoint(endpoint_name, config_name, tags=None, wait=True, live_logging=False)¶
Create an Amazon SageMaker
Endpoint
according to the configuration in the request.Once the
Endpoint
is created, client applications can send requests to obtain inferences. The endpoint configuration is created using theCreateEndpointConfig
API.- Parameters
endpoint_name (str) – Name of the Amazon SageMaker
Endpoint
being created.config_name (str) – Name of the Amazon SageMaker endpoint configuration to deploy.
wait (bool) – Whether to wait for the endpoint deployment to complete before returning (default: True).
tags (Optional[Tags]) – A list of key-value pairs for tagging the endpoint (default: None).
- Returns
Name of the Amazon SageMaker
Endpoint
created.- Return type
- Raises
botocore.exceptions.ClientError – If Sagemaker throws an exception while creating
endpoint. –
- endpoint_in_service_or_not(endpoint_name)¶
Check whether an Amazon SageMaker
Endpoint`
is in IN_SERVICE status.Raise any exception that is not recognized as “not found”.
- Parameters
endpoint_name (str) – Name of the Amazon SageMaker
Endpoint
tostatus. (check) –
- Returns
True if
Endpoint
is IN_SERVICE, False ifEndpoint
not exists or it’s in other status.- Return type
Raises:
- update_endpoint(endpoint_name, endpoint_config_name, wait=True)¶
Update an Amazon SageMaker
Endpoint
, Raise an error endpoint_name does not exist.- Parameters
- Returns
Name of the Amazon SageMaker
Endpoint
being updated.- Return type
- Raises
- ValueError – if the endpoint does not already exist
- botocore.exceptions.ClientError – If SageMaker throws an error while
creating endpoint config, describing endpoint or updating endpoint –
- is_inference_component_based_endpoint(endpoint_name)¶
Returns ‘True’ if endpoint is inference-component-based, ‘False’ otherwise.
An endpoint is inference component based if and only if the associated endpoint config has a role associated with it and no production variants with a
ModelName
field.- Parameters
endpoint_name (str) – Name of the Amazon SageMaker
Endpoint
to determine if inference-component-based.
- describe_endpoint(endpoint_name)¶
Describe an Amazon SageMaker
Endpoint
.- Parameters
endpoint_name (str) – Name of the Amazon SageMaker
Endpoint
to describe.
- delete_endpoint(endpoint_name)¶
Delete an Amazon SageMaker
Endpoint
.- Parameters
endpoint_name (str) – Name of the Amazon SageMaker
Endpoint
to delete.
- delete_endpoint_config(endpoint_config_name)¶
Delete an Amazon SageMaker endpoint configuration.
- Parameters
endpoint_config_name (str) – Name of the Amazon SageMaker endpoint configuration to delete.
- create_inference_component(inference_component_name, endpoint_name, variant_name, specification, runtime_config=None, tags=None, wait=True)¶
Create an Amazon SageMaker Inference Component.
- Parameters
inference_component_name (str) – Name of the Amazon SageMaker inference component to create.
endpoint_name (str) – Name of the Amazon SageMaker endpoint that the inference component will deploy to.
variant_name (str) – Name of the Amazon SageMaker variant that the inference component will deploy to.
specification (Dict[str, Any]) – The inference component specification.
runtime_config (Optional[Dict[str, Any]]) – Optional. The inference component runtime configuration. (Default: None).
tags (Optional[Tags]) – Optional. Either a dictionary or a list of dictionaries containing key-value pairs. (Default: None).
wait (bool) – Optional. Wait for the inference component to finish being created before returning a value. (Default: True).
- Returns
Name of the Amazon SageMaker
InferenceComponent
if created.- Return type
- wait_for_inference_component(inference_component_name, poll=20)¶
Wait for an Amazon SageMaker
Inference Component
deployment to complete.- Parameters
- Raises
exceptions.CapacityError – If the inference component creation fails with CapacityError.
exceptions.UnexpectedStatusException – If the inference component creation fails.
- Returns
Return value from the
DescribeInferenceComponent
API.- Return type
- describe_inference_component(inference_component_name)¶
Describe an Amazon SageMaker
InferenceComponent
- wait_for_inference_component_deletion(inference_component_name, poll=20)¶
Wait for an Amazon SageMaker
Inference Component
deployment to complete.
- update_inference_component(inference_component_name, specification=None, runtime_config=None, wait=True)¶
Update an Amazon SageMaker
InferenceComponent
- Parameters
inference_component_name (str) – Name of the Amazon SageMaker
InferenceComponent
.specification ([dict[str,int]]) – Resource configuration. Optional. Example: { “MinMemoryRequiredInMb”: 1024, “NumberOfCpuCoresRequired”: 1, “NumberOfAcceleratorDevicesRequired”: 1, “MaxMemoryRequiredInMb”: 4096, },
runtime_config ([dict[str,int]]) – Number of copies. Optional. Default: { “copyCount”: 1 }
wait – Wait for inference component to be created before return. Optional. Default is True.
- Returns
inference component name
- Return type
- Raises
ValueError – If the inference_component_name does not exist.
- delete_inference_component(inference_component_name, wait=False)¶
Deletes an InferenceComponent.
- list_and_paginate_inference_component_names_associated_with_endpoint(endpoint_name)¶
List inference component names for an endpoint, concatenating paginated results.
- list_inference_components(endpoint_name_equals=None, variant_name_equals=None, name_contains=None, creation_time_after=None, creation_time_before=None, last_modified_time_after=None, last_modified_time_before=None, status_equals=None, sort_order=None, sort_by=None, max_results=None, next_token=None)¶
List inference components under current endpoint.
- Parameters
endpoint_name_equals (str) – Optional. A string that matches the name of the endpoint that the inference components are deployed to. (Default: None).
variant_name_equals (str) – Optional. A string that matches the name of the variant associated with the inference components. (Default: None).
name_contains (str) – Optional. A string that partially matches the names of one or more inference components. Filters inference components by name. (Default: None).
creation_time_after (datetime.datetime) – Optional. Use this parameter to search for inference components that were created after a specific date and time. (Default: None).
creation_time_before (datetime.datetime) – Optional. Use this parameter to search for inference components that were created before a specific date and time. (Default: None).
last_modified_time_after (datetime.datetime) – Optional. Use this parameter to search for inference components that were last modified after a specific date and time. (Default: None).
last_modified_time_before (datetime.datetime) – Optional. Use this parameter to search for inference components that were last modified before a specific date and time. (Default: None).
status_equals (str) – Optional. The inference component status to apply as a filter to the response. (Default: None).
sort_order (str) – Optional. The order in which inference components are listed in the response. (Default: None).
sort_by (str) – Optional. The value on which the inference component list is sorted. (Default: None).
max_results (int) – Optional. The maximum number of results returned by list_inference_components. (Default: None).
next_token (str) – Optional. A token to resume pagination of list_inference_components results. (Default: None).
- Returns
Dict of inference component details.
- Return type
- delete_model(model_name)¶
Delete an Amazon SageMaker Model.
- Parameters
model_name (str) – Name of the Amazon SageMaker model to delete.
- list_group_resources(group, filters, next_token='')¶
To list group resources with given filters
- delete_resource_group(group)¶
To delete a resource group
- Parameters
group (str) – The name or the ARN of the resource group to delete.
- get_resource_group_query(group)¶
To get the group query for an AWS Resource Group
- Parameters
group (str) – The name or the ARN of the resource group to query.
- get_tagging_resources(tag_filters, resource_type_filters)¶
To list the complete resources for a particular resource group tag
tag_filters: filters for the tag resource_type_filters: resource filter for the tag
- create_group(name, resource_query, tags)¶
To create a AWS Resource Group
- list_tags(resource_arn, max_results=50)¶
List the tags given an Amazon Resource Name.
- wait_for_job(job, poll=5)¶
Wait for an Amazon SageMaker training job to complete.
- Parameters
- Returns
Return value from the
DescribeTrainingJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the training job fails with CapacityError.
exceptions.UnexpectedStatusException – If the training job fails.
- wait_for_processing_job(job, poll=5)¶
Wait for an Amazon SageMaker Processing job to complete.
- Parameters
- Returns
Return value from the
DescribeProcessingJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the processing job fails with CapacityError.
exceptions.UnexpectedStatusException – If the processing job fails.
- wait_for_compilation_job(job, poll=5)¶
Wait for an Amazon SageMaker Neo compilation job to complete.
- Parameters
- Returns
Return value from the
DescribeCompilationJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the compilation job fails with CapacityError.
exceptions.UnexpectedStatusException – If the compilation job fails.
- wait_for_edge_packaging_job(job, poll=5)¶
Wait for an Amazon SageMaker Edge packaging job to complete.
- Parameters
- Returns
Return value from the
DescribeEdgePackagingJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the edge packaging job fails with CapacityError.
exceptions.UnexpectedStatusException – If the edge packaging job fails.
- wait_for_tuning_job(job, poll=5)¶
Wait for an Amazon SageMaker hyperparameter tuning job to complete.
- Parameters
- Returns
Return value from the
DescribeHyperParameterTuningJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the hyperparameter tuning job fails with CapacityError.
exceptions.UnexpectedStatusException – If the hyperparameter tuning job fails.
- describe_transform_job(job_name)¶
Calls the DescribeTransformJob API for the given job name and returns the response.
- wait_for_transform_job(job, poll=5)¶
Wait for an Amazon SageMaker transform job to complete.
- Parameters
- Returns
Return value from the
DescribeTransformJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the transform job fails with CapacityError.
exceptions.UnexpectedStatusException – If the transform job fails.
- stop_transform_job(name)¶
Stop the Amazon SageMaker hyperparameter tuning job with the specified name.
- Parameters
name (str) – Name of the Amazon SageMaker batch transform job.
- Raises
ClientError – If an error occurs while trying to stop the batch transform job.
- wait_for_endpoint(endpoint, poll=30, live_logging=False)¶
Wait for an Amazon SageMaker endpoint deployment to complete.
- Parameters
- Raises
exceptions.CapacityError – If the endpoint creation job fails with CapacityError.
exceptions.UnexpectedStatusException – If the endpoint creation job fails.
- Returns
Return value from the
DescribeEndpoint
API.- Return type
- endpoint_from_job(job_name, initial_instance_count, instance_type, image_uri=None, name=None, role=None, wait=True, model_environment_vars=None, vpc_config_override='VPC_CONFIG_DEFAULT', accelerator_type=None, data_capture_config=None)¶
Create an
Endpoint
using the results of a successful training job.Specify the job name, Docker image containing the inference code, and hardware configuration to deploy the model. Internally the API, creates an Amazon SageMaker model (that describes the model artifacts and the Docker image containing inference code), endpoint configuration (describing the hardware to deploy for hosting the model), and creates an
Endpoint
(launches the EC2 instances and deploys the model on them). In response, the API returns the endpoint name to which you can send requests for inferences.- Parameters
job_name (str) – Name of the training job to deploy the results of.
initial_instance_count (int) – Minimum number of EC2 instances to launch. The actual number of active instances for an endpoint at any given time varies due to autoscaling.
instance_type (str) – Type of EC2 instance to deploy to an endpoint for prediction, for example, ‘ml.c4.xlarge’.
image_uri (str) – The Docker image which defines the inference code to be used as the entry point for accepting prediction requests. If not specified, uses the image used for the training job.
name (str) – Name of the
Endpoint
to create. If not specified, uses the training job name.role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. You must grant sufficient permissions to this role.
wait (bool) – Whether to wait for the endpoint deployment to complete before returning (default: True).
model_environment_vars (dict[str, str]) – Environment variables to set on the model container (default: None).
vpc_config_override (dict[str, list[str]]) – Overrides VpcConfig set on the model. Default: use VpcConfig from training job. * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
accelerator_type (str) – Type of Elastic Inference accelerator to attach to the instance. For example, ‘ml.eia1.medium’. For more information: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html
data_capture_config (sagemaker.model_monitor.DataCaptureConfig) – Specifies configuration related to Endpoint data capture for use with Amazon SageMaker Model Monitoring. Default: None.
- Returns
Name of the
Endpoint
that is created.- Return type
- endpoint_from_model_data(model_s3_location, image_uri, initial_instance_count, instance_type, name=None, role=None, wait=True, model_environment_vars=None, model_vpc_config=None, accelerator_type=None, data_capture_config=None, tags=None)¶
Create and deploy to an
Endpoint
using existing model data stored in S3.- Parameters
model_s3_location (str or dict) – S3 location of the model artifacts to use for the endpoint.
image_uri (str) – The Docker image URI which defines the runtime code to be used as the entry point for accepting prediction requests.
initial_instance_count (int) – Minimum number of EC2 instances to launch. The actual number of active instances for an endpoint at any given time varies due to autoscaling.
instance_type (str) – Type of EC2 instance to deploy to an endpoint for prediction, e.g. ‘ml.c4.xlarge’.
name (str) – Name of the
Endpoint
to create. If not specified, uses a name generated by combining the image name with a timestamp.role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. You must grant sufficient permissions to this role.
wait (bool) – Whether to wait for the endpoint deployment to complete before returning (default: True).
model_environment_vars (dict[str, str]) – Environment variables to set on the model container (default: None).
model_vpc_config (dict[str, list[str]]) – The VpcConfig set on the model (default: None) * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
accelerator_type (str) – Type of Elastic Inference accelerator to attach to the instance. For example, ‘ml.eia1.medium’. For more information: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html
data_capture_config (sagemaker.model_monitor.DataCaptureConfig) – Specifies configuration related to Endpoint data capture for use with Amazon SageMaker Model Monitoring. Default: None.
tags (Optional[Tags]) – A list of key-value pairs for tagging the endpoint (default: None).
- Returns
Name of the
Endpoint
that is created.- Return type
- endpoint_from_production_variants(name, production_variants, tags=None, kms_key=None, wait=True, data_capture_config_dict=None, async_inference_config_dict=None, explainer_config_dict=None, live_logging=False, vpc_config=None, enable_network_isolation=None, role=None)¶
Create an SageMaker
Endpoint
from a list of production variants.- Parameters
name (str) – The name of the
Endpoint
to create.production_variants (list[dict[str, str]]) – The list of production variants to deploy.
tags (Optional[Tags]) – A list of key-value pairs for tagging the endpoint (default: None).
kms_key (str) – The KMS key that is used to encrypt the data on the storage volume attached to the instance hosting the endpoint.
wait (bool) – Whether to wait for the endpoint deployment to complete before returning (default: True).
data_capture_config_dict (dict) – Specifies configuration related to Endpoint data capture for use with Amazon SageMaker Model Monitoring. Default: None.
async_inference_config_dict (dict) – specifies configuration related to async endpoint. Use this configuration when trying to create async endpoint and make async inference (default: None)
explainer_config_dict (dict) – Specifies configuration related to explainer. Use this configuration when trying to use online explainability. (default: None).
(dict[str (vpc_config) – The VpcConfig set on the model (default: None). * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
list[str]] – The VpcConfig set on the model (default: None). * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
enable_network_isolation (Boolean) – Default False. If True, enables network isolation in the endpoint, isolating the model container. No inbound or outbound network calls can be made to or from the model container.
role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. After the endpoint is created, the inference code might use the IAM role if it needs to access some AWS resources. (default: None).
- Returns
The name of the created
Endpoint
.- Return type
- expand_role(role)¶
Expand an IAM role name into an ARN.
If the role is already in the form of an ARN, then the role is simply returned. Otherwise we retrieve the full ARN and return it.
- get_caller_identity_arn()¶
Returns the ARN user or role whose credentials are used to call the API.
- Returns
The ARN user or role
- Return type
- logs_for_job(job_name, wait=False, poll=10, log_type='All', timeout=None)¶
Display logs for a given training job, optionally tailing them until job is complete.
If the output is a tty or a Jupyter cell, it will be color-coded based on which instance the log entry is from.
- Parameters
job_name (str) – Name of the training job to display the logs for.
wait (bool) – Whether to keep looking for new log entries until the job completes (default: False).
poll (int) – The interval in seconds between polling for new log entries and job completion (default: 5).
log_type ([str]) – A list of strings specifying which logs to print. Acceptable strings are “All”, “None”, “Training”, or “Rules”. To maintain backwards compatibility, boolean values are also accepted and converted to strings.
timeout (int) – Timeout in seconds to wait until the job is completed.
None
by default.
- Raises
exceptions.CapacityError – If the training job fails with CapacityError.
exceptions.UnexpectedStatusException – If waiting and the training job fails.
- logs_for_processing_job(job_name, wait=False, poll=10)¶
Display logs for a given processing job, optionally tailing them until the is complete.
- Parameters
- Raises
ValueError – If the processing job fails.
- logs_for_transform_job(job_name, wait=False, poll=10)¶
Display logs for a given training job, optionally tailing them until job is complete.
If the output is a tty or a Jupyter cell, it will be color-coded based on which instance the log entry is from.
- Parameters
- Raises
ValueError – If the transform job fails.
- delete_feature_group(feature_group_name)¶
Deletes a FeatureGroup in the FeatureStore service.
- Parameters
feature_group_name (str) – name of the feature group to be deleted.
- create_feature_group(feature_group_name, record_identifier_name, event_time_feature_name, feature_definitions, role_arn=None, online_store_config=None, offline_store_config=None, throughput_config=None, description=None, tags=None)¶
Creates a FeatureGroup in the FeatureStore service.
- Parameters
feature_group_name (str) – name of the FeatureGroup.
record_identifier_name (str) – name of the record identifier feature.
event_time_feature_name (str) – name of the event time feature.
feature_definitions (Sequence[Dict[str, str]]) – list of feature definitions.
role_arn (str) – ARN of the role will be used to execute the api.
online_store_config (Dict[str, str]) – dict contains configuration of the feature online store.
offline_store_config (Dict[str, str]) – dict contains configuration of the feature offline store.
throughput_config (Dict[str, str]) – dict contains throughput configuration for the feature group.
description (str) – description of the FeatureGroup.
tags (Optional[Tags]) – tags for labeling a FeatureGroup.
- Returns
Response dict from service.
- Return type
- describe_feature_group(feature_group_name, next_token=None)¶
Describe a FeatureGroup by name in FeatureStore service.
- update_feature_group(feature_group_name, feature_additions=None, online_store_config=None, throughput_config=None)¶
Update a FeatureGroup
Supports modifications like adding new features from the given feature definitions, updating online store and throughput configurations.
- Parameters
feature_group_name (str) – name of the FeatureGroup to update.
feature_additions (Sequence[Dict[str, str]) – list of feature definitions to be updated.
online_store_config (Dict[str, Any]) – updates to online store config
throughput_config (Dict[str, Any]) – target throughput configuration of the feature group
- Returns
Response dict from service.
- Return type
- list_feature_groups(name_contains, feature_group_status_equals, offline_store_status_equals, creation_time_after, creation_time_before, sort_order, sort_by, max_results, next_token)¶
List all FeatureGroups satisfying given filters.
- Parameters
name_contains (str) – A string that partially matches one or more FeatureGroups’ names. Filters FeatureGroups by name.
feature_group_status_equals (str) – A FeatureGroup status. Filters FeatureGroups by FeatureGroup status.
offline_store_status_equals (str) – An OfflineStore status. Filters FeatureGroups by OfflineStore status.
creation_time_after (datetime.datetime) – Use this parameter to search for FeatureGroups created after a specific date and time.
creation_time_before (datetime.datetime) – Use this parameter to search for FeatureGroups created before a specific date and time.
sort_order (str) – The order in which FeatureGroups are listed.
sort_by (str) – The value on which the FeatureGroup list is sorted.
max_results (int) – The maximum number of results returned by ListFeatureGroups.
next_token (str) – A token to resume pagination of ListFeatureGroups results.
- Returns
Response dict from service.
- Return type
- update_feature_metadata(feature_group_name, feature_name, description=None, parameter_additions=None, parameter_removals=None)¶
Update a feature metadata and add/remove metadata.
- Parameters
feature_group_name (str) – name of the FeatureGroup to update.
feature_name (str) – name of the feature to update.
description (str) – description of the feature to update.
parameter_additions (Sequence[Dict[str, str]) – list of feature parameter to be added.
parameter_removals (Sequence[Dict[str, str]) – list of feature parameter to be removed.
- Returns
Response dict from service.
- Return type
- describe_feature_metadata(feature_group_name, feature_name)¶
Describe feature metadata by feature name in FeatureStore service.
- search(resource, search_expression=None, sort_by=None, sort_order=None, next_token=None, max_results=None)¶
Search for SageMaker resources satisfying given filters.
- Parameters
resource (str) – The name of the Amazon SageMaker resource to search for.
search_expression (Dict[str, any]) – A Boolean conditional statement. Resources must satisfy this condition to be included in search results.
sort_by (str) – The name of the resource property used to sort the
SearchResults
. The default isLastModifiedTime
.sort_order (str) – How
SearchResults
are ordered. Valid values areAscending
orDescending
. The default isDescending
. next_token (str): If more thanMaxResults
resources match the specifiedSearchExpression
, the response includes aNextToken
. TheNextToken
can be passed to the nextSearchRequest
to continue retrieving results.max_results (int) – The maximum number of results to return.
- Returns
Response dict from service.
- Return type
- put_record(feature_group_name, record, target_stores=None, ttl_duration=None)¶
Puts a single record in the FeatureGroup.
- Parameters
feature_group_name (str) – Name of the FeatureGroup.
record (Sequence[Dict[str, str]]) – List of FeatureValue dicts to be ingested into FeatureStore.
target_stores (Sequence[str]) – Optional. List of target stores to put the record.
ttl_duration (Dict[str, str]) – Optional. Time-to-Live (TTL) duration for the record.
- Returns
Response dict from service.
- delete_record(feature_group_name, record_identifier_value_as_string, event_time, deletion_mode=None)¶
Deletes a single record from the FeatureGroup.
- get_record(record_identifier_value_as_string, feature_group_name, feature_names, expiration_time_response=None)¶
Gets a single record in the FeatureGroup.
- Parameters
- Return type
- batch_get_record(identifiers, expiration_time_response=None)¶
Gets a batch of record from FeatureStore.
- start_query_execution(catalog, database, query_string, output_location, kms_key=None, workgroup=None)¶
Start Athena query execution.
- Parameters
catalog (str) – name of the data catalog.
database (str) – name of the data catalog database.
query_string (str) – SQL expression.
output_location (str) – S3 location of the output file.
kms_key (str) – KMS key id will be used to encrypt the result if given.
workgroup (str) – The name of the workgroup in which the query is being started.
specified (If the workgroup is not) –
used. (the default workgroup is) –
- Returns
Response dict from the service.
- Return type
- get_query_execution(query_execution_id)¶
Get execution status of the Athena query.
- wait_for_athena_query(query_execution_id, poll=5)¶
Wait for Athena query to finish.
- download_athena_query_result(bucket, prefix, query_execution_id, filename)¶
Download query result file from S3.
- create_inference_recommendations_job(role, sample_payload_url, supported_content_types, job_name=None, job_type='Default', model_name=None, model_package_version_arn=None, job_duration_in_seconds=None, nearest_model_name=None, supported_instance_types=None, framework=None, framework_version=None, endpoint_configurations=None, traffic_pattern=None, stopping_conditions=None, resource_limit=None)¶
Creates an Inference Recommendations Job
- Parameters
role (str) – An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. You must grant sufficient permissions to this role.
sample_payload_url (str) – The S3 path where the sample payload is stored.
supported_content_types (List[str]) – The supported MIME types for the input data.
model_name (str) – Name of the Amazon SageMaker
Model
to be used.model_package_version_arn (str) – The Amazon Resource Name (ARN) of a versioned model package.
job_name (str) – The name of the job being run.
job_type (str) – The type of job being run. Must either be Default or Advanced.
job_duration_in_seconds (int) – The maximum job duration that a job can run for. Will be used for Advanced jobs.
nearest_model_name (str) – The name of a pre-trained machine learning model benchmarked by Amazon SageMaker Inference Recommender that matches your model.
supported_instance_types (List[str]) – A list of the instance types that are used to generate inferences in real-time.
framework (str) – The machine learning framework of the Image URI.
framework_version (str) – The framework version of the Image URI.
endpoint_configurations (List[Dict[str, any]]) – Specifies the endpoint configurations to use for a job. Will be used for Advanced jobs.
traffic_pattern (Dict[str, any]) – Specifies the traffic pattern for the job. Will be used for Advanced jobs.
stopping_conditions (Dict[str, any]) – A set of conditions for stopping a recommendation job. If any of the conditions are met, the job is automatically stopped. Will be used for Advanced jobs.
resource_limit (Dict[str, any]) – Defines the resource limit for the job. Will be used for Advanced jobs.
- Returns
The name of the job created. In the form of SMPYTHONSDK-<timestamp>
- Return type
- wait_for_inference_recommendations_job(job_name, poll=120, log_level='Verbose')¶
Wait for an Amazon SageMaker Inference Recommender job to complete.
- Parameters
- Returns
Return value from the
DescribeInferenceRecommendationsJob
API.- Return type
(dict)
- Raises
exceptions.CapacityError – If the Inference Recommender job fails with CapacityError.
exceptions.UnexpectedStatusException – If the Inference Recommender job fails.
- create_presigned_mlflow_tracking_server_url(tracking_server_name, expires_in_seconds=None, session_expiration_duration_in_seconds=None)¶
Creates a Presigned Url to acess the Mlflow UI.
- Parameters
- Returns
Return value from the
CreatePresignedMlflowTrackingServerUrl
API.- Return type
(dict)
- create_hub(hub_name, hub_description, hub_display_name=None, hub_search_keywords=None, s3_storage_config=None, tags=None)¶
Creates a SageMaker Hub
- Parameters
hub_name (str) – The name of the Hub to create.
hub_description (str) – A description of the Hub.
hub_display_name (str) – The display name of the Hub.
hub_search_keywords (list) – The searchable keywords for the Hub.
s3_storage_config (S3StorageConfig) – The Amazon S3 storage configuration for the Hub.
tags (list) – Any tags to associate with the Hub.
- Returns
Return value from the
CreateHub
API.- Return type
(dict)
- describe_hub(hub_name)¶
Describes a SageMaker Hub
- list_hubs(creation_time_after=None, creation_time_before=None, max_results=None, max_schema_version=None, name_contains=None, next_token=None, sort_by=None, sort_order=None)¶
Lists all existing SageMaker Hubs
- Parameters
creation_time_after (str) – Only list HubContent that was created after the time specified.
creation_time_before (str) – Only list HubContent that was created before the time specified.
max_results (int) – The maximum amount of HubContent to list.
max_schema_version (str) – The upper bound of the HubContentSchemaVersion.
name_contains (str) – Only list HubContent if the name contains the specified string.
next_token (str) – If the response to a previous
ListHubContents
request was truncated, the response includes aNextToken
. To retrieve the next set of hub content, use the token in the next request.sort_by (str) – Sort HubContent versions by either name or creation time.
sort_order (str) – Sort Hubs by ascending or descending order.
- Returns
Return value for
ListHubs
API- Return type
(dict)
- list_hub_contents(hub_name, hub_content_type, creation_time_after=None, creation_time_before=None, max_results=None, max_schema_version=None, name_contains=None, next_token=None, sort_by=None, sort_order=None)¶
Lists the HubContents in a SageMaker Hub
- Parameters
hub_name (str) – The name of the Hub to list the contents of.
hub_content_type (str) – The type of the HubContent to list.
creation_time_after (str) – Only list HubContent that was created after the time specified.
creation_time_before (str) – Only list HubContent that was created before the time specified.
max_results (int) – The maximum amount of HubContent to list.
max_schema_version (str) – The upper bound of the HubContentSchemaVersion.
name_contains (str) – Only list HubContent if the name contains the specified string.
next_token (str) – If the response to a previous
ListHubContents
request was truncated, the response includes aNextToken
. To retrieve the next set of hub content, use the token in the next request.sort_by (str) – Sort HubContent versions by either name or creation time.
sort_order (str) – Sort Hubs by ascending or descending order.
- Returns
Return value for
ListHubContents
API- Return type
(dict)
- delete_hub(hub_name)¶
Deletes a SageMaker Hub
- Parameters
hub_name (str) – The name of the hub to delete.
- Return type
None
- create_hub_content_reference(hub_name, source_hub_content_arn, hub_content_name=None, min_version=None)¶
Creates a given HubContent reference in a SageMaker Hub
- Parameters
hub_name (str) – The name of the Hub that you want to delete content in.
source_hub_content_arn (str) – Hub content arn in the public/source Hub.
hub_content_name (str) – The name of the reference that you want to add to the Hub.
min_version (str) – A minimum version of the hub content to add to the Hub.
- Returns
Return value for
CreateHubContentReference
API- Return type
(dict)
- delete_hub_content_reference(hub_name, hub_content_type, hub_content_name)¶
Deletes a given HubContent reference in a SageMaker Hub
- describe_hub_content(hub_content_name, hub_content_type, hub_name, hub_content_version=None)¶
Describes a HubContent in a SageMaker Hub
- Parameters
- Returns
Return value for
DescribeHubContent
API- Return type
(dict)
- list_hub_content_versions(hub_name, hub_content_type, hub_content_name, min_version=None, max_schema_version=None, creation_time_after=None, creation_time_before=None, max_results=None, next_token=None, sort_by=None, sort_order=None)¶
List all versions of a HubContent in a SageMaker Hub
- sagemaker.session.get_model_package_args(content_types=None, response_types=None, inference_instances=None, transform_instances=None, model_package_name=None, model_package_group_name=None, model_data=None, image_uri=None, model_metrics=None, metadata_properties=None, marketplace_cert=False, approval_status=None, description=None, tags=None, container_def_list=None, drift_check_baselines=None, customer_metadata_properties=None, validation_specification=None, domain=None, sample_payload_url=None, task=None, skip_model_validation=None, source_uri=None, model_card=None, model_life_cycle=None)¶
Get arguments for create_model_package method.
- Parameters
content_types (list) – The supported MIME types for the input data.
response_types (list) – The supported MIME types for the output data.
inference_instances (list) – A list of the instance types that are used to generate inferences in real-time (default: None).
transform_instances (list) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed (default: None).
model_package_name (str) – Model Package name, exclusive to model_package_group_name, using model_package_name makes the Model Package un-versioned (default: None).
model_package_group_name (str) – Model Package Group name, exclusive to model_package_name, using model_package_group_name makes the Model Package
model_data (str) – s3 URI to the model artifacts from training (default: None).
image_uri (str) – Inference image uri for the container. Model class’ self.image will be used if it is None (default: None).
model_metrics (ModelMetrics) – ModelMetrics object (default: None).
metadata_properties (MetadataProperties) – MetadataProperties object (default: None).
marketplace_cert (bool) – A boolean value indicating if the Model Package is certified for AWS Marketplace (default: False).
approval_status (str) – Model Approval Status, values can be “Approved”, “Rejected”, or “PendingManualApproval” (default: “PendingManualApproval”).
description (str) – Model Package description (default: None).
tags (Optional[Tags]) – A list of dictionaries containing key-value pairs (default: None).
container_def_list (list) – A list of container defintiions (default: None).
drift_check_baselines (DriftCheckBaselines) – DriftCheckBaselines object (default: None).
customer_metadata_properties (dict[str, str]) – A dictionary of key-value paired metadata properties (default: None).
domain (str) – Domain values can be “COMPUTER_VISION”, “NATURAL_LANGUAGE_PROCESSING”, “MACHINE_LEARNING” (default: None).
sample_payload_url (str) – The S3 path where the sample payload is stored (default: None).
task (str) – Task values which are supported by Inference Recommender are “FILL_MASK”, “IMAGE_CLASSIFICATION”, “OBJECT_DETECTION”, “TEXT_GENERATION”, “IMAGE_SEGMENTATION”, “CLASSIFICATION”, “REGRESSION”, “OTHER” (default: None).
skip_model_validation (str) – Indicates if you want to skip model validation. Values can be “All” or “None” (default: None).
source_uri (str) – The URI of the source for the model package (default: None).
model_card (ModeCard or ModelPackageModelCard) – document contains qualitative and quantitative information about a model (default: None).
model_life_cycle (ModelLifeCycle) – ModelLifeCycle object (default: None).
- Returns
A dictionary of method argument names and values.
- Return type
- sagemaker.session.get_create_model_package_request(model_package_name=None, model_package_group_name=None, containers=None, content_types=None, response_types=None, inference_instances=None, transform_instances=None, model_metrics=None, metadata_properties=None, marketplace_cert=False, approval_status='PendingManualApproval', description=None, tags=None, drift_check_baselines=None, customer_metadata_properties=None, validation_specification=None, domain=None, sample_payload_url=None, task=None, skip_model_validation='None', source_uri=None, model_card=None, model_life_cycle=None)¶
Get request dictionary for CreateModelPackage API.
- Parameters
model_package_name (str) – Model Package name, exclusive to model_package_group_name, using model_package_name makes the Model Package un-versioned (default: None).
model_package_group_name (str) – Model Package Group name, exclusive to model_package_name, using model_package_group_name makes the Model Package
containers (list) – A list of inference containers that can be used for inference specifications of Model Package (default: None).
content_types (list) – The supported MIME types for the input data (default: None).
response_types (list) – The supported MIME types for the output data (default: None).
inference_instances (list) – A list of the instance types that are used to generate inferences in real-time (default: None).
transform_instances (list) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed (default: None).
model_metrics (ModelMetrics) – ModelMetrics object (default: None).
metadata_properties (MetadataProperties) – MetadataProperties object (default: None).
marketplace_cert (bool) – A boolean value indicating if the Model Package is certified for AWS Marketplace (default: False).
approval_status (str) – Model Approval Status, values can be “Approved”, “Rejected”, or “PendingManualApproval” (default: “PendingManualApproval”).
description (str) – Model Package description (default: None).
tags (Optional[Tags]) – A list of dictionaries containing key-value pairs (default: None).
drift_check_baselines (DriftCheckBaselines) – DriftCheckBaselines object (default: None).
customer_metadata_properties (dict[str, str]) – A dictionary of key-value paired metadata properties (default: None).
domain (str) – Domain values can be “COMPUTER_VISION”, “NATURAL_LANGUAGE_PROCESSING”, “MACHINE_LEARNING” (default: None).
sample_payload_url (str) – The S3 path where the sample payload is stored (default: None).
task (str) – Task values which are supported by Inference Recommender are “FILL_MASK”, “IMAGE_CLASSIFICATION”, “OBJECT_DETECTION”, “TEXT_GENERATION”, “IMAGE_SEGMENTATION”, “CLASSIFICATION”, “REGRESSION”, “OTHER” (default: None).
skip_model_validation (str) – Indicates if you want to skip model validation. Values can be “All” or “None” (default: None).
source_uri (str) – The URI of the source for the model package (default: None).
model_card (ModeCard or ModelPackageModelCard) – document contains qualitative and quantitative information about a model (default: None).
model_life_cycle (ModelLifeCycle) – ModelLifeCycle object (default: None).
- sagemaker.session.get_update_model_package_inference_args(model_package_arn, containers=None, content_types=None, response_types=None, inference_instances=None, transform_instances=None)¶
Get request dictionary for UpdateModelPackage API for inference specification.
- Parameters
model_package_arn (str) – Arn for the model package.
containers (dict) – The Amazon ECR registry path of the Docker image that contains the inference code.
content_types (list[str]) – The supported MIME types for the input data.
response_types (list[str]) – The supported MIME types for the output data.
inference_instances (list[str]) – A list of the instance types that are used to generate inferences in real-time (default: None).
transform_instances (list[str]) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed (default: None).
- sagemaker.session.get_add_model_package_inference_args(model_package_arn, name, containers=None, content_types=None, response_types=None, inference_instances=None, transform_instances=None, description=None)¶
Get request dictionary for UpdateModelPackage API for additional inference.
- Parameters
model_package_arn (str) – Arn for the model package.
name (str) – Name to identify the additional inference specification
containers (dict) – The Amazon ECR registry path of the Docker image that contains the inference code.
image_uris (List[str]) – The ECR path where inference code is stored.
description (str) – Description for the additional inference specification
content_types (list[str]) – The supported MIME types for the input data.
response_types (list[str]) – The supported MIME types for the output data.
inference_instances (list[str]) – A list of the instance types that are used to generate inferences in real-time (default: None).
transform_instances (list[str]) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed (default: None).
- sagemaker.session.update_args(args, **kwargs)¶
Updates the request arguments dict with the value if populated.
This is to handle the case that the service API doesn’t like NoneTypes for argument values.
- sagemaker.session.container_def(image_uri, model_data_url=None, env=None, container_mode=None, image_config=None, accept_eula=None, additional_model_data_sources=None, model_reference_arn=None)¶
Create a definition for executing a container as part of a SageMaker model.
- Parameters
image_uri (str) – Docker image URI to run for this container.
model_data_url (str or dict[str, Any]) – S3 location of model data required by this container, e.g. SageMaker training job model artifacts. It can either be a string representing S3 URI of model data, or a dictionary representing a
ModelDataSource
object. (default: None).env (dict[str, str]) – Environment variables to set inside the container (default: None).
container_mode (str) – The model container mode. Valid modes: * MultiModel: Indicates that model container can support hosting multiple models * SingleModel: Indicates that model container can support hosting a single model This is the default model container mode when container_mode = None
image_config (dict[str, str]) – Specifies whether the image of model container is pulled from ECR, or private registry in your VPC. By default it is set to pull model container image from ECR. (default: None).
accept_eula (bool) – For models that require a Model Access Config, specify True or False to indicate whether model terms of use have been accepted. The accept_eula value must be explicitly defined as True in order to accept the end-user license agreement (EULA) that some models require. (Default: None).
additional_model_data_sources (PipelineVariable or dict) – Additional location of SageMaker model data (default: None).
- Returns
A complete container definition object usable with the CreateModel API if passed via PrimaryContainers field.
- Return type
- sagemaker.session.pipeline_container_def(models, instance_type=None)¶
Create a definition for executing a pipeline of containers as part of a SageMaker model.
- Parameters
- Returns
- list of container definition objects usable with with the
CreateModel API for inference pipelines if passed via Containers field.
- Return type
- sagemaker.session.production_variant(model_name=None, instance_type=None, initial_instance_count=None, variant_name='AllTraffic', initial_weight=1, accelerator_type=None, serverless_inference_config=None, volume_size=None, model_data_download_timeout=None, container_startup_health_check_timeout=None, managed_instance_scaling=None, routing_config=None, inference_ami_version=None)¶
Create a production variant description suitable for use in a
ProductionVariant
list.This is also part of a
CreateEndpointConfig
request.- Parameters
model_name (str) – The name of the SageMaker model this production variant references.
instance_type (str) – The EC2 instance type for this production variant. For example, ‘ml.c4.8xlarge’.
initial_instance_count (int) – The initial instance count for this production variant (default: 1).
variant_name (string) – The
VariantName
of this production variant (default: ‘AllTraffic’).initial_weight (int) – The relative
InitialVariantWeight
of this production variant (default: 1).accelerator_type (str) – Type of Elastic Inference accelerator for this production variant. For example, ‘ml.eia1.medium’. For more information: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html
serverless_inference_config (dict) – Specifies configuration dict related to serverless endpoint. The dict is converted from sagemaker.model_monitor.ServerlessInferenceConfig object (default: None)
volume_size (int) – The size, in GB, of the ML storage volume attached to individual inference instance associated with the production variant. Currenly only Amazon EBS gp2 storage volumes are supported.
model_data_download_timeout (int) – The timeout value, in seconds, to download and extract model data from Amazon S3 to the individual inference instance associated with this production variant.
container_startup_health_check_timeout (int) – The timeout value, in seconds, for your inference container to pass health check by SageMaker Hosting. For more information about health check see: https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html #your-algorithms -inference-algo-ping-requests
- Returns
An SageMaker
ProductionVariant
description- Return type
- sagemaker.session.get_execution_role(sagemaker_session=None, use_default=False)¶
Return the role ARN whose credentials are used to call the API.
Throws an exception if role doesn’t exist.
- sagemaker.session.generate_default_sagemaker_bucket_name(boto_session)¶
Generates a name for the default sagemaker S3 bucket.
- Parameters
boto_session (boto3.session.Session) – The underlying Boto3 session which AWS service
- sagemaker.session.get_log_events_for_inference_recommender(cw_client, log_group_name, log_stream_name)¶
Retrieves log events from the specified CloudWatch log group and log stream.