Model Builder¶
This module contains classes related to Amazon Sagemaker Model Builder
-
class
sagemaker.serve.builder.model_builder.
ModelBuilder
(model_path='/tmp/sagemaker/model-builder/d5f05e0a9abe11ee97180242ac110002', role_arn=None, sagemaker_session=None, name='model-name-d5f05fa49abe11ee97180242ac110002', mode=<Mode.SAGEMAKER_ENDPOINT: 3>, shared_libs=<factory>, dependencies=<factory>, env_vars=<factory>, log_level=10, content_type=None, accept_type=None, s3_model_data_url=None, instance_type='ml.c5.xlarge', schema_builder=None, model=None, inference_spec=None, image_uri=None, model_server=None)¶ Class that builds a deployable model.
- Parameters
role_arn (Optional[str]) – The role for the endpoint.
model_path (Optional[str]) – The path of the model directory.
sagemaker_session (Optional[sagemaker.session.Session]) – The SageMaker session to use for the execution.
name (Optional[str]) – The model name.
mode (Optional[Mode]) –
The mode of operation. The following modes are supported:
Mode.SAGEMAKER_ENDPOINT
: Launch on a SageMaker endpointMode.LOCAL_CONTAINER
: Launch locally with a container
shared_libs (List[str]) – Any shared libraries you want to bring into the model packaging.
dependencies (Optional[Dict[str, Any]) –
The dependencies of the model or container. Takes a dict as an input where you can specify autocapture as
True
orFalse
, a requirements file, or custom dependencies as a list. A sampledependencies
dict:{ "auto": False, "requirements": "/path/to/requirements.txt", "custom": ["custom_module==1.2.3", "other_module@http://some/website.whl"], }
env_vars (Optional[Dict[str, str]) – The environment variables for the runtime execution.
log_level (Optional[int]) – The log level. Possible values are
CRITICAL
,ERROR
,WARNING
,INFO
,DEBUG
, andNOTSET
.content_type (Optional[str]) – The content type of the endpoint input data. This value is automatically derived from the input sample, but you can override it.
accept_type (Optional[str]) – The content type of the data accepted from the endpoint. This value is automatically derived from the output sample, but you can override the value.
s3_model_data_url (Optional[str]) – The S3 location where you want to upload the model package. Defaults to a S3 bucket prefixed with the account ID.
instance_type (Optional[str]) – The instance type of the endpoint. Defaults to the CPU instance type to help narrow the container type based on the instance family.
schema_builder (Optional[SchemaBuilder]) – The input/output schema of the model. The schema builder translates the input into bytes and converts the response into a stream. All translations between the server and the client are handled automatically with the specified input and output.
model (Optional[Union[object, str]) – Model object (with
predict
method to perform inference) or a HuggingFace/JumpStart Model ID. Eithermodel
orinference_spec
is required for the model builder to build the artifact.inference_spec (InferenceSpec) – The inference spec file with your customized
invoke
andload
functions.image_uri (Optional[str]) – The container image uri (which is derived from a SageMaker-based container).
model_server (Optional[ModelServer]) – The model server to which to deploy. You need to provide this argument when you specify an
image_uri
in order for model builder to build the artifacts correctly (according to the model server). Possible values for this argument areTORCHSERVE
,MMS
,TENSORFLOW_SERVING
,DJL_SERVING
,TRITON
, andTGI
.
- Return type
-
ModelBuilder.
build
(mode=None, role_arn=None, sagemaker_session=None)¶ Create a deployable
Model
instance withModelBuilder
.
-
ModelBuilder.
save
(save_path=None, s3_path=None, sagemaker_session=None, role_arn=None)¶ WARNING: This function is expremental and not intended for production use.
This function is available for models served by DJL serving.
- Parameters
- Return type
Type[sagemaker.model.Model]
-
class
sagemaker.serve.spec.inference_spec.
InferenceSpec
¶ Abstract base class for holding custom
load
,invoke
andprepare
functions.Provides a skeleton for customization to override the methods
load
,invoke
andprepare
.
-
class
sagemaker.serve.builder.schema_builder.
SchemaBuilder
(sample_input, sample_output, input_translator=None, output_translator=None)¶ Automatically detects the serializer and deserializer for your model.
This is done by inspecting the sample_input and sample_output object. Alternatively, provide your custom serializer and deserializer for your request or response by creating a class that inherits
CustomPayloadTranslator
and provide it toSchemaBuilder
.- Parameters
sample_input (object) – Sample input to the model which can be used for testing. The schema builder internally generates the content type and corresponding serializing functions.
sample_output (object) – Sample output to the model which can be used for testing. The schema builder internally generates the accept type and corresponding serializing functions.
input_translator (Optional[CustomPayloadTranslator]) – If you want to define your own serialization method for the payload, you can implement your functions for translation.
output_translator (Optional[CustomPayloadTranslator]) – If you want to define your own serialization method for the output, you can implement your functions for translation.
-
class
sagemaker.serve.marshalling.custom_payload_translator.
CustomPayloadTranslator
(content_type='application/custom', accept_type='application/custom')¶ Abstract base class for handling custom payload serialization and deserialization.
Provides a skeleton for customization requiring the overriding of the serialize_payload and deserialize_payload methods.