Hugging Face¶
Hugging Face Estimator¶
-
class
sagemaker.huggingface.
HuggingFace
(py_version, entry_point, transformers_version=None, tensorflow_version=None, pytorch_version=None, source_dir=None, hyperparameters=None, image_uri=None, distribution=None, compiler_config=None, **kwargs)¶ Bases:
sagemaker.estimator.Framework
Handle training of custom HuggingFace code.
This
Estimator
executes a HuggingFace script in a managed execution environment.The managed HuggingFace environment is an Amazon-built Docker container that executes functions defined in the supplied
entry_point
Python script within a SageMaker Training Job.Training is started by calling
fit()
on this Estimator.- Parameters
py_version (str) – Python version you want to use for executing your model training code. Defaults to
None
. Required unlessimage_uri
is provided. If using PyTorch, the current supported version ispy36
. If using TensorFlow, the current supported version ispy37
.entry_point (str) – Path (absolute or relative) to the Python source file which should be executed as the entry point to training. If
source_dir
is specified, thenentry_point
must point to a file located at the root ofsource_dir
.transformers_version (str) – Transformers version you want to use for executing your model training code. Defaults to
None
. Required unlessimage_uri
is provided. The current supported version is4.6.1
.tensorflow_version (str) – TensorFlow version you want to use for executing your model training code. Defaults to
None
. Required unlesspytorch_version
is provided. The current supported version is2.4.1
.pytorch_version (str) – PyTorch version you want to use for executing your model training code. Defaults to
None
. Required unlesstensorflow_version
is provided. The current supported versions are1.7.1
and1.6.0
.source_dir (str) – Path (absolute, relative or an S3 URI) to a directory with any other training source code dependencies aside from the entry point file (default: None). If
source_dir
is an S3 URI, it must point to a tar.gz file. Structure within this directory are preserved when training on Amazon SageMaker.hyperparameters (dict) – Hyperparameters that will be used for training (default: None). The hyperparameters are made accessible as a dict[str, str] to the training code on SageMaker. For convenience, this accepts other types for keys and values, but
str()
will be called to convert them before training.image_uri (str) –
If specified, the estimator will use this image for training and hosting, instead of selecting the appropriate SageMaker official image based on framework_version and py_version. It can be an ECR url or dockerhub image and tag. .. rubric:: Examples
123412341234.dkr.ecr.us-west-2.amazonaws.com/my-custom-image:1.0
custom-image:latest
If
framework_version
orpy_version
areNone
, thenimage_uri
is required. If alsoNone
, then aValueError
will be raised.distribution (dict) –
A dictionary with information on how to run distributed training (default: None). Currently, the following are supported: distributed training with parameter servers, SageMaker Distributed (SMD) Data and Model Parallelism, and MPI. SMD Model Parallelism can only be used with MPI. To enable parameter server use the following setup:
{ "parameter_server": { "enabled": True } }
To enable MPI:
{ "mpi": { "enabled": True } }
To enable SMDistributed Data Parallel or Model Parallel:
{ "smdistributed": { "dataparallel": { "enabled": True }, "modelparallel": { "enabled": True, "parameters": {} } } }
compiler_config (
TrainingCompilerConfig
) – Configures SageMaker Training Compiler to accelerate training.**kwargs – Additional kwargs passed to the
Framework
constructor.
Tip
You can find additional parameters for initializing this class at
Framework
andEstimatorBase
.-
hyperparameters
()¶ Return hyperparameters used by your custom PyTorch code during model training.
-
create_model
(model_server_workers=None, role=None, vpc_config_override='VPC_CONFIG_DEFAULT', entry_point=None, source_dir=None, dependencies=None, **kwargs)¶ Create a SageMaker
HuggingFaceModel
object that can be deployed to anEndpoint
.- Parameters
model_server_workers (int) – Optional. The number of worker processes used by the inference server. If None, server will use one worker per vCPU.
role (str) – The
ExecutionRoleArn
IAM Role ARN for theModel
, which is also used during transform jobs. If not specified, the role from the Estimator will be used.vpc_config_override (dict[str, list[str]]) – Optional override for VpcConfig set on the model. Default: use subnets and security groups from this Estimator. * ‘Subnets’ (list[str]): List of subnet ids. * ‘SecurityGroupIds’ (list[str]): List of security group ids.
entry_point (str) – Path (absolute or relative) to the local Python source file which should be executed as the entry point to training. If
source_dir
is specified, thenentry_point
must point to a file located at the root ofsource_dir
. Defaults to None.source_dir (str) – Path (absolute or relative) to a directory with any other serving source code dependencies aside from the entry point file. If not specified, the model source directory from training is used.
dependencies (list[str]) – A list of paths to directories (absolute or relative) with any additional libraries that will be exported to the container. If not specified, the dependencies from training are used. This is not supported with “local code” in Local Mode.
**kwargs – Additional kwargs passed to the
HuggingFaceModel
constructor.
- Returns
A SageMaker
HuggingFaceModel
object. SeeHuggingFaceModel()
for full details.- Return type
Hugging Face Training Compiler Configuration¶
-
class
sagemaker.huggingface.
TrainingCompilerConfig
(enabled=True, debug=False)¶ Bases:
object
The configuration class for accelerating SageMaker training jobs through compilation.
SageMaker Training Compiler speeds up training by optimizing the model execution graph.
This class initializes a
TrainingCompilerConfig
instance.Pass the output of it to the
compiler_config
parameter of theHuggingFace
class.- Parameters
Example: The following example shows the basic
compiler_config
parameter configuration, enabling compilation with default parameter values.from sagemaker.huggingface import TrainingCompilerConfig compiler_config = TrainingCompilerConfig()
-
DEBUG_PATH
= '/opt/ml/output/data/compiler/'¶
-
SUPPORTED_INSTANCE_CLASS_PREFIXES
= ['p3', 'g4dn', 'p4']¶
-
HP_ENABLE_COMPILER
= 'sagemaker_training_compiler_enabled'¶
-
HP_ENABLE_DEBUG
= 'sagemaker_training_compiler_debug_mode'¶
-
disclaimers_and_warnings
()¶ Disclaimers and warnings.
Logs disclaimers and warnings about the requested configuration of SageMaker Training Compiler.
-
classmethod
validate
(image_uri, instance_type, distribution)¶ Checks if SageMaker Training Compiler is configured correctly.
- Parameters
image_uri (str) – A string of a Docker image URI that’s specified to
HuggingFace
. If SageMaker Training Compiler is enabled, the HuggingFace estimator automatically chooses the right image URI. You cannot specify and override the image URI.instance_type (str) – A string of the training instance type that’s specified to
HuggingFace
. The validate classmethod raises error if an instance type not in theSUPPORTED_INSTANCE_CLASS_PREFIXES
list orlocal
is passed to the instance_type parameter.distribution (dict) – A dictionary of the distributed training option that’s specified to
HuggingFace
. SageMaker’s distributed data parallel and model parallel libraries are currently not compatible with SageMaker Training Compiler.
- Raises
ValueError – Raised if the requested configuration is not compatible with SageMaker Training Compiler.
Hugging Face Model¶
-
class
sagemaker.huggingface.model.
HuggingFaceModel
(role, model_data=None, entry_point=None, transformers_version=None, tensorflow_version=None, pytorch_version=None, py_version=None, image_uri=None, predictor_cls=<class 'sagemaker.huggingface.model.HuggingFacePredictor'>, model_server_workers=None, **kwargs)¶ Bases:
sagemaker.model.FrameworkModel
A Hugging Face SageMaker
Model
that can be deployed to a SageMakerEndpoint
.Initialize a HuggingFaceModel.
- Parameters
model_data (str) – The Amazon S3 location of a SageMaker model data
.tar.gz
file.role (str) – An AWS IAM role specified with either the name or full ARN. The Amazon SageMaker training jobs and APIs that create Amazon SageMaker endpoints use this role to access training data and model artifacts. After the endpoint is created, the inference code might use the IAM role, if it needs to access an AWS resource.
entry_point (str) – The absolute or relative path to the Python source file that should be executed as the entry point to model hosting. If
source_dir
is specified, thenentry_point
must point to a file located at the root ofsource_dir
. Defaults to None.transformers_version (str) – Transformers version you want to use for executing your model training code. Defaults to None. Required unless
image_uri
is provided.tensorflow_version (str) – TensorFlow version you want to use for executing your inference code. Defaults to
None
. Required unlesspytorch_version
is provided. List of supported versions: https://github.com/aws/sagemaker-python-sdk#huggingface-sagemaker-estimators.pytorch_version (str) – PyTorch version you want to use for executing your inference code. Defaults to
None
. Required unlesstensorflow_version
is provided. List of supported versions: https://github.com/aws/sagemaker-python-sdk#huggingface-sagemaker-estimators.py_version (str) – Python version you want to use for executing your model training code. Defaults to
None
. Required unlessimage_uri
is provided.image_uri (str) – A Docker image URI. Defaults to None. If not specified, a default image for PyTorch will be used. If
framework_version
orpy_version
areNone
, thenimage_uri
is required. If alsoNone
, then aValueError
will be raised.predictor_cls (callable[str, sagemaker.session.Session]) – A function to call to create a predictor with an endpoint name and SageMaker
Session
. If specified,deploy()
returns the result of invoking this function on the created endpoint name.model_server_workers (int) – Optional. The number of worker processes used by the inference server. If None, server will use one worker per vCPU.
**kwargs – Keyword arguments passed to the superclass
FrameworkModel
and, subsequently, its superclassModel
.
Tip
You can find additional parameters for initializing this class at
FrameworkModel
andModel
.-
register
(content_types, response_types, inference_instances, transform_instances, model_package_name=None, model_package_group_name=None, image_uri=None, model_metrics=None, metadata_properties=None, marketplace_cert=False, approval_status=None, description=None, drift_check_baselines=None)¶ Creates a model package for creating SageMaker models or listing on Marketplace.
- Parameters
content_types (list) – The supported MIME types for the input data.
response_types (list) – The supported MIME types for the output data.
inference_instances (list) – A list of the instance types that are used to generate inferences in real-time.
transform_instances (list) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed.
model_package_name (str) – Model Package name, exclusive to model_package_group_name, using model_package_name makes the Model Package un-versioned. Defaults to
None
.model_package_group_name (str) – Model Package Group name, exclusive to model_package_name, using model_package_group_name makes the Model Package versioned. Defaults to
None
.image_uri (str) – Inference image URI for the container. Model class’ self.image will be used if it is None. Defaults to
None
.model_metrics (ModelMetrics) – ModelMetrics object. Defaults to
None
.metadata_properties (MetadataProperties) – MetadataProperties object. Defaults to
None
.marketplace_cert (bool) – A boolean value indicating if the Model Package is certified for AWS Marketplace. Defaults to
False
.approval_status (str) – Model Approval Status, values can be “Approved”, “Rejected”, or “PendingManualApproval”. Defaults to
PendingManualApproval
.description (str) – Model Package description. Defaults to
None
.drift_check_baselines (DriftCheckBaselines) – DriftCheckBaselines object (default: None).
- Returns
A sagemaker.model.ModelPackage instance.
-
prepare_container_def
(instance_type=None, accelerator_type=None)¶ A container definition with framework configuration set in model environment variables.
- Parameters
- Returns
A container definition object usable with the CreateModel API.
- Return type
-
serving_image_uri
(region_name, instance_type, accelerator_type=None)¶ Create a URI for the serving image.
- Parameters
region_name (str) – AWS region where the image is uploaded.
instance_type (str) – SageMaker instance type. Used to determine device type (cpu/gpu/family-specific optimized).
accelerator_type (str) – The Elastic Inference accelerator type to deploy to the instance for loading and making inferences to the model.
- Returns
The appropriate image URI based on the given parameters.
- Return type
Hugging Face Predictor¶
-
class
sagemaker.huggingface.model.
HuggingFacePredictor
(endpoint_name, sagemaker_session=None, serializer=<sagemaker.serializers.JSONSerializer object>, deserializer=<sagemaker.deserializers.JSONDeserializer object>)¶ Bases:
sagemaker.predictor.Predictor
A Predictor for inference against Hugging Face Endpoints.
This is able to serialize Python lists, dictionaries, and numpy arrays to multidimensional tensors for Hugging Face inference.
Initialize an
HuggingFacePredictor
.- Parameters
endpoint_name (str) – The name of the endpoint to perform inference on.
sagemaker_session (sagemaker.session.Session) – Session object that manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the estimator creates one using the default AWS configuration chain.
serializer (sagemaker.serializers.BaseSerializer) – Optional. Default serializes input data to .npy format. Handles lists and numpy arrays.
deserializer (sagemaker.deserializers.BaseDeserializer) – Optional. Default parses the response from .npy format to numpy array.