RLEstimator

RLEstimator Estimator

class sagemaker.rl.estimator.RLEstimator(entry_point, toolkit=None, toolkit_version=None, framework=None, source_dir=None, hyperparameters=None, image_name=None, metric_definitions=None, **kwargs)

Bases: sagemaker.estimator.Framework

Handle end-to-end training and deployment of custom RLEstimator code.

This Estimator executes an RLEstimator script in a managed Reinforcement Learning (RL) execution environment within a SageMaker Training Job. The managed RL environment is an Amazon-built Docker container that executes functions defined in the supplied entry_point Python script.

Training is started by calling fit() on this Estimator. After training is complete, calling deploy() creates a hosted SageMaker endpoint and based on the specified framework returns an MXNetPredictor or Predictor instance that can be used to perform inference against the hosted model.

Technical documentation on preparing RLEstimator scripts for SageMaker training and using the RLEstimator is available on the project homepage: https://github.com/aws/sagemaker-python-sdk

Parameters:
  • entry_point (str) – Path (absolute or relative) to the Python source file which should be executed as the entry point to training. This should be compatible with Python 3.5 for MXNet or Python 3.6 for TensorFlow.
  • toolkit (sagemaker.rl.RLToolkit) – RL toolkit you want to use for executing your model training code.
  • toolkit_version (str) – RL toolkit version you want to be use for executing your model training code.
  • framework (sagemaker.rl.RLFramework) – Framework (MXNet or TensorFlow) you want to be used as a toolkit backed for reinforcement learning training.
  • source_dir (str) – Path (absolute or relative) to a directory with any other training source code dependencies aside from the entry point file (default: None). Structure within this directory is preserved when training on Amazon SageMaker.
  • hyperparameters (dict) – Hyperparameters that will be used for training (default: None). The hyperparameters are made accessible as a dict[str, str] to the training code on SageMaker. For convenience, this accepts other types for keys and values.
  • image_name (str) – An ECR url. If specified, the estimator will use this image for training and hosting, instead of selecting the appropriate SageMaker official image based on framework_version and py_version. Example: 123.dkr.ecr.us-west-2.amazonaws.com/my-custom-image:1.0
  • metric_definitions (list[dict]) – A list of dictionaries that defines the metric(s) used to evaluate the training jobs. Each dictionary contains two keys: ‘Name’ for the name of the metric, and ‘Regex’ for the regular expression used to extract the metric from the logs. This should be defined only for jobs that don’t use an Amazon algorithm.
  • **kwargs – Additional kwargs passed to the Framework constructor.

Tip

You can find additional parameters for initializing this class at Framework and EstimatorBase.

COACH_LATEST_VERSION_TF = '0.11.1'
COACH_LATEST_VERSION_MXNET = '0.11.0'
RAY_LATEST_VERSION = '0.6.5'
create_model(role=None, vpc_config_override='VPC_CONFIG_DEFAULT', entry_point=None, source_dir=None, dependencies=None, **kwargs)

Create a SageMaker RLEstimatorModel object that can be deployed to an Endpoint.

Parameters:
  • role (str) – The ExecutionRoleArn IAM Role ARN for the Model, which is also used during transform jobs. If not specified, the role from the Estimator will be used.
  • vpc_config_override (dict[str, list[str]]) –

    Optional override for VpcConfig set on the model. Default: use subnets and security groups from this Estimator.

    • ’Subnets’ (list[str]): List of subnet ids.
    • ’SecurityGroupIds’ (list[str]): List of security group ids.
  • entry_point (str) – Path (absolute or relative) to the Python source file which should be executed as the entry point for MXNet hosting. This should be compatible with Python 3.5 (default: self.entry_point)
  • source_dir (str) – Path (absolute or relative) to a directory with any other training source code dependencies aside from the entry point file (default: self.source_dir). Structure within this directory are preserved when hosting on Amazon SageMaker.
  • dependencies (list[str]) – A list of paths to directories (absolute or relative) with any additional libraries that will be exported to the container (default: self.dependencies). The library folders will be copied to SageMaker in the same folder where the entry_point is copied. If the `source_dir` points to S3, code will be uploaded and the S3 location will be used instead.
  • **kwargs – Additional kwargs passed to the FrameworkModel constructor.
Returns:

Depending on input parameters returns

one of the following:

  • FrameworkModel - if image_name was specified
    on the estimator;
  • MXNetModel - if image_name wasn’t specified and
    MXNet was used as the RL backend;
  • Model - if image_name wasn’t specified
    and TensorFlow was used as the RL backend.
Return type:

sagemaker.model.FrameworkModel
Raises:

ValueError – If image_name was not specified and framework enum is not valid.
train_image()

Return the Docker image to use for training.

The fit() method, which does the model training, calls this method to find the image to use for model training.

Returns:The URI of the Docker image.
Return type:str
hyperparameters()

Return hyperparameters used by your custom TensorFlow code during model training.

classmethod default_metric_definitions(toolkit)

Provides default metric definitions based on provided toolkit.

Parameters:toolkit (sagemaker.rl.RLToolkit) – RL Toolkit to be used for training.
Returns:metric definitions
Return type:list
Raises:ValueError – If toolkit enum is not valid.