Pipelines¶

ConditionStep¶

class sagemaker.workflow.condition_step.ConditionStep(name: str, depends_on: List[str] = None, conditions: List[sagemaker.workflow.conditions.Condition] = None, if_steps: List[Union[sagemaker.workflow.steps.Step, sagemaker.workflow.step_collections.StepCollection]] = None, else_steps: List[Union[sagemaker.workflow.steps.Step, sagemaker.workflow.step_collections.StepCollection]] = None)¶

Conditional step for pipelines to support conditional branching in the execution of steps.

Construct a ConditionStep for pipelines to support conditional branching.

If all of the conditions in the condition list evaluate to True, the if_steps are marked as ready for execution. Otherwise, the else_steps are marked as ready for execution.

Parameters

conditions (List[Condition]) – A list of sagemaker.workflow.conditions.Condition instances.
if_steps (List[Union[Step, StepCollection]]) – A list of sagemaker.workflow.steps.Step and sagemaker.workflow.step_collections.StepCollection instances that are marked as ready for execution if the list of conditions evaluates to True.
else_steps (List[Union[Step, StepCollection]]) – A list of sagemaker.workflow.steps.Step and sagemaker.workflow.step_collections.StepCollection instances that are marked as ready for execution if the list of conditions evaluates to False.

class sagemaker.workflow.condition_step.JsonGet(step: sagemaker.workflow.steps.Step, property_file: Union[sagemaker.workflow.properties.PropertyFile, str], json_path: str)¶

Get JSON properties from PropertyFiles.

step¶

The step from which to get the property file.

Type: Step

property_file¶

Either a PropertyFile instance or the name of a property file.

Type: Union[PropertyFile, str]

json_path¶

The JSON path expression to the requested value.

Type: str

Method generated by attrs for class JsonGet.

Conditions¶

class sagemaker.workflow.conditions.ConditionTypeEnum(*args, value=<object object>, **kwargs)¶: Condition type enum.

class sagemaker.workflow.conditions.Condition(condition_type: sagemaker.workflow.conditions.ConditionTypeEnum = NOTHING)¶

Abstract Condition entity.

condition_type¶

The type of condition.

Type: ConditionTypeEnum

Method generated by attrs for class Condition.

class sagemaker.workflow.conditions.ConditionComparison(condition_type: sagemaker.workflow.conditions.ConditionTypeEnum = NOTHING, left: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties] = None, right: Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]] = None)¶

Generic comparison condition that can be used to derive specific condition comparisons.

left¶

The execution variable, parameter, or property to use in the comparison.

Type: ConditionValueType

right¶

The execution variable, parameter, property, or Python primitive value to compare to.

Type: Union[ConditionValueType, PrimitiveType]

Method generated by attrs for class ConditionComparison.

class sagemaker.workflow.conditions.ConditionEquals(left: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties], right: Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]])¶

A condition for equality comparisons.

Construct A condition for equality comparisons.

Parameters

left (ConditionValueType) – The execution variable, parameter, or property to use in the comparison.
right (Union[ConditionValueType, PrimitiveType]) – The execution variable, parameter, property, or Python primitive value to compare to.

class sagemaker.workflow.conditions.ConditionGreaterThan(left: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties], right: Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]])¶

A condition for greater than comparisons.

Construct an instance of ConditionGreaterThan for greater than comparisons.

Parameters

left (ConditionValueType) – The execution variable, parameter, or property to use in the comparison.
right (Union[ConditionValueType, PrimitiveType]) – The execution variable, parameter, property, or Python primitive value to compare to.

class sagemaker.workflow.conditions.ConditionGreaterThanOrEqualTo(left: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties], right: Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]])¶

A condition for greater than or equal to comparisons.

Construct of ConditionGreaterThanOrEqualTo for greater than or equal to comparisons.

Parameters

left (ConditionValueType) – The execution variable, parameter, or property to use in the comparison.
right (Union[ConditionValueType, PrimitiveType]) – The execution variable, parameter, property, or Python primitive value to compare to.

class sagemaker.workflow.conditions.ConditionLessThan(left: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties], right: Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]])¶

A condition for less than comparisons.

Construct an instance of ConditionLessThan for less than comparisons.

Parameters

left (ConditionValueType) – The execution variable, parameter, or property to use in the comparison.
right (Union[ConditionValueType, PrimitiveType]) – The execution variable, parameter, property, or Python primitive value to compare to.

class sagemaker.workflow.conditions.ConditionLessThanOrEqualTo(left: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties], right: Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]])¶

A condition for less than or equal to comparisons.

Construct ConditionLessThanOrEqualTo for less than or equal to comparisons.

Parameters

left (ConditionValueType) – The execution variable, parameter, or property to use in the comparison.
right (Union[ConditionValueType, PrimitiveType]) – The execution variable, parameter, property, or Python primitive value to compare to.

class sagemaker.workflow.conditions.ConditionIn(value: Union[sagemaker.workflow.execution_variables.ExecutionVariable, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties], in_values: List[Optional[Union[sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties, str, int, float]]])¶

A condition to check membership.

Construct a ConditionIn condition to check membership.

Parameters

value (ConditionValueType) – The execution variable, parameter, or property to use for the in comparison.
in_values (List[Union[ConditionValueType, PrimitiveType]]) – The list of values to check for membership in.

class sagemaker.workflow.conditions.ConditionNot(expression: sagemaker.workflow.conditions.Condition)¶

A condition for negating another Condition.

Construct a ConditionNot condition for negating another Condition.

expression¶

A Condition to take the negation of.

Type: Condition

class sagemaker.workflow.conditions.ConditionOr(conditions: List[sagemaker.workflow.conditions.Condition] = None)¶

A condition for taking the logical OR of a list of Condition instances.

Construct a ConditionOr condition.

conditions¶

A list of Condition instances to logically OR.

Type: List[Condition]

sagemaker.workflow.conditions.primitive_or_expr(value: Union[sagemaker.workflow.entities.Expression, str, int, float, None, sagemaker.workflow.parameters.Parameter, sagemaker.workflow.properties.Properties]) → Optional[Union[Dict[str, str], str, int, float]]¶

Provide the expression of the value or return value if it is a primitive.

Parameters: value (Union[ConditionValueType, PrimitiveType]) – The value to evaluate.
Returns: Either the expression of the value or the primitive value.

Entities¶

class sagemaker.workflow.entities.Entity¶

Base object for workflow entities.

Entities must implement the to_request method.

class sagemaker.workflow.entities.DefaultEnumMeta(cls, bases, classdict)¶: An EnumMeta which defaults to the first value in the Enum list.

class sagemaker.workflow.entities.Expression¶

Base object for expressions.

Expressions must implement the expr property.

Execution_variables¶

class sagemaker.workflow.execution_variables.ExecutionVariable(*args, **kwargs)¶

Pipeline execution variables for workflow.

Create a pipeline execution variable.

Parameters: name (str) – The name of the execution variable.

class sagemaker.workflow.execution_variables.ExecutionVariables¶

Enum-like class for all ExecutionVariable instances.

Considerations to move these as module-level constants should be made.

Functions¶

class sagemaker.workflow.functions.Join(on: str = NOTHING, values: List = NOTHING)¶

Join together properties.

values¶

The primitive types and parameters to join.

Type: List[Union[PrimitiveType, Parameter]]

on_str¶

The string to join the values on (Defaults to “”).

Type: str

Method generated by attrs for class Join.

Parameters¶

class sagemaker.workflow.parameters.ParameterTypeEnum(*args, value=<object object>, **kwargs)¶: Parameter type enum.

class sagemaker.workflow.parameters.Parameter(name: str = NOTHING, parameter_type: sagemaker.workflow.parameters.ParameterTypeEnum = NOTHING, default_value: Optional[Union[str, int, float]] = None)¶

Pipeline parameter for workflow.

name¶

The name of the parameter.

Type: str

parameter_type¶

The type of the parameter.

Type: ParameterTypeEnum

default_value¶

The default Python value of the parameter.

Type: PrimitiveType

Method generated by attrs for class Parameter.

class sagemaker.workflow.parameters.ParameterString(*args, **kwargs)¶

Pipeline string parameter for workflow.

Create a pipeline string parameter.

Parameters

name (str) – The name of the parameter.
default_value (str) – The default Python value of the parameter. Defaults to None.

class sagemaker.workflow.parameters.ParameterInteger(*args, **kwargs)¶

Pipeline string parameter for workflow.

Create a pipeline integer parameter.

Parameters

name (str) – The name of the parameter.
default_value (int) – The default Python value of the parameter.

class sagemaker.workflow.parameters.ParameterFloat(*args, **kwargs)¶

Pipeline float parameter for workflow.

Create a pipeline float parameter.

Parameters

name (str) – The name of the parameter.
default_value (float) – The default Python value of the parameter.

Pipeline¶

class sagemaker.workflow.pipeline.Pipeline(name: str = NOTHING, parameters: Sequence[sagemaker.workflow.parameters.Parameter] = NOTHING, steps: Sequence[Union[sagemaker.workflow.steps.Step, sagemaker.workflow.step_collections.StepCollection]] = NOTHING, sagemaker_session: sagemaker.session.Session = NOTHING)¶

Pipeline for workflow.

name¶

The name of the pipeline.

Type: str

parameters¶

The list of the parameters.

Type: Sequence[Parameters]

steps¶

The list of the non-conditional steps associated with the pipeline. Any steps that are within the if_steps or else_steps of a ConditionStep cannot be listed in the steps of a pipeline. Of particular note, the workflow service rejects any pipeline definitions that specify a step in the list of steps of a pipeline and that step in the if_steps or else_steps of any ConditionStep.

Type: Sequence[Steps]

sagemaker_session¶

Session object that manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the pipeline creates one using the default AWS configuration chain.

Type: sagemaker.session.Session

Method generated by attrs for class Pipeline.

sagemaker.workflow.pipeline.format_start_parameters(parameters: Dict[str, Any]) → List[Dict[str, Any]]¶

Formats start parameter overrides as a list of dicts.

This list of dicts adheres to the request schema of:

{“Name”: “MyParameterName”, “Value”: “MyValue”}

Parameters: parameters (Dict[str, Any]) – A dict of named values where the keys are the names of the parameters to pass values into.

sagemaker.workflow.pipeline.interpolate(request_obj: Union[Dict[str, Any], List[Dict[str, Any]]]) → Union[Dict[str, Any], List[Dict[str, Any]]]¶

Replaces Parameter values in a list of nested Dict[str, Any] with their workflow expression.

Parameters: request_obj (RequestType) – The request dict.
Returns: The request dict with Parameter values replaced by their expression.
Return type: RequestType

sagemaker.workflow.pipeline.update_args(args: Dict[str, Any], **kwargs)¶

Updates the request arguments dict with a value, if populated.

This handles the case when the service API doesn’t like NoneTypes for argument values.

Parameters

request_args (Dict[str, Any]) – The request arguments dict.
kwargs – key, value pairs to update the args dict with.

Properties¶

class sagemaker.workflow.properties.PropertiesMeta(*args, **kwargs)¶

Load an internal shapes attribute from the botocore sagemaker service model.

Loads up the shapes from the botocore sagemaker service model.

class sagemaker.workflow.properties.Properties(path: str, shape_name: str = None)¶

Properties for use in workflow expressions.

Create a Properties instance representing the given shape.

Parameters

path (str) – The parent path of the Properties instance.
shape_name (str) – The botocore sagemaker service model shape name.

class sagemaker.workflow.properties.PropertiesList(path: str, shape_name: str = None)¶

PropertiesList for use in workflow expressions.

Create a PropertiesList instance representing the given shape.

Parameters

path (str) – The parent path of the PropertiesList instance.
shape_name (str) – The botocore sagemaker service model shape name.

class sagemaker.workflow.properties.PropertyFile(name: str, output_name: str, path: str)¶

Provides a property file struct.

name¶: The name of the property file for reference with JsonGet functions.

output_name¶: The name of the processing job output channel.

path¶: The path to the file at the output channel location.

Method generated by attrs for class PropertyFile.

Step Collections¶

class sagemaker.workflow.step_collections.StepCollection(steps: List[sagemaker.workflow.steps.Step] = NOTHING)¶

A wrapper of pipeline steps for workflow.

steps¶

A list of steps.

Type: List[Step]

Method generated by attrs for class StepCollection.

class sagemaker.workflow.step_collections.RegisterModel(name: str, estimator: sagemaker.estimator.EstimatorBase, model_data, content_types, response_types, inference_instances, transform_instances, depends_on: List[str] = None, model_package_group_name=None, model_metrics=None, approval_status=None, image_uri=None, compile_model_family=None, description=None, **kwargs)¶

Register Model step collection for workflow.

Construct steps _RepackModelStep and _RegisterModelStep based on the estimator.

Parameters

name (str) – The name of the training step.
estimator – The estimator instance.
model_data – The S3 uri to the model data from training.
content_types (list) – The supported MIME types for the input data (default: None).
response_types (list) – The supported MIME types for the output data (default: None).
inference_instances (list) – A list of the instance types that are used to generate inferences in real-time (default: None).
transform_instances (list) – A list of the instance types on which a transformation job can be run or on which an endpoint can be deployed (default: None).
depends_on (List[str]) – The list of step names the first step in the collection depends on
model_package_group_name (str) – The Model Package Group name, exclusive to model_package_name, using model_package_group_name makes the Model Package versioned (default: None).
model_metrics (ModelMetrics) – ModelMetrics object (default: None).
approval_status (str) – Model Approval Status, values can be “Approved”, “Rejected”, or “PendingManualApproval” (default: “PendingManualApproval”).
image_uri (str) – The container image uri for Model Package, if not specified, Estimator’s training container image is used (default: None).
compile_model_family (str) – The instance family for the compiled model. If specified, a compiled model is used (default: None).
description (str) – Model Package description (default: None).
**kwargs – additional arguments to create_model.

class sagemaker.workflow.step_collections.EstimatorTransformer(name: str, estimator: sagemaker.estimator.EstimatorBase, model_data, model_inputs, instance_count, instance_type, transform_inputs, image_uri=None, predictor_cls=None, env=None, strategy=None, assemble_with=None, output_path=None, output_kms_key=None, accept=None, max_concurrent_transforms=None, max_payload=None, tags=None, volume_kms_key=None, depends_on: List[str] = None, **kwargs)¶

Creates a Transformer step collection for workflow.

Construct steps required for a Transformer step collection:

An estimator-centric step collection. It models what happens in workflows when invoking the transform() method on an estimator instance: First, if custom model artifacts are required, a _RepackModelStep is included. Second, a CreateModelStep with the model data passed in from a training step or other training job output. Finally, a TransformerStep.

If repacking the model artifacts is not necessary, only the CreateModelStep and TransformerStep are in the step collection.

Parameters

name (str) – The name of the Transform Step.
estimator – The estimator instance.
instance_count (int) – The number of EC2 instances to use.
instance_type (str) – The type of EC2 instance to use.
strategy (str) – The strategy used to decide how to batch records in a single request (default: None). Valid values: ‘MultiRecord’ and ‘SingleRecord’.
assemble_with (str) – How the output is assembled (default: None). Valid values: ‘Line’ or ‘None’.
output_path (str) – The S3 location for saving the transform result. If not specified, results are stored to a default bucket.
output_kms_key (str) – Optional. A KMS key ID for encrypting the transform output (default: None).
accept (str) – The accept header passed by the client to the inference endpoint. If it is supported by the endpoint, it will be the format of the batch transform output.
env (dict) – The Environment variables to be set for use during the transform job (default: None).
depends_on (List[str]) – The list of step names the first step in the collection depends on

Steps¶

class sagemaker.workflow.steps.StepTypeEnum(*args, value=<object object>, **kwargs)¶: Enum of step types.

class sagemaker.workflow.steps.Step(name: str = NOTHING, step_type: sagemaker.workflow.steps.StepTypeEnum = NOTHING, depends_on: List[str] = None)¶

Pipeline step for workflow.

name¶

The name of the step.

Type: str

step_type¶

The type of the step.

Type: StepTypeEnum

depends_on¶

The list of step names the current step depends on

Type: List[str]

Method generated by attrs for class Step.

class sagemaker.workflow.steps.TrainingStep(name: str, estimator: sagemaker.estimator.EstimatorBase, inputs: sagemaker.inputs.TrainingInput = None, cache_config: sagemaker.workflow.steps.CacheConfig = None, depends_on: List[str] = None)¶

Training step for workflow.

Construct a TrainingStep, given an EstimatorBase instance.

In addition to the estimator instance, the other arguments are those that are supplied to the fit method of the sagemaker.estimator.Estimator.

Parameters

name (str) – The name of the training step.
estimator (EstimatorBase) – A sagemaker.estimator.EstimatorBase instance.
inputs (TrainingInput) – A sagemaker.inputs.TrainingInput instance. Defaults to None.
cache_config (CacheConfig) – A sagemaker.workflow.steps.CacheConfig instance.
depends_on (List[str]) – A list of step names this sagemaker.workflow.steps.TrainingStep depends on

class sagemaker.workflow.steps.CreateModelStep(name: str, model: sagemaker.model.Model, inputs: sagemaker.inputs.CreateModelInput, depends_on: List[str] = None)¶

CreateModel step for workflow.

Construct a CreateModelStep, given an sagemaker.model.Model instance.

In addition to the Model instance, the other arguments are those that are supplied to the _create_sagemaker_model method of the sagemaker.model.Model._create_sagemaker_model.

Parameters

name (str) – The name of the CreateModel step.
model (Model) – A sagemaker.model.Model instance.
inputs (CreateModelInput) – A sagemaker.inputs.CreateModelInput instance. Defaults to None.
depends_on (List[str]) – A list of step names this sagemaker.workflow.steps.CreateModelStep depends on

class sagemaker.workflow.steps.TransformStep(name: str, transformer: sagemaker.transformer.Transformer, inputs: sagemaker.inputs.TransformInput, cache_config: sagemaker.workflow.steps.CacheConfig = None, depends_on: List[str] = None)¶

Transform step for workflow.

Constructs a TransformStep, given an Transformer instance.

In addition to the transformer instance, the other arguments are those that are supplied to the transform method of the sagemaker.transformer.Transformer.

Parameters

name (str) – The name of the transform step.
transformer (Transformer) – A sagemaker.transformer.Transformer instance.
inputs (TransformInput) – A sagemaker.inputs.TransformInput instance.
cache_config (CacheConfig) – A sagemaker.workflow.steps.CacheConfig instance.
depends_on (List[str]) – A list of step names this sagemaker.workflow.steps.TransformStep depends on

class sagemaker.workflow.steps.ProcessingStep(name: str, processor: sagemaker.processing.Processor, inputs: List[sagemaker.processing.ProcessingInput] = None, outputs: List[sagemaker.processing.ProcessingOutput] = None, job_arguments: List[str] = None, code: str = None, property_files: List[sagemaker.workflow.properties.PropertyFile] = None, cache_config: sagemaker.workflow.steps.CacheConfig = None, depends_on: List[str] = None)¶

Processing step for workflow.

Construct a ProcessingStep, given a Processor instance.

In addition to the processor instance, the other arguments are those that are supplied to the process method of the sagemaker.processing.Processor.

Parameters

name (str) – The name of the processing step.
processor (Processor) – A sagemaker.processing.Processor instance.
inputs (List[ProcessingInput]) – A list of sagemaker.processing.ProcessorInput instances. Defaults to None.
outputs (List[ProcessingOutput]) – A list of sagemaker.processing.ProcessorOutput instances. Defaults to None.
job_arguments (List[str]) – A list of strings to be passed into the processing job. Defaults to None.
code (str) – This can be an S3 URI or a local path to a file with the framework script to run. Defaults to None.
property_files (List[PropertyFile]) – A list of property files that workflow looks for and resolves from the configured processing output list.
cache_config (CacheConfig) – A sagemaker.workflow.steps.CacheConfig instance.
depends_on (List[str]) – A list of step names this sagemaker.workflow.steps.ProcessingStep depends on

Utilities¶

sagemaker.workflow.utilities.list_to_request(entities: Sequence[Union[sagemaker.workflow.entities.Entity, sagemaker.workflow.step_collections.StepCollection]]) → List[Union[Dict[str, Any], List[Dict[str, Any]]]]¶

Get the request structure for list of entities.

Parameters: entities (Sequence[Entity]) – A list of entities.
Returns: A request structure for a workflow service call.
Return type: list