S3 Utilities

This module contains Enums and helper methods related to S3.

class sagemaker.s3.S3Uploader

Bases: object

Contains static methods for uploading directories or files to S3.

static upload(local_path, desired_s3_uri, kms_key=None, sagemaker_session=None, callback=None)

Static method that uploads a given file or directory to S3.

Parameters:

  • local_path (str) – Path (absolute or relative) of local file or directory to upload.

  • desired_s3_uri (str) – The desired S3 location to upload to. It is the prefix to which the local filename will be added.

  • kms_key (str) – The KMS key to use to encrypt the files.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

Returns:

The S3 uri of the uploaded file(s).

static upload_string_as_file_body(body, desired_s3_uri=None, kms_key=None, sagemaker_session=None)

Static method that uploads a given file or directory to S3.

Parameters:

  • body (str) – String representing the body of the file.

  • desired_s3_uri (str) – The desired S3 uri to upload to.

  • kms_key (str) – The KMS key to use to encrypt the files.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, the estimator creates one using the default AWS configuration chain.

Returns:

The S3 uri of the uploaded file.

Return type:

str

static upload_bytes(b, s3_uri, kms_key=None, sagemaker_session=None)

Static method that uploads a given file or directory to S3.

Parameters:

  • b (bytes or io.BytesIO) – bytes.

  • s3_uri (str) – The S3 uri to upload to.

  • kms_key (str) – The KMS key to use to encrypt the files.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

Returns:

The S3 uri of the uploaded file.

Return type:

str

class sagemaker.s3.S3Downloader

Bases: object

Contains static methods for downloading directories or files from S3.

static download(s3_uri, local_path, kms_key=None, sagemaker_session=None)

Static method that downloads a given S3 uri to the local machine.

Parameters:

  • s3_uri (str) – An S3 uri to download from.

  • local_path (str) – A local path to download the file(s) to.

  • kms_key (str) – The KMS key to use to decrypt the files.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

Returns:

List of local paths of downloaded files

Return type:

list[str]

static read_file(s3_uri, sagemaker_session=None)

Static method that returns the contents of a s3 uri file body as a string.

Parameters:

  • s3_uri (str) – An S3 uri that refers to a single file.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

Returns:

The body of the file.

Return type:

str

static read_bytes(s3_uri, sagemaker_session=None)

Static method that returns the contents of a s3 object as bytes.

Parameters:

  • s3_uri (str) – An S3 uri that refers to a s3 object.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

Returns:

The body of the file.

Return type:

bytes

static list(s3_uri, sagemaker_session=None)

Static method that lists the contents of an S3 uri.

Parameters:

  • s3_uri (str) – The S3 base uri to list objects in.

  • sagemaker_session (sagemaker.session.Session) – Session object which manages interactions with Amazon SageMaker APIs and any other AWS services needed. If not specified, one is created using the default AWS configuration chain.

Returns:

The list of S3 URIs in the given S3 base uri.

Return type:

[str]