sagemaker.core.remote_function.core.serialization

sagemaker.core.remote_function.core.serialization#

SageMaker remote function data serializer/deserializer.

Functions

deserialize_exception_from_s3(...)

Downloads from S3 and then deserializes exception with plain SHA-256 verification.

deserialize_func_from_s3(sagemaker_session, ...)

Downloads from S3 and then deserializes function with asymmetric signature verification.

deserialize_obj_from_s3(sagemaker_session, ...)

Downloads from S3 and then deserializes data objects.

json_serialize_obj_to_s3(obj, json_key, ...)

Json serializes data object and uploads it to S3.

serialize_exception_to_s3(exc, ...[, s3_kms_key])

Serializes exception with traceback and uploads it to S3 with plain SHA-256 hashing.

serialize_func_to_s3(func, ...[, s3_kms_key])

Serializes function and uploads it to S3 with ECDSA signing.

serialize_obj_to_s3(obj, sagemaker_session, ...)

Serializes data object and uploads it to S3.

Classes

CloudpickleSerializer()

Serializer using cloudpickle.
class sagemaker.core.remote_function.core.serialization.CloudpickleSerializer[source]#

Bases: object

Serializer using cloudpickle.

static deserialize(s3_uri: str, bytes_to_deserialize: bytes) → Any[source]#

Downloads from S3 and then deserializes data objects.

Parameters:

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • bytes_to_deserialize – bytes to be deserialized.

Returns :

List of deserialized python objects.

Raises:

DeserializationError – when fail to serialize object to bytes.

static serialize(obj: Any) → bytes[source]#

Serializes data object and uploads it to S3.

Parameters:

obj – object to be serialized and persisted

Raises:

SerializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.deserialize_exception_from_s3(sagemaker_session: Session, s3_uri: str) → Any[source]#

Downloads from S3 and then deserializes exception with plain SHA-256 verification.

This function is always called from the client side (verifying job-uploaded exception), so no key is needed — only plain SHA-256 hash verification.

Parameters:

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying sagemaker session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

Returns :

Deserialized exception with traceback.

Raises:

DeserializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.deserialize_func_from_s3(sagemaker_session: Session, s3_uri: str, public_key_pem: str) → Callable[source]#

Downloads from S3 and then deserializes function with asymmetric signature verification.

This function is always called from the job side (verifying client-uploaded function).

Parameters:

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying sagemaker session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • public_key_pem (str) – PEM-encoded public key for asymmetric signature verification.

Returns :

The deserialized function.

Raises:

DeserializationError – when fail to serialize function to bytes.

sagemaker.core.remote_function.core.serialization.deserialize_obj_from_s3(sagemaker_session: Session, s3_uri: str, verification_key=None) → Any[source]#

Downloads from S3 and then deserializes data objects.

Called from both job (verifying client-uploaded args) and client (verifying job-uploaded results) sides. When verification_key is a public key PEM string, uses asymmetric signature verification. When verification_key is None, uses plain SHA-256 hash verification.

Parameters:

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying sagemaker session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • verification_key – PEM-encoded public key string for asymmetric verification (job verifying client-uploaded args), or None for plain SHA-256 verification (client verifying job-uploaded results).

Returns :

Deserialized python objects.

Raises:

DeserializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.json_serialize_obj_to_s3(obj: Any, json_key: str, sagemaker_session: Session, s3_uri: str, s3_kms_key: str | None = None)[source]#

Json serializes data object and uploads it to S3.

If a function step’s output is data referenced by other steps via JsonGet, its output should be json serialized and uploaded to S3.

Parameters:

  • obj – (Any) object to be serialized and persisted.

  • json_key – (str) the json key pointing to function step output.

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

sagemaker.core.remote_function.core.serialization.serialize_exception_to_s3(exc: Exception, sagemaker_session: Session, s3_uri: str, s3_kms_key: str | None = None)[source]#

Serializes exception with traceback and uploads it to S3 with plain SHA-256 hashing.

This function is always called from the job side (job-to-client path), so no secret key is needed — only a plain SHA-256 hash for integrity checking.

Parameters:

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

  • exc – Exception to be serialized and persisted

Raises:

SerializationError – when fail to serialize object to bytes.

sagemaker.core.remote_function.core.serialization.serialize_func_to_s3(func: Callable, sagemaker_session: Session, s3_uri: str, private_key: EllipticCurvePrivateKey, s3_kms_key: str | None = None)[source]#

Serializes function and uploads it to S3 with ECDSA signing.

This function is always called from the client side (client-to-job path).

Parameters:

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • private_key (ec.EllipticCurvePrivateKey) – ECDSA private key for signing the payload.

  • s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

  • func – function to be serialized and persisted

Raises:

SerializationError – when fail to serialize function to bytes.

sagemaker.core.remote_function.core.serialization.serialize_obj_to_s3(obj: Any, sagemaker_session: Session, s3_uri: str, signing_key=None, s3_kms_key: str | None = None)[source]#

Serializes data object and uploads it to S3.

Called from both client (args) and job (results) sides. When signing_key is an EllipticCurvePrivateKey (client signing args), uses ECDSA signing. When signing_key is None (job hashing results), uses plain SHA-256 hashing.

Parameters:

  • sagemaker_session (sagemaker.core.helper.session.Session) – The underlying Boto3 session which AWS service calls are delegated to.

  • s3_uri (str) – S3 root uri to which resulting serialized artifacts will be uploaded.

  • s3_kms_key (str) – KMS key used to encrypt artifacts uploaded to S3.

  • signing_key – ECDSA private key (ec.EllipticCurvePrivateKey) for client-side signing, or None for job-side plain SHA-256 hashing.

  • obj – object to be serialized and persisted

Raises:

SerializationError – when fail to serialize object to bytes.

Contents