Distributed Training APIs

SageMaker distributed training libraries offer both data parallel and model parallel training strategies. They combine software and hardware technologies to improve inter-GPU and inter-node communications. They extend SageMaker’s training capabilities with built-in options that require only small code changes to your training scripts.

• The SageMaker Distributed Data Parallel Library
  • Use with the SageMaker Python SDK
  • API Documentation
    • Version 1.2.x (Latest)
    • Version 1.1.x
    • Version 1.0.0
  • Release Notes
    • SageMaker Distributed Data Parallel 1.2.2 Release Notes
    • Release History
• The SageMaker Distributed Model Parallel Library
  • Use the Library’s API to Adapt Training Scripts
    • Version 1.6.0 (Latest)
    • Documentation Archive
  • Use with the SageMaker Python SDK
    • Configuration Parameters for distribution
    • Ranking Basics without Tensor Parallelism
    • Placement Strategy with Tensor Parallelism
    • Prescaled Batch
  • Release Notes
    • Sagemaker Distributed Model Parallel 1.6.0 Release Notes
    • Release History