AWS ML Infrastructure
- What it is
- The compute, storage, and networking that machine learning runs on — including AWS’s own purpose-built ML silicon.
- Purpose-built chips
- AWS Trainium (Trn instances) — for training large models at lower cost than comparable GPU instances.
- AWS Inferentia (Inf instances) — for high-throughput, low-cost inference.
- Both are the reason AWS can undercut general-purpose GPU pricing for these specific jobs.
- GPU instances
- P family — NVIDIA GPUs for large-scale training. G family — inference and graphics workloads.
- See Accelerated Computing (F1) for the wider accelerated-computing family.
- Supporting infrastructure
- Elastic Fabric Adapter (EFA) — low-latency, high-bandwidth interconnect for distributed training.
- Amazon FSx for Lustre — high-throughput training data access backed by Amazon S3.
- Spot Instances — large savings for checkpointed, interruption-tolerant training jobs.