Revolutionizing Compute with Amazon EC2 P5e Instances
In today’s technology landscape, state-of-the-art generative AI models and high-performance computing (HPC) applications are driving the need for unprecedented levels of compute. Customers across industries are pushing the boundaries of these technologies to bring higher fidelity products and experiences to market.
The exponential growth in the size of large language models (LLMs) over the past 5 years reflects a significant trend in the field of AI. As model sizes have increased to hundreds of billions of parameters, their performance on natural language processing tasks has improved significantly. However, this growth has also introduced computational and resource challenges, requiring vast amounts of computing power, memory, and storage for training and deployment.
With the demand for compute at an all-time high, Amazon Web Services (AWS) is proud to announce the general availability of Amazon Elastic Compute Cloud (Amazon EC2) P5e instances. These instances, powered by NVIDIA H200 Tensor Core GPUs, are designed to meet the high-performance and scalability needs of deep learning, generative AI, and HPC workloads.
Introducing EC2 P5e Instances
The EC2 P5e instances are equipped with NVIDIA H200 GPUs, offering 1.7 times more GPU memory capacity and 1.5 times faster GPU memory bandwidth compared to their predecessors. With 8 NVIDIA H200 GPUs, 1128 GB of high-bandwidth GPU memory, 3rd Gen AMD EPYC processors, and 30 TB of local NVMe storage, these instances provide the compute power needed for demanding workloads.
The P5e instances also offer 3,200 Gbps of aggregate network bandwidth with support for GPUDirect RDMA, enabling lower latency and efficient scale-out performance. These instances are ideal for training, fine-tuning, and running inference for complex LLMs and multimodal foundation models behind generative AI applications such as question answering, code generation, and more.
Key Features of P5e Instances
| Instance Size | vCPUs | Instance Memory (TiB) | GPU | GPU Memory | Network Bandwidth (Gbps) | GPUDirect RDMA | GPU Peer to Peer | Instance Storage (TB) | EBS Bandwidth (Gbps) |
|---|---|---|---|---|---|---|---|---|---|
| p5e.48xlarge | 192 | 2 | 8 x NVIDIA H200 | 1128 GB HBM3e | 3200 Gbps EFA | Yes | 900 GB/s NVSwitch | 8 x 3.84 NVMe SSD | 80 |
Benefits of EC2 P5e Instances
The P5e instances offer several advantages, including higher memory bandwidth, increased GPU memory capacity, and support for larger batch sizes during inference. These features result in reduced inference latency, improved throughput, and cost savings compared to alternative options.
Customers deploying LLMs for inference on P5e instances can expect up to 1.871 times higher throughput and up to 40% lower cost when compared to using comparable P5 instances. The P5e instances are also well-suited for memory-intensive HPC applications like simulations, pharmaceutical discovery, and more, providing support for DPX instruction set.
Get Started with P5e Instances
When launching P5 instances, customers can leverage AWS Deep Learning AMIs to support their workloads. By using AWS Deep Learning Containers and preconfigured environments, ML practitioners and researchers can quickly build scalable ML applications on P5 instances.
EC2 P5e instances are now available in the US East (Ohio) AWS Region, offering unmatched performance and scalability for deep learning, generative AI, and HPC workloads. Experience the next evolution in compute with Amazon EC2 P5e instances.
About the Authors
- Avi Kulkarni, Senior Specialist
- Karthik Venna, Principal Product Manager
- Khaled Rawashdeh, Senior Product Manager
- Aman Shanbhag, Associate Specialist Solutions Architect
- Pavel Belevich, Senior Applied Scientist
- Dr. Maxime Hugues, Principal WW Specialist Solutions Architect
- Shruti Koparkar, Senior Product Marketing Manager