Welcome to the World of Llama 3.2 Models in Amazon SageMaker JumpStart

Exciting news! The latest Llama 3.2 models are now available in Amazon SageMaker JumpStart. These models offer multi-modal vision and lightweight capabilities, representing Meta’s cutting-edge advancement in large language models (LLMs). With a focus on responsible innovation and system-level safety, these models showcase state-of-the-art performance on various industry benchmarks and introduce features that enable the creation of a new generation of AI experiences.

If you’re eager to explore and deploy the Llama 3.2 11B Vision model using SageMaker JumpStart, this post is for you. We’ll also provide insights into the supported instance types and contexts for all the Llama 3.2 models available in SageMaker JumpStart.

Llama 3.2 models are currently accessible in SageMaker JumpStart in the US East (Ohio) AWS Region. Please be aware of Meta’s restrictions on multi-modal models usage if you are located in the European Union. Refer to Meta’s community license agreement for more information.

Llama 3.2 Overview

Llama 3.2 models mark Meta’s latest progress in LLMs. These models range from small to medium-sized multi-modal models, with larger models offered in 11B and 90B parameter sizes. The lightweight text-only models are also available in 1B and 3B parameter sizes, suitable for edge devices.

The Llama 3.2 models are the first to support vision tasks, featuring a novel model architecture that merges image encoder representations into the language model. These models, characterized by responsible innovation and safety at the system level, are designed to facilitate the deployment of cutting-edge AI models, enabling innovations like image reasoning and accessibility for on-edge applications. The models are also optimized for efficiency, with reduced latency and improved performance, catering to a wide range of applications.

SageMaker JumpStart Overview

SageMaker JumpStart provides access to a diverse range of publicly available foundation models (FMs). These pre-trained models serve as robust starting points that can be customized to address specific use cases. With SageMaker JumpStart, you can leverage state-of-the-art model architectures without the need to build them from scratch.

Deploying models in a secure environment is effortless with SageMaker JumpStart, as the models can be provisioned on dedicated SageMaker Inference instances, ensuring data security and compliance within your virtual private cloud (VPC). Further customization and fine-tuning of models can be done using Amazon SageMaker’s extensive capabilities, streamlining the entire model deployment process.

Prerequisites

To explore Llama 3.2 models in SageMaker JumpStart, ensure you meet the following prerequisites:

Discover Llama 3.2 Models in SageMaker JumpStart

Discovering FMs through SageMaker JumpStart is made easy with interfaces like SageMaker Studio and the SageMaker Python SDK, providing multiple avenues to explore and utilize hundreds of models for specific use cases.

In SageMaker Studio, you can access SageMaker JumpStart by selecting “JumpStart” from the navigation pane or the Home page. Alternatively, you can use the SageMaker Python SDK for programmatic access to SageMaker JumpStart models, offering flexibility and integration with existing AI/ML workflows.

Deploy Llama 3.2 Multi-Modality Models with SageMaker JumpStart

On the SageMaker JumpStart landing page, you can browse all public pre-trained models offered by SageMaker. Select the Meta model provider tab to explore the available Meta models in SageMaker. If you’re using SageMaker Classic Studio and do not see the Llama 3.2 models, update your SageMaker Studio version for a seamless experience.

By selecting the model card, you can access details about the model, including licensing information, training data, and instructions on how to use the model. The “Deploy” and “Open Notebook” buttons enable you to utilize the model efficiently. Accept the End-User License Agreement (EULA) and acceptable use policy to proceed with model deployment.

Deploy Llama 3.2 11B Vision Model with SageMaker JumpStart Using Python SDK

By choosing “Deploy” and accepting the terms, you can initiate model deployment. Alternatively, deploy through the provided notebook by selecting “Open Notebook” for step-by-step guidance on deploying the model for inference and resource cleanup.

To deploy using a notebook, specify the model_id and deploy the selected model on SageMaker. Ensure to set accept_eula=True as a deploy method argument for successful deployment. After deployment, run inference against the endpoint using the SageMaker predictor.

Recommended Instances and Benchmark

The table below lists the Llama 3.2 models available in SageMaker JumpStart along with default instance types and supported instance types for each model. These models have been rigorously evaluated on over 150 benchmark datasets, showcasing competitive performance with leading FMs.

Inference and Example Prompts for Llama-3.2 11B Vision

Utilize Llama 3.2 11B and 90B models for a variety of text and image reasoning tasks, including image captioning, image text retrieval, visual question answering, and more. Input payloads to the endpoint can take various forms, including text-only, single-image, and multi-image inputs.

Clean Up

Upon completion, remember to delete SageMaker endpoints to avoid unnecessary costs. You can use code snippets or the SageMaker console to delete endpoints effortlessly.

Conclusion

In conclusion, SageMaker JumpStart offers a powerful platform for data scientists and ML engineers to leverage pre-trained models for inference, including Meta’s advanced models like Llama 3.2. Dive into SageMaker JumpStart today and experience the world of AI innovation with Llama 3.2 models.

About the Authors

Meet the team of passionate individuals behind this content: