Use Meta Llama 3.1 70B and 405B in OCI Generative AI

OCI Generative AI now supports the pretrained Meta Llama 3.1 70 billion-parameter and 405 billion-parameter large language models. These models support eight languages including English, French, Hindi, Italian, Portuguese, Spanish, and Thai and have a context length of 128,000 tokens which is 16 times more than their previous Meta Llama 3 models.

Meta Llama 3.1 405B

• Can handle demanding workloads making it well-suited for high performance enterprise-level applications and research initiatives
• Provides high quality responses in areas such as general knowledge, synthetic data generation, advanced reasoning, and contextual understanding
• Supports a range of tasks including long-form text generation, multilingual translation, coding, math, and tool use

Meta Llama 3.1 70B

• Offers great results for content creation, conversational AI, and enterprise applications
• Can summarize, reword, and classify text with high accuracy
• Has sentiment analysis and language modeling capabilities
• Supports effective dialogue systems and code generation

Inference with these ready-to-use models through the OCI Generative AI Chat API or in the Console's playground. The 70B model is available in Brazil East (Sao Paulo), Germany Central (Frankfurt), UK South (London), and US Midwest (Chicago). The 405 B model is available in US Midwest (Chicago). The three other regions require that you create your own dedicated AI clusters and endpoints to host the 405B model on those clusters for inferencing. You can also fine-tune the 70B model with your own dataset using Low Rank Adaptation (LoRA).

For a list of offered models, see Pretrained Foundational Models in Generative AI. For information about the service, see the Generative AI documentation.