Oracle Cloud Infrastructure Documentation

Chat in OCI Generative AI

Use the provided large language chat models in OCI Generative AI to ask questions and get conversational responses through an AI chatbot.

    1. In the navigation bar of the Console, select a region with Generative AI, for example, US Midwest (Chicago). See which models are offered in your region.
    2. Open the navigation menu  and select Analytics & AI. Under AI Services, select Generative AI.
    3. Select a compartment that you have permission to work in. If you don't see the playground, ask an administrator to give you access to Generative AI resources and then return to the following steps.
    4. Select Playground.
    5. Select Chat.
    6. Select a model for the chat experience by performing one of the following actions:
      • In the Model list, select a pretrained model such as meta.llama-3.1-70b-instruct, cohere.command-r-08-2024, cohere.command-r-plus-08-2024, or a custom model. The custom models are displayed as model name (endpoint name).
      • Select View model details, select a model and then select Choose model.
      Note

      • The meta.llama-3.1-405b-instruct model isn't available for on-demand access in all regions. To access this model, perform one of the following options:

        • Set up dedicated access: Switch to a a region supported for dedicated clusters for the meta.llama-3.1-405b-instruct chat model. Then, create a a hosting cluster and an endpoint for this model.
        • Switch to an on-demand region: Switch to the US Midwest (Chicago) region that's supported for on-demand inferencing for the meta.llama-3.1-405b-instruct chat model.

      • The meta.llama-3.2-11b-vision-instruct model isn't available for on-demand access in any region. To access this model, perform the following option:

      Learn about costs and model retirements for on-demand and dedicated serving modes.

    7. Start a conversation by typing a prompt or selecting an example from the Example list to use as a base prompt or to learn from.
      For models that accept images such as meta.llama-3.2-90b-vision-instruct, you can add a .png or .jpg image with a size of 5 MB or less.
    8. (Optional) Set new values for the parameters. For parameter details, see the parameter definitions select the model that you're using, listed in Chat Models.
    9. Select Submit.
    10. Enter a new prompt or to continue the chat conversation, enter a follow-up prompt and select Submit.
      Important

      In the playground, for the vision models that accept images:
      • Submitting an image without a prompt doesn't work. When you submit an image, you must submit a prompt about that image in the same request. You can then submit follow-up prompts and the model keeps the context of the conversation.
      • To add the next image and text, you must clear the chat which results in losing context of the previous conversation because of clearing the chat.
    11. (Optional) To change the responses, select Clear chat, update the prompts and parameters, and select Submit. Repeat this step until you're happy with the output.
    12. (Optional) To copy the code that generated the output, select View code, select a programming language or a framework, select Copy code, paste the code into a file and save the file. Ensure that the file maintains the format of the pasted code.
      Tip

      If you're using the code in an application:

    13. (Optional) To start a new conversation, select Clear chat.
      Note

      • When you select Clear chat, the chat conversation is erased, but the model parameters remain unchanged, and you can continue using the last settings you applied.

        If you switch to a different feature, such as Generation, and then return to the Chat playground, both the chat conversation and model parameters reset to their default values.

      Learn about Cohere chat parameters.

  • To chat, use the chat-result operation in Generative AI Inference CLI.

    Enter the following command for a list of options to use with the chat-result operation.

    oci generative-ai-inference chat-result -h

    For a complete list of parameters and values for the OCI Generative AI CLI commands, see Generative AI Inference CLI and Generative AI Management CLI.

  • Run the Chat operation to chat using the large language models.

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.