Cohere Command Light

The cohere.command-light model is deprecated.

Important

The cohere.command-light model supported for the on-demand serving mode is now retired and this model is deprecated for the dedicated serving mode. If you're hosting cohere.command-light on a dedicated AI cluster, (dedicated serving mode), you can continue to use this hosted model replica with the generation API and in the playground until the cohere.command-light model retires for the dedicated serving mode. This model, when hosted on a dedicated AI cluster is only available in US Midwest (Chicago). See Retiring the Models for retirement dates and definitions. We recommend that you use the chat models instead which offer the same text generation capabilities, including control over summary length and style.

Available in This Region

US Midwest (Chicago)

Key Features

Model has 6 billion parameters.
User prompt and response can be up to 4,096 tokens for each run.
You can fine-tune this model with your dataset.

On-Demand Mode

You can reach the pretrained foundational models in Generative AI through two modes: on-demand and dedicated. Here are key features for the on-demand mode:

You pay as you go for each inference call when you use the models in the playground or when you call the models through the API.
Low barrier to start using Generative AI.
Great for experimentation, proof of concept, and model evaluation.
Available for the pretrained models in regions not listed as (dedicated AI cluster only).

Important

Dynamic Throttling Limit Adjustment for On-Demand Mode

OCI Generative AI dynamically adjusts the request throttling limit for each active tenancy based on model demand and system capacity to optimize resource allocation and ensure fair access.

This adjustment depends on the following factors:

The current maximum throughput supported by the target model.
Any unused system capacity at the time of adjustment.
Each tenancy’s historical throughput usage and any specified override limits set for that tenancy.

Note: Because of dynamic throttling, rate limits are undocumented and can change to meet system-wide demand.

Tip

Because of the dynamic throttling limit adjustment, we recommend implementing a back-off strategy, which involves delaying requests after a rejection. Without one, repeated rapid requests can lead to further rejections over time, increased latency, and potential temporary blocking of client by the Generative AI service. By using a back-off strategy, such as an exponential back-off strategy, you can distribute requests more evenly, reduce load, and improve retry success, following industry best practices and enhancing the overall stability and performance of your integration to the service.

Dedicated AI Cluster for the Model

In the preceding region list, models in regions that aren't marked with (dedicated AI cluster only) have both on-demand and dedicated AI cluster options. For the on-demand option, you don't need clusters and you can reach the model in the Console playground or through the API.

To reach a model through a dedicated AI cluster in any listed region, you must create an endpoint for that model on a dedicated AI cluster. For the cluster unit size that matches this model, see the following table.


Base Model	Fine-Tuning Cluster	Hosting Cluster	Pricing Page Information	Request Cluster Limit Increase
Model Name: Cohere Command Light (6B) OCI Model Name: `cohere.command-light` (deprecated)	Unit Size: Small Cohere Required Units: 2	Unit Size: Small Cohere Required Units: 1	Pricing Page Product Name: Small Cohere - Dedicated For Hosting, Multiply the Unit Price: x1 For Fine-Tuning, Multiply the Unit Price: x2	Limit Name: `dedicated-unit-small-cohere-count` For Hosting, Request Limit Increase by: 1 For Fine-Tuning, Request Limit Increase by: 2

Release and Retirement Dates


Model	Release Date	On-Demand Retirement Date	Dedicated Mode Retirement Date
`cohere.command`	2024-02-07	`2024-10-02`	`2025-08-07`

Important

For a list of all model time lines and retirement details, see Retiring the Models.

Generation Model Parameters

When using the generation models, you can vary the output by changing the following parameters.

Maximum output tokens: The maximum number of tokens that you want the model to generate for each response. Estimate four characters per token.
Temperature: The level of randomness used to generate the output text.

Tip

Start with the temperature set to 0 or less than one, and increase the temperature as you regenerate the prompts for a more creative output. High temperatures can introduce hallucinations and factually incorrect information.
Top k: A sampling method in which the model chooses the next token randomly from the top k most likely tokens. A higher value for k generates more random output, which makes the output text sound more natural. The default value for k is 0 for command models and -1 for Llama models, which means that the models should consider all tokens and not use this method.
Top p: A sampling method that controls the cumulative probability of the top tokens to consider for the next token. Assign p a decimal number between 0 and 1 for the probability. For example, enter 0.75 for the top 75 percent to be considered. Set p to 1 to consider all tokens.
Stop sequences: A sequence of characters—such as a word, a phrase, a newline (\n), or a period—that tells the model when to stop the generated output. If you have more than one stop sequence, then the model stops when it reaches any of those sequences.
Frequency penalty: A penalty that's assigned to a token when that token appears frequently. High penalties encourage fewer repeated tokens and produce a more random output.
Presence penalty: A penalty that's assigned to each token when it appears in the output to encourage generating outputs with tokens that haven't been used.
Show likelihoods: Every time a new token is to be generated, a number between -15 and 0 is assigned to all tokens, where tokens with higher numbers are more likely to follow the current token. For example, it's more likely that the word favorite is followed by the word food or book rather than the word zebra. This parameter is available only for the cohere models.