Paying for Dedicated AI Clusters
You get the following benefits for using dedicated AI clusters in OCI Generative AI:
- Predictable pricing that doesn’t fluctuate with demand.
- Great for fine-tuning or hosting models.
- Minimum hosting commitment: 744 unit-hours per hosting cluster.
- Minimum fine-tuning commitment: 1 unit-hour per fine-tuning job. (Fine-tuning requires 2 units to run).
The following examples calculate dedicated AI cluster cost in OCI Generative AI. For calculating on-demand inferencing cost, see Paying for On-Demand Inferencing.
Hosting a Foundational Model Example 1
John wants to host an instance of the Cohere
command
(52 billion parameters)
(cohere.command
) model on dedicated infrastructure. John
deletes the cluster after 40 days and wants to know cost of the cluster. To host a
cohere.command
model, John first needs to identify the unit
size that can host the cohere.command
model. The unit size for
cohere.command
model is a Large
Cohere unit. See matching
clusters to base models.
John needs a minimum of one Large Cohere unit to host the
cohere.command
model. Here are the steps to calculate the
cost of a hosting cluster with one Large Cohere unit.
Hosting a Foundational Model Example 2
Alice wants to host an instance of the Cohere
command
light (6 billion parameters)
(cohere.command-light
) model on dedicated infrastructure. To
host a cohere.command-light
model, Alice first needs to identify
the unit size that can host the command light model. The unit size for command light
is a Small Cohere unit. See matching clusters to base
models.
Alice decides to buy three units of Small Cohere to handle a higher call volume to the model than a single unit would provide. Alice plans to delete the cluster after five days. Here are the steps to calculate the cost of a hosting cluster with three Small Cohere units for five days.
Fine-Tuning and Hosting a Model Example
Bob wants to fine-tune a Cohere command
(52
billion parameters) (cohere.command
) model. Bob creates a
fine-tuning dedicated AI cluster with the preset value of two Large Cohere units. Bob creates a custom model on the
fine-tuning dedicated and fine-tunes a foundational model with training
data. The fine-tuning job takes 5 hours to complete. Bob creates a fine-tuning
cluster every week.
To host a cohere.command
model, Bob needs to identify the unit size
that can host the cohere.command
model. The unit size for
cohere.command
model is a Large Cohere
unit. See matching clusters to
base models. Bob can host up to 50 fine-tuned models on a single hosting
cluster. Here are the steps to calculate the monthly cost for fine-tuning and
hosting the models.