Service Limits for Generative AI
Learn about the limits for dedicated AI cluster resources in OCI Generative AI.
By default, the number of dedicated AI clusters that you get per tenancy is 0. For the name of the dedicated AI cluster that you're requesting an increase for, go to Offered Pretrained Foundational Models in Generative AI, select a model card and see the section about the Dedicated AI cluster for the model. Note that (on-demand only) models don't have a dedicated AI cluster option.
For imported models, see the Dedicated AI Cluster Hardware Unit Shapes.
To request dedicated AI clusters for a tenancy, see Creating a Limit Increase Request.
Model Endpoint Limits
By default, a hosting dedicated AI cluster can have up to 50 model endpoints. On the dedicated AI cluster details page, Remaining endpoint capacity shows how many additional model endpoints the cluster can support under its current limit. This count is separate from the model replicas and dedicated AI units that provide the cluster's inference capacity.
To create more than 50 model endpoints on a hosting cluster, request a service limit increase for
endpoint-per-dedicated-unit-count. See Endpoint Capacity.
Project Limits
In a tenancy, you can have up to 50 projects in OCI Generative AI.
Vector Store Connector Limits
In a tenancy, you can have up to 5 active connectors per vector store.
Application Limits
The following table list the limits for applications in OCI Generative AI.
| Limit | Default | Maximum | Can request a service limit increase | Limit name |
|---|---|---|---|---|
| Applications per tenancy | 10 | 50 | Yes | hosted-application-count |
| Artifacts per application | 20 | 50 | Yes | artifacts-per-application-count |
| Managed storage options per application | 3 | 3 | No | |
| Environment variables per application | 20 | 100 | Yes | environment-variables-per-application-count |
| Maximum replicas per application | 30 | 50 | Yes | max-replicas-per-application-count |
| PE/RCE DNS proxies per tenancy | 3 | 10 | Yes | pe-rce-dns-proxy-count |
| Managed storage systems per storage type per tenancy | 3 | 10 | Yes | managed-storage-per-type-count |