Managing Routing Profiles
Routing profiles define how OCI Generative AI routes on-demand model requests across user-selected OCI regions.
A routing profile specifies one supported on-demand model and one or more target OCI regions within the same realm where that model is available. When you reference the routing profile in a supported inference request, Generative AI routes the request only among the target regions in the profile.
Use routing profiles to access a selected model across an approved regional scope without building application-side cross-region routing. For the regional availability of each model, see Generative AI Models by Region.
In CPU-only regions, models aren't hosted locally. Use a routing profile to route supported on-demand model requests to GPU regions. Chat Completions and Responses APIs are supported when the selected model supports the API.
Routing profiles also let you access a model that isn't hosted in your current region. Select the model and its available target regions when creating the profile, then select the profile in your region's Playground or reference it in a supported inference API request. Inference through routing profiles uses on-demand mode.
You can perform the following tasks to create and manage routing profiles: