Data Science Model Deployment and Model Catalog services now support large model deployments.

Large model artifacts can be stored in the model catalog service and be used to create model deployments. The endpoint mapping feature lets you integrate inference containers, such as the Text Generation Interface (TGI), even if they don't comply with standard API contracts for the /predict and /health endpoints.

 

For more information, see the Data Science docuemntation on Large Models.