Default Networking

Create a model deployment by using the default networking configuration. You can choose from two modes: one with internet access enabled, and one with internet access disabled.

The workload connects by using a secondary VNIC to a preconfigured, service-managed virtual cloud network (VCN) and subnet. The provided subnet gives access to other Oracle Cloud services through a service gateway but not to the public internet. If you enable Default networking with internet, the subnet also provides an optional path to the public internet through an OCI-managed NAT gateway.

If you need access only to OCI services, we recommend using this option. It doesn't require you to create networking resources or write policies for networking permissions.

You can create and run default networking model deployments using the Console, the OCI Python SDK, the OCI CLI, or the Data Science API.

1. From the model deployments page, select Create model deployment. If you need help finding the list of model deployments, see Listing Model Deployments.
2. (Optional) Enter a unique name for the model (limit of 255 characters). If you don't provide a name, a name is automatically generated.
  
  For example, modeldeployment20200108222435.
3. (Optional) Enter a description (limit of 400 characters) for the model deployment.
4. (Optional) Under Default configuration, enter a custom environment variable key and corresponding value. Select + Additional custom environment key to add more environment variables.
5. In the Models section, select Select to select an active model to deploy from the model catalog.
  
  Find a model by using the default compartment and project, or by selecting Using OCID and searching for the model by entering its OCID.
  
  Select the model.
  
  Select Submit.
  
  Important
  
  Model artifacts that exceed 400 GB aren't supported for deployment. Select a smaller model artifact for deployment.
6. (Optional) Change the Compute shape by selecting Change shape. Then, follow these steps in the Select compute panel.
  
  Select an instance type.
  
  Select a shape series.
  
  Select one of the supported Compute shapes in the series.
  
  Select the shape that best suits how you want to use the resource.
  
  For each OCPU, select up to 64 GB of memory and a maximum total of 512 GB. The minimum amount of memory allowed is either 1 GB or a value matching the number of OCPUs, whichever is greater.
  
  If using burstable VMs, toggle Burstable.
  In Baseline utilization per OCPU select the percentage of OCPUs that you usually want to use. The supported values are 12.5% and 50%.
  
  Select Select shape.
7. Enter the number of instances for the model deployment to replicate the model on.
8. Select one of the following Networking resources:
  
  Default networking: Restricts traffic to Oracle services only.
  
  Default networking with internet: Allows outbound internet access through the Data Science NAT gateway.
  Note
  
  You cannot use Default networking with internet in disconnected realms and Oracle development tenancies.
  If your tenancy or compartment has a Data Science security zone policy that denies public network access (for example, deny model_deploy_public_network—see Data Science security zone policy), the service-managed public internet access option is disabled. If you try to use this option, you receive a 404 NotAuthorizedOrNotFound error.
9. Select one of the following options to configure the endpoint type:
  
  Public endpoint: Enables data access to a managed instance from outside a virtual cloud network (VCN).
  
  Private endpoint: Specifies the private endpoint to use for the model deployment.
  Deployments that use private networking or private endpoints cannot enable service-managed public internet access.
  
  If you select Private endpoint, select Private Endpoint from Private Endpoint in Data Science.
  Select Change compartment to select the compartment that contains the private endpoint.
10. (Optional) If you configured access or predict logging, in the Logging section, select Select and then follow these steps:
  
  For access logs, select a compartment, log group, and log name.
  
  For predict logs, select a compartment, log group, and log name.
  
  Select Submit.
11. (Optional) Select Show Advanced Options to add tags.
  
  (Optional) Select the serving mode for the model deployment, either as an HTTPS endpoint or using a Streaming service stream.
  
  (Optional) Select the load balancing bandwidth in Mbps or use the 10 Mbps default.
  
  Tips for load balancing:
  
  If you know the common payload size and the frequency of requests per second, you can use the following formula to estimate the bandwidth of the load balancer that you need. We recommend that you add an extra 20% to account for estimation errors and sporadic peak traffic.
  
  (Payload size in KB) * (Estimated requests per second) * 8 / 1024
  
  For example, if the payload is 1,024 KB and you estimate 120 requests per second, then the recommended load balancer bandwidth would be (1024 * 120 * 8 / 1024) * 1.2 = 1152 Mbps.
  
  Remember that the maximum supported payload size is 10 MB when dealing with image payloads.
  
  If the request payload size is more than the allocated bandwidth of the load balancer that was defined, then the request is rejected with a 429 status code.
  
  (Optional) Select Use a Custom Container Image and enter the following:
  
  Repository in <tenancy>: Enter the repository that contains the custom image.
  
  Image: Enter the custom image to use at runtime.
  
  Image digest: Enter the image digest. For example: sha256:<digest>. The digest must match the exact image you are deploying.
  
  Image Signature Id: Enter the image signature OCID that is generated when you sign the image. For more information about the signing process, see Signing Images for Security.
  The signature entry is optional, unless your tenancy or policy enforces image signature verification for custom images. If verification is enforced, you must provide all required signature fields. Otherwise, deployment will not proceed.
  If verification fails, the Console shows an error such as "Signature <ID> is not verified." or "Digest of the image provided does not match the digest on the signature." In this case, model deployment creation is blocked.
  
  CMD: Enter commands for the container, one command per text box. For example, enter --host in one text-box and 0.0.0.0 in another. Do not use quotation marks at the end.
  
  Entrypoint: Enter one or more entry point files to run when the container starts, such as /opt/script/entrypoint.sh. Do not use quotation marks at the end.
  
  Server port: Enter the port for the inference web server. The default is 8080. Valid range: 1024–65535, except 24224, 8446, 8447.
  
  Health check port: Enter the port for the container health check. The default is the server port. Valid range: 1024–65535, except 24224, 8446, 8447.
  
  (Optional) In the Tags section, add one or more tags to the model deployment. If you have permissions to create a resource, then you also have permissions to apply free-form tags to that resource. To apply a defined tag, you must have permissions to use the tag namespace. For more information about tagging, see Resource Tags. If you're not sure whether to apply tags, skip this option or ask an administrator. You can apply tags later.
12. Select Create.

You can use the OCI CLI to create a model deployment as in this example.

Deploy the model with:

oci data-science model-deployment create \
--compartment-id <MODEL_DEPLOYMENT_COMPARTMENT_OCID> \
--model-deployment-configuration-details file://<MODEL_DEPLOYMENT_CONFIGURATION_FILE> \
--project-id <PROJECT_OCID> \
--category-log-details file://<OPTIONAL_LOGGING_CONFIGURATION_FILE> \
--display-name <MODEL_DEPLOYMENT_NAME>

Use this model deployment JSON configuration file:

{
      "deploymentType": "SINGLE_MODEL",
      "modelConfigurationDetails": {
        "bandwidthMbps": <YOUR_BANDWIDTH_SELECTION>,
        "instanceConfiguration": {
          "instanceShapeName": "<YOUR_VM_SHAPE>"
        },
        "modelId": "<YOUR_MODEL_OCID>",
        "scalingPolicy": {
            "instanceCount": <YOUR_INSTANCE_COUNT>,
            "policyType": "FIXED_SIZE"
         }
     }
 }

If you are specifying an environment configuration, you must include the environmentConfigurationDetails object as in this example:


{
  "modelDeploymentConfigurationDetails": {
    "deploymentType": "SINGLE_MODEL",
    "modelConfigurationDetails": {
      "modelId": "ocid1.datasciencemodel.oc1.iad........",
      "instanceConfiguration": {
        "instanceShapeName": "VM.Standard.E4.Flex",
        "modelDeploymentInstanceShapeConfigDetails": {
          "ocpus": 1,
          "memoryInGBs": 16
        }
      },
      "scalingPolicy": {
        "policyType": "FIXED_SIZE",
        "instanceCount": 1
      },
      "bandwidthMbps": 10
    },
    "environmentConfigurationDetails" : {
      "environmentConfigurationType": "OCIR_CONTAINER",
      "image": "iad.ocir.io/testtenancy/image_name:1.0.0",
      "imageDigest": "sha256:a9c8468cb671929aec7ad947b9dccd6fe8e6d77f7bcecfe2e10e1c935a88c2a5",
      "imageSignatureId": "ocid1.containerimagesignature.oc1.iad.0.ociodscprod.aaaaaaaavkjvrldo4etdpdas3o5vuom3t6anoixneey737cr57if7jhkh6nq",
      "entrypoint": [
        "python",
        "/opt/entrypoint.py"
      ],
      "serverPort": "5000",
      "healthCheckPort": "5000"
    },
    "streamConfigurationDetails": {
      "inputStreamIds": null,
      "outputStreamIds": null
    }
  }
}

Note

The imageSignatureId field is optional.

(Optional) Use this logging JSON configuration file:

{
    "access": {
      "logGroupId": "<YOUR_LOG_GROUP_OCID>",
      "logId": "<YOUR_LOG_OCID>"
    },
    "predict": {
      "logGroupId": "<YOUR_LOG_GROUP_OCID>",
      "logId": "<YOUR_LOG_OCID>"
    }
}

(Optional) Use this to use a custom container:

oci data-science model-deployment create \
--compartment-id <MODEL_DEPLOYMENT_COMPARTMENT_OCID> \
--model-deployment-configuration-details file://<MODEL_DEPLOYMENT_CONFIGURATION_FILE> \
--project-id <PROJECT_OCID> \
--category-log-details file://<OPTIONAL_LOGGING_CONFIGURATION_FILE> \
--display-name <MODEL_DEPLOYMENT_NAME>

Use the CreateModelDeployment operation to create a model deployment.

Using the OCI Python SDK

We've developed an OCI Python SDK model deployment example that includes authentication.

Important

Artifacts that exceed 400 GB aren't supported for deployment. Select a smaller model artifact for deployment.

Note

You must upgrade the OCI SDK to version 2.33.0 or later before creating a deployment with the Python SDK. Use the following command:

pip install --upgrade oci

Use this example to create a model deployment that uses a custom container:

# create a model configuration details object
model_config_details = ModelConfigurationDetails(
    model_id=<model-id>,
    bandwidth_mbps=<bandwidth-mbps>,
    instance_configuration=<instance-configuration>,
    scaling_policy=<scaling-policy>
)
 
# create the container environment configiguration
environment_config_details = OcirModelDeploymentEnvironmentConfigurationDetails(
    environment_configuration_type="OCIR_CONTAINER",
    environment_variables={'key1': 'value1', 'key2': 'value2'},
    image="iad.ocir.io/testtenancy/ml_flask_app_demo:1.0.0",
    image_digest="sha256:243590ea099af4019b6afc104b8a70b9552f0b001b37d0442f8b5a399244681c",
    entrypoint=[
        "python",
        "/opt/ds/model/deployed_model/api.py"
    ],
    server_port=5000,
    health_check_port=5000
)
 
# create a model type deployment
single_model_deployment_config_details = data_science.models.SingleModelDeploymentConfigurationDetails(
    deployment_type="SINGLE_MODEL",
    model_configuration_details=model_config_details,
    environment_configuration_details=environment_config_details
)
 
# set up parameters required to create a new model deployment.
create_model_deployment_details = CreateModelDeploymentDetails(
    display_name=<deployment_name>,
    model_deployment_configuration_details=single_model_deployment_config_details,
    compartment_id=<compartment-id>,
    project_id=<project-id>
)

Notebook Examples

We have provided various notebook examples that show you how to train, prepare, save, deploy, and invoke model deployments.

Oracle Cloud Infrastructure Documentation

Default Networking

Using the OCI Python SDK

Notebook Examples