The runtime.yaml File

The runtime.yaml file defines the runtime configuration required to deploy a model using the Model Deployment feature of the Data Science service.

A model artifact must include both:

score.py
runtime.yaml

The runtime.yaml file specifies the Conda environment used during inference, ensuring that the deployed model runs with the correct runtime dependencies.

Several Model Catalog examples include sample runtime.yaml files.

We've provided various model catalog examples including runtime.yaml files.

Supported Fields

Following are the fields supported in runtime.yaml.

As the model artifact structure evolves, the runtime.yaml schema might also change. Supported fields correspond to the current MODEL_ARTIFACT_VERSION (3.0).

Important

Additionally, ensure that INFERENCE_PYTHON_VERSION and INFERENCE_ENV_PATH are compatible with each other to prevent deployment errors.


Field	Required	Description
`MODEL_ARTIFACT_VERSION`	Yes	The version of this artifact format. This version is 3.0. This is automatically extracted by ADS when the model is saved in a notebook session.
`MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_ENV_SLUG`	Yes	The slug of the conda environment you want to use for deployment and scoring purposes. Usually, the inference environment is the same as the training environment thought that doesn't have to be the case. For example, you can train a model in one conda environment, serialize it as a general ML model, and then use a general ML conda environment to deploy it.
`MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_ENV_TYPE`	Yes	The type of conda environment you want to use for deployment and scoring purposes. Two possible values, either `data_science` or `published`.
`MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_ENV_PATH`	Yes	The path on Object Storage of the conda environment you want to use for deployment and scoring purposes. The path uses the following syntax: `oci://<bucket-name>@<namespace>/<file-path>` Note: If the inference Conda environment fields aren't specified, the service automatically falls back to a default Conda environment managed by the service team. The default `INFERENCE_ENV_PATH` runtime uses the following: `AMD (x86_64):service_pack/cpu/General_Machine_Learning_for_CPUs_on_Python_3.12/1.0/generalml_p312_cpu_x86_64_v1` `ARM (aarch64):service_pack/cpu/ARM_Pack_for_Machine_Learning_on_Python_3.12/1.0/armml_p312_cpu_aarch64_v1` ` Fallback to default setting may lead to deployment failure `
`MODEL_DEPLOYMENT.INFERENCE_CONDA_ENV.INFERENCE_PYTHON_VERSION`	Yes	The Python version of the conda environment you want to use for model deployment. The supported Python versions are `3.10`, `3.11` and `3.12`. Note: If the `INFERENCE_PYTHON_VERSION` field isn't specified, the service automatically falls back to a default Python version managed by the service team. The default `INFERENCE_PYTHON_VERSION` runtime uses Python 3.12. ` Fallback to default setting may lead to deployment failure `

Default Behavior and Compatibility Requirements

If one or more inference Conda environment fields aren't specified, the service falls back to a service-managed default runtime.

Default INFERENCE_ENV_PATH values:

AMD (x86_64): service_pack/cpu/General_Machine_Learning_for_CPUs_on_Python_3.12/1.0/generalml_p312_cpu_x86_64_v1
ARM (aarch64): service_pack/cpu/ARM_Pack_for_Machine_Learning_on_Python_3.12/1.0/armml_p312_cpu_aarch64_v1

The default INFERENCE_PYTHON_VERSION is Python 3.12.

Fallback to default settings can still cause model-deployment creation to fail. Ensure that INFERENCE_ENV_PATH and INFERENCE_PYTHON_VERSION reference compatible environment versions.

Schema

Following is the schema of the runtime.yaml file.

{
    "title": "Model Artifact schema",
    "type": "object",
    "properties": {
        "MODEL_ARTIFACT_VERSION": {"type":"string"},
        "MODEL_DEPLOYMENT": {
            "type": "object",
            "title": "Model Deployment",
            "properties": {
                "INFERENCE_CONDA_ENV": {
                    "type": "object",
                    "title": "Inference Conda Env",
                    "properties": {
                        "INFERENCE_ENV_SLUG": {"type":"string"},
                        "INFERENCE_ENV_TYPE": {"type":"string", "enum": ["published", "data_science"]},
                        "INFERENCE_ENV_PATH": {"type":"string"},
                        "INFERENCE_PYTHON_VERSION": {"type":"string"}
                    },
                    "required": [
                        "INFERENCE_ENV_SLUG",
                        "INFERENCE_ENV_TYPE",
                        "INFERENCE_ENV_PATH",
                        "INFERENCE_PYTHON_VERSION"
                    ]
                }
            }
        }
    } 
}

Example File for Model Deployment

Following is an example runtime.yaml file for a Data Science model deployment resource:

MODEL_ARTIFACT_VERSION: '3.0' 
MODEL_DEPLOYMENT: 
	INFERENCE_CONDA_ENV: 
		INFERENCE_ENV_SLUG: envslug 
		INFERENCE_ENV_TYPE: published 
		INFERENCE_ENV_PATH: oci://<bucket-name>@<namespace>/<prefix>/<env>.tar.gz 
		INFERENCE_PYTHON_VERSION: '3.12'

Other Artifact Files

In addition to score.py and runtime.yaml, you can include any more files that are necessary to run your model in your artifact. These might include:

A serialized representation of your estimator object. For example, onnx, pkl, hdf5, or json.
A CSV file, a lookup data table, and so on.
Extra Python modules that are imported in score.py.
Important

For extra Python modules that are imported in score.py, zip any code used for inference at the same level as score.py (or in a child directory). If any required files are present at directory levels higher than the score.py file, then they're ignored and could result in deployment failure.

If you're saving your model using the Oracle Cloud Console, CLI, or SDKs, just zip the files along with the score.py and runtime.yaml files.

If you're using ADS to save your model, then copy all the files in the directory where the artifact is temporarily created before saving the model to the catalog. ADS compresses the entire artifact directory and sends it to the model catalog.

Note

The artifact now supports uncompressed models up to 6 GB when the model is saved using ADS, the CLI, or SDKs. The limit is 100 MB when using Oracle Cloud Console.