Model Deployment Metrics
Learn about how to use metrics for model deployments.
Metrics are automatically available for any Data Science model deployments that you create in the oci_datascience_modeldeploy
namespace. You don't need to enable monitoring on OCI resources to get these metrics.
Data Science model deployments metrics include these dimensions:
- resourceId
-
The OCID of the model deployment.
- statusCode
-
The HTTP response status code.
- result
-
Result of response:
-
Success
-
Failure
-
- statusFamily
-
Status family of result:
-
Success: 2XX
-
Failure: 4XX and 5XX
-
- instanceId
-
The ID of instance.
- networkType
-
Types of network:
BytesIn: Network receive throughput
-
BytesOut: Network transmission throughput
Metric Names | Display Name | Unit | Description | Dimensions |
---|---|---|---|---|
PredictRequestCount |
Predict Request Count | Count | Number of predict requests. |
|
PredictResponse |
Predict Response Success Rate | Percentage | Predict response success rate. It's calculated based on the number of successful predict requests out of total number of predict requests. |
|
Predict Response Status | Count | Predict response result and status code. | ||
PredictLatency |
Predict Latency | Milliseconds | Latency of predict calls. |
|
PredictBandwidth |
Predict Bandwidth Utilization | Percentage |
Predict provisioned and consumed bandwidth. Provisioned bandwidth is the customer expected bandwidth set during the model deployment creation. Consumed bandwidth is the active bandwidth consumed by all ongoing predict requests for a specific model deployment OCID. This bandwidth is computed as current consumed bandwidth relative to the total provisioned load balancer bandwidth on every predict request. |
|
CpuUtilization |
CPU Utilization | Percentage | Activity level from the CPU. |
|
MemoryUtilization |
Memory Utilization | Percentage | Memory in use. |
|
NetworkBytes |
Network Receive and Transmit Bytes | Bytes/sec | Network receive and transmission throughput. |
|
GPUCoreUtilization |
GPU Utilization | Percentage | Shows how busy the GPU devices are in a model deployment cluster. |
|
GPUMemoryUtilization |
GPU Memory Utilization | Percentage | Shows the mean memory consumption of all GPU devices in a model deployment cluster. |
|
Viewing Model Deployment Metrics
You can view the default metric charts using a model deployment details page in the Data Science service.
For more information about the emitted metrics, see model deployment metrics.
If you don't see the metrics data for the model deployment session that you expect, see missing metrics data.