Scenario A: Automatically Resizing VMs
Set up automatic resizing for virtual machines (VMs) that exceed memory by using Notifications, Functions, and Monitoring services.
This scenario involves writing a function to resize VMs and creating an alarm that sends a message to that function. When the alarm fires, the Notifications service sends the alarm message to the destination topic, which then fans out to the topic's subscriptions. In this scenario, the topic's subscriptions include the function as well as your email address and an SMS phone number. The function is invoked on receipt of the alarm message.
The Notifications service has no information about a function after it's invoked. For details, see the troubleshooting information in Function Not Invoked or Run.
Required IAM Policy
To use Oracle Cloud Infrastructure, you must be granted security access in a policy by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don't have permission or are unauthorized, verify with your administrator what type of access you have and which compartment to work in.
If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring, Notifications, and Functions. You must have FN_INVOCATION
permission against the function to be able to add the function as a subscription to a topic. To resize VMs, the function must be authorized to update compute instances. To authorize your function for access to other Oracle Cloud Infrastructure resources, such as compute instances, include the function in a dynamic group and create a policy to grant the dynamic group access to those resources. For more information, see Accessing Other Oracle Cloud Infrastructure Resources from Running Functions.
Task 1: Create and Authorize Your Function
Once you create your function to resize VMs using your preferred SDK and authorize your function to access VMs (include the function in a dynamic group and grant that dynamic group access), all other scenario steps can be completed in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.
For more information about authorizing functions to access other Oracle Cloud Infrastructure resources, see Accessing Other Oracle Cloud Infrastructure Resources from Running Functions.
For this code sample, we recommend handling idempotency via a database.
The following code sample is for a function to resize VMs. For instructions on creating and deploying functions, see Creating and Deploying Functions.
import io
import json
import oci
from fdk import response
def increase_compute_shape(instance_id, alarm_msg_shape):
signer = oci.auth.signers.get_resource_principals_signer()
compute_client = oci.core.ComputeClient(config={}, signer=signer)
current_shape = compute_client.get_instance(instance_id).data.shape
print("INFO: current shape for Instance {0}: {1}".format(instance_id,current_shape), flush=True)
if current_shape != alarm_msg_shape:
return "The shape of Instance {} differs from the Alarm message".format(instance_id)
# improve the logic below to handle more scenarios, make sure the shapes you select are available in the region and AD
if current_shape == "VM.Standard1.1":
new_shape = "VM.Standard2.1"
elif current_shape == "VM.Standard2.1":
new_shape = "VM.Standard2.2"
else:
return "Instance {0} cannot get a bigger shape than its current shape {1}".format(instance_id,current_shape)
print("INFO: new shape for Instance {0}: {1}".format(instance_id,new_shape), flush=True)
try:
update_instance_details = oci.core.models.UpdateInstanceDetails(shape=new_shape)
resp = compute_client.update_instance(instance_id=instance_id, update_instance_details=update_instance_details)
print(resp, flush=True)
except Exception as ex:
print('ERROR: cannot update instance {}'.format(instance_id), flush=True)
raise
return "The shape of Instance {} is updated, the instance is rebooting...".format(instance_id)
def handler(ctx, data: io.BytesIO=None):
alarm_msg = {}
message_id = func_response = ""
try:
headers = ctx.Headers()
message_id = headers["x-oci-ns-messageid"]
except Exception as ex:
print('ERROR: Missing Message ID in the header', ex, flush=True)
raise
print("INFO: Message ID = ", message_id, flush=True)
# the Message Id can be stored in a database and be used to check for duplicate messages
try:
alarm_msg = json.loads(data.getvalue())
print("INFO: Alarm message: ")
print(alarm_msg, flush=True)
except (Exception, ValueError) as ex:
print(str(ex), flush=True)
if alarm_msg["type"] == "OK_TO_FIRING":
if alarm_msg["alarmMetaData"][0]["dimensions"]:
alarm_metric_dimension = alarm_msg["alarmMetaData"][0]["dimensions"][0] #assuming the first dimension matches the instance to resize
print("INFO: Instance to resize: ", alarm_metric_dimension["resourceId"], flush=True)
func_response = increase_compute_shape(alarm_metric_dimension["resourceId"], alarm_metric_dimension["shape"])
print("INFO: ", func_response, flush=True)
else:
print('ERROR: There is no metric dimension in this alarm message', flush=True)
func_response = "There is no metric dimension in this alarm message"
else:
print('INFO: Nothing to do, alarm is not FIRING', flush=True)
func_response = "Nothing to do, alarm is not FIRING"
return response.Response(
ctx,
response_data=func_response,
headers={"Content-Type": "application/json"}
)
Find and note your function OCID (format is ocid1.fnfunc.oc1.iad.exampleuniqueID
), then specify the following rule in the relevant dynamic group :
resource.id = '<function-ocid>'
Add the following policy :
allow dynamic-group <dynamic-group-name> to use instances in tenancy
Task 2: Create the Topic
For help with troubleshooting, see Troubleshooting Notifications.
- Note
Another Console workflow for this scenario involves creating a new topic and the first subscription when you create the alarm, then creating additional subscriptions in that topic.- Open the Create Topic panel: On the Topics list page, select Create Topic. If you need help finding the list page, see Listing Topics.
- For Name, type the following: Alarm Topic
- Select Create.
Use the oci ons topic create command and required parameters to create a topic:
oci ons topic create --name <name> --compartment-id <compartment_OCID>
Example:
oci ons topic create --name "Alarm Topic" --compartment-id "<compartment_OCID>"
For a complete list of parameters and values for CLI commands, see the Command Line Reference for Notifications.
Run the CreateTopic operation to create a topic.
Example:
POST /20181201/topics Host: notification.us-phoenix-1.oraclecloud.com <authorization and other headers> { "name": "Alarm Topic", "compartmentId": "<compartment_OCID>" }
Task 3: Create the Subscriptions
Your function must be deployed before creating the function subscription.
For help with troubleshooting, see Troubleshooting Notifications.
- Select the topic that you created earlier (example name was Alarm Topic): On the Topics list page, select the topic that you want to work with. If you need help finding the list page or the topic, see Listing Topics.
-
Create the function subscription.
-
Create the SMS subscription.
-
Create the email subscription.
Use the oci ons subscription create command and required parameters to create each subscription:
oci ons subscription create --protocol <subscription_type> --subscription-endpoint <endpoint> --compartment-id <compartment_OCID> --topic-id <topic_OCID>
Function subscription example:
oci ons subscription create --protocol "ORACLE_FUNCTIONS" --subscription-endpoint "<function-ocid>" --compartment-id "<compartment_OCID>" --topic-id "<topic_OCID>"
SMS subscription example:
oci ons subscription create --protocol "SMS" --subscription-endpoint "<sms-endpoint>" --compartment-id "<compartment_OCID>" --topic-id "<topic_OCID>"
Email subscription example:
oci ons subscription create --protocol "EMAIL" --subscription-endpoint "john.smith@example.com" --compartment-id "<compartment_OCID>" --topic-id "<topic_OCID>"
For a complete list of parameters and values for CLI commands, see the Command Line Reference for Notifications.
Run the CreateSubscription operation to create each subscription.
Function subscription example:
POST /20181201/subscriptions Host: notification.us-phoenix-1.oraclecloud.com <authorization and other headers> { "topicId": "<topic_OCID>", "compartmentId": "<compartment_OCID>", "protocol": "ORACLE_FUNCTIONS", "endpoint": "<function_OCID>" }
SMS subscription example:
POST /20181201/subscriptions Host: notification.us-phoenix-1.oraclecloud.com <authorization and other headers> { "topicId": "<topic_OCID>", "compartmentId": "<compartment_OCID>", "protocol": "SMS", "endpoint": "<sms-endpoint>" }
Email subscription example:
POST /20181201/subscriptions Host: notification.us-phoenix-1.oraclecloud.com <authorization and other headers> { "topicId": "<topic_OCID>", "compartmentId": "<compartment_OCID>", "protocol": "EMAIL", "endpoint": "john.smith@example.com" }
Task 4: Create the Alarm
For help with troubleshooting, see Troubleshooting Notifications.
- Open the Create Alarm page.
- Open the navigation menu and click Observability & Management. Under Monitoring, click Alarm Definitions.
-
Select Create Alarm.
- For Alarm name, type the following: VM Memory Alarm
-
Under Metric description, select the metric, interval, and statistic.
Field Example value for this scenario Compartment Select the compartment that contains the VM that you want to automatically resize. Metric namespace oci_computeagent Metric name MemoryUtilization Interval 1m Statistic Max -
Under Trigger rule, set up the alarm threshold.
Field Example value for this scenario Operator greater than Value 90 Trigger delay minutes 1 - Under Notifications, Destinations, select the topic that you created earlier.
Field Example value for this scenario Destination Service Notifications Service Compartment Select the compartment that contains the topic that you created earlier. Topic Select the topic that you created earlier. -
Select Save alarm.
- Open the Create Alarm page.
Use the oci monitoring alarm create command and required parameters to create an alarm:
oci monitoring alarm create --display-name <name> --compartment-id <compartment_OCID> --metric-compartment-id <compartment_OCID> --namespace <metric_namespace> --query-text <mql_expression> --severity <level> --destinations <file_or_text> --is-enabled <true_or_false>
Example:
oci monitoring alarm create --display-name "VM Memory Alarm" --compartment-id "<compartment_OCID>" --metric-compartment-id "<compartment_OCID>" --namespace "oci_computeagent" --query-text "MemoryUtilization[1m].max() > 90" --severity "CRITICAL" --destinations "<topic-ocid>" --is-enabled true
For a complete list of parameters and values for CLI commands, see the Command Line Reference for Monitoring.
Run the CreateAlarm operation to create an alarm.
Example:
POST /20180401/alarms Host: telemetry.us-phoenix-1.oraclecloud.com <authorization and other headers> { "displayName": "VM Memory Alarm", "compartmentId": "<compartment_OCID>", "metricCompartmentId": "<compartment_OCID>", "namespace": "oci_computeagent", "query": "MemoryUtilization[1m].max() > 90", "severity": "CRITICAL", "destinations": [ "<topic_OCID>" ], "isEnabled": true }