Scenario A: Automatically Resizing VMs

Set up automatic resizing for virtual machines (VMs) that exceed memory by using Notifications, Functions, and Monitoring services.

This scenario involves writing a function to resize VMs and creating an alarm  that sends a message to that function. When the alarm fires, the Notifications service sends the alarm message to the destination topic, which then fans out to the topic's subscriptions. In this scenario, the topic's subscriptions include the function as well as your email address and an SMS phone number. The function is invoked on receipt of the alarm message.

Note

The Notifications service has no information about a function after it's invoked. For details, see the troubleshooting information in Function Not Invoked or Run.

This image shows Notifications in the context of a scenario that uses a function to resize VMs.

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don't have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  to work in.

If you're a member of the Administrators group, you already have the required access to execute this scenario. Otherwise, you need access to Monitoring, Notifications, and Functions. You must have FN_INVOCATION permission against the function to be able to add the function as a subscription to a topic. To resize VMs, the function must be authorized to update compute instances. To authorize your function for access to other Oracle Cloud Infrastructure resources, such as compute instances, include the function in a dynamic group  and create a policy to grant the dynamic group access to those resources. For more information, see Accessing Other Oracle Cloud Infrastructure Resources from Running Functions.

Task 1: Create and Authorize Your Function

Once you create your function to resize VMs using your preferred SDK and authorize your function to access VMs (include the function in a dynamic group and grant that dynamic group access), all other scenario steps can be completed in the Console. Alternatively, you can use the Oracle Cloud Infrastructure CLI or API, which lets you execute the individual operations yourself.

For more information about authorizing functions to access other Oracle Cloud Infrastructure resources, see Accessing Other Oracle Cloud Infrastructure Resources from Running Functions.

Function code sample
Note

For this code sample, we recommend handling idempotency via a database.

The following code sample is for a function to resize VMs. For instructions on creating and deploying functions, see Creating and Deploying Functions.

import io
import json
import oci

from fdk import response

def increase_compute_shape(instance_id, alarm_msg_shape):
    signer = oci.auth.signers.get_resource_principals_signer()
    compute_client = oci.core.ComputeClient(config={}, signer=signer)
    current_shape = compute_client.get_instance(instance_id).data.shape
    print("INFO: current shape for Instance {0}: {1}".format(instance_id,current_shape), flush=True)
    if current_shape != alarm_msg_shape:
        return "The shape of Instance {} differs from the Alarm message".format(instance_id)
    # improve the logic below to handle more scenarios, make sure the shapes you select are available in the region and AD
    if  current_shape == "VM.Standard1.1":
        new_shape = "VM.Standard2.1"
    elif current_shape == "VM.Standard2.1":
        new_shape = "VM.Standard2.2"
    else:
        return "Instance {0} cannot get a bigger shape than its current shape {1}".format(instance_id,current_shape)
    print("INFO: new shape for Instance {0}: {1}".format(instance_id,new_shape), flush=True)
    try:
        update_instance_details = oci.core.models.UpdateInstanceDetails(shape=new_shape)
        resp = compute_client.update_instance(instance_id=instance_id, update_instance_details=update_instance_details)
        print(resp, flush=True)
    except Exception as ex:
        print('ERROR: cannot update instance {}'.format(instance_id), flush=True)
        raise
    return "The shape of Instance {} is updated, the instance is rebooting...".format(instance_id)

def handler(ctx, data: io.BytesIO=None):
    alarm_msg = {}
    message_id = func_response = ""
    try:
        headers = ctx.Headers()
        message_id = headers["x-oci-ns-messageid"]
    except Exception as ex:
        print('ERROR: Missing Message ID in the header', ex, flush=True)
        raise
    print("INFO: Message ID = ", message_id, flush=True)
    # the Message Id can be stored in a database and be used to check for duplicate messages
    try:
        alarm_msg = json.loads(data.getvalue())
        print("INFO: Alarm message: ")
        print(alarm_msg, flush=True)
    except (Exception, ValueError) as ex:
        print(str(ex), flush=True)

    if alarm_msg["type"] == "OK_TO_FIRING":
        if alarm_msg["alarmMetaData"][0]["dimensions"]:
            alarm_metric_dimension = alarm_msg["alarmMetaData"][0]["dimensions"][0]   #assuming the first dimension matches the instance to resize
            print("INFO: Instance to resize: ", alarm_metric_dimension["resourceId"], flush=True)
            func_response = increase_compute_shape(alarm_metric_dimension["resourceId"], alarm_metric_dimension["shape"])
            print("INFO: ", func_response, flush=True)
        else:
            print('ERROR: There is no metric dimension in this alarm message', flush=True)
            func_response = "There is no metric dimension in this alarm message"
    else:
        print('INFO: Nothing to do, alarm is not FIRING', flush=True)
        func_response = "Nothing to do, alarm is not FIRING"

    return response.Response(
        ctx, 
        response_data=func_response,
        headers={"Content-Type": "application/json"}
    )
Include your function in a dynamic group

Find and note your function OCID  (format is ocid1.fnfunc.oc1.iad.exampleuniqueID), then specify the following rule in the relevant dynamic group :

resource.id = '<function-ocid>'
Create a policy to grant the dynamic group access to VMs (compute instances)

Add the following policy  :

allow dynamic-group <dynamic-group-name> to use instances in tenancy

Task 2: Create the Topic

For help with troubleshooting, see Troubleshooting Notifications.

  • Note

    Another Console workflow for this scenario involves creating a new topic and the first subscription when you create the alarm, then creating additional subscriptions in that topic.
    1. Open the Create Topic panel: On the Topics list page, select Create Topic. If you need help finding the list page, see Listing Topics.
    2. For Name, type the following: Alarm Topic
    3. Select Create.
  • Use the oci ons topic create command and required parameters to create a topic:

    oci ons topic create --name <name> --compartment-id <compartment_OCID>

    Example:

    oci ons topic create --name "Alarm Topic" --compartment-id "<compartment_OCID>"

    For a complete list of parameters and values for CLI commands, see the Command Line Reference for Notifications.

  • Run the CreateTopic operation to create a topic.

    Example:

    POST /20181201/topics
    Host: notification.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "name": "Alarm Topic",
      "compartmentId": "<compartment_OCID>"
    }

Task 3: Create the Subscriptions

Your function must be deployed before creating the function subscription.

For help with troubleshooting, see Troubleshooting Notifications.

    1. Select the topic that you created earlier (example name was Alarm Topic): On the Topics list page, select the topic that you want to work with. If you need help finding the list page or the topic, see Listing Topics.
    2. Create the function subscription.
      1. Open the Create Subscription panel: In the detail page for the topic, select Create Subscription.
        The Create Subscription panel opens.
      2. For Protocol, select Function.
      3. Fill in the remaining fields.
        Field Description
        Function Compartment Select the compartment containing the function.
        Function Application Select the application containing the function.
        Function Select the function.
      4. Select Create.
        No confirmation is needed for new function subscriptions.
    3. Create the SMS subscription.
      1. Open the Create Subscription panel: In the detail page for the topic, select Create Subscription.
        The Create Subscription panel opens.
      2. For Protocol, select SMS.
      3. Fill in the remaining fields.
        Field Description
        Country Select the country for the phone number. See Before You Begin.
        Phone Number Enter the phone number, using E.164 format.
      4. Select Create.
      5. Confirm the new SMS subscription: Follow the instructions received on the phone.
    4. Create the email subscription.
      1. Open the Create Subscription panel: In the detail page for the topic, select Create Subscription.
        The Create Subscription panel opens.
      2. For Protocol, select Email.
      3. Fill in the remaining fields.
        Field Description
        Email Type an email address.
      4. Select Create.
      5. Confirm the new email subscription: Open the email and navigate to the confirmation URL.
  • Note

    After creating the SMS and email subscriptions, confirm them.

    Use the oci ons subscription create command and required parameters to create each subscription:

    oci ons subscription create --protocol <subscription_type> --subscription-endpoint <endpoint> --compartment-id <compartment_OCID> --topic-id <topic_OCID>

    Function subscription example:

    oci ons subscription create --protocol "ORACLE_FUNCTIONS" --subscription-endpoint "<function-ocid>" --compartment-id "<compartment_OCID>" --topic-id "<topic_OCID>"

    SMS subscription example:

    oci ons subscription create --protocol "SMS" --subscription-endpoint "<sms-endpoint>" --compartment-id "<compartment_OCID>" --topic-id "<topic_OCID>"

    Email subscription example:

    oci ons subscription create --protocol "EMAIL" --subscription-endpoint "john.smith@example.com" --compartment-id "<compartment_OCID>" --topic-id "<topic_OCID>"

    For a complete list of parameters and values for CLI commands, see the Command Line Reference for Notifications.

  • Note

    After creating the SMS and email subscriptions, confirm them.

    Run the CreateSubscription operation to create each subscription.

    Function subscription example:

    POST /20181201/subscriptions
    Host: notification.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "topicId": "<topic_OCID>",
      "compartmentId": "<compartment_OCID>",
      "protocol": "ORACLE_FUNCTIONS",
      "endpoint": "<function_OCID>"
    }

    SMS subscription example:

    POST /20181201/subscriptions
    Host: notification.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "topicId": "<topic_OCID>",
      "compartmentId": "<compartment_OCID>",
      "protocol": "SMS",
      "endpoint": "<sms-endpoint>"
    }

    Email subscription example:

    POST /20181201/subscriptions
    Host: notification.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "topicId": "<topic_OCID>",
      "compartmentId": "<compartment_OCID>",
      "protocol": "EMAIL",
      "endpoint": "john.smith@example.com"
    }

Task 4: Create the Alarm

For help with troubleshooting, see Troubleshooting Notifications.

    1. Open the Create Alarm page.
      1. Open the navigation menu and click Observability & Management. Under Monitoring, click Alarm Definitions.
      2. Select Create Alarm.

    2. For Alarm name, type the following: VM Memory Alarm
    3. Under Metric description, select the metric, interval, and statistic.

      Field Example value for this scenario
      Compartment Select the compartment  that contains the VM that you want to automatically resize.
      Metric namespace oci_computeagent
      Metric name MemoryUtilization
      Interval 1m
      Statistic Max
    4. Under Trigger rule, set up the alarm threshold.

      Field Example value for this scenario
      Operator greater than
      Value 90
      Trigger delay minutes 1
    5. Under Notifications, Destinations, select the topic that you created earlier.
      Field Example value for this scenario
      Destination Service Notifications Service
      Compartment Select the compartment  that contains the topic that you created earlier.
      Topic Select the topic that you created earlier.
    6. Select Save alarm.

  • Use the oci monitoring alarm create command and required parameters to create an alarm:

    oci monitoring alarm create --display-name <name> --compartment-id <compartment_OCID> --metric-compartment-id <compartment_OCID> --namespace <metric_namespace> --query-text <mql_expression> --severity <level> --destinations <file_or_text> --is-enabled <true_or_false>

    Example:

    oci monitoring alarm create --display-name "VM Memory Alarm" --compartment-id "<compartment_OCID>" --metric-compartment-id "<compartment_OCID>" --namespace "oci_computeagent" --query-text "MemoryUtilization[1m].max() > 90" --severity "CRITICAL" --destinations "<topic-ocid>" --is-enabled true

    For a complete list of parameters and values for CLI commands, see the Command Line Reference for Monitoring.

  • Run the CreateAlarm operation to create an alarm.

    Example:

    POST /20180401/alarms
    Host: telemetry.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "displayName": "VM Memory Alarm",
      "compartmentId": "<compartment_OCID>",
      "metricCompartmentId": "<compartment_OCID>",
      "namespace": "oci_computeagent",
      "query": "MemoryUtilization[1m].max() > 90",
      "severity": "CRITICAL",
      "destinations":
      [
        "<topic_OCID>"
      ],
      "isEnabled": true
    }