Management Agent Metrics

Overview of the Management Agent Service Metrics

Management Agent provides a unidirectional pipe that allows Oracle Cloud Infrastructure services to interact with the customer VM securely. The agent operates in the simplest of network topologies without requiring any special configuration on the VMs. The agent provides an extensible model for Oracle Cloud Infrastructure services to collect performance metrics and execute work.

Service metrics for the Management Agent resource allow the users to visualize the resource consumption of the agent in terms of CPU, memory and disk.

Prerequisites

IAM policies: To monitor resources, you must be given the required type of access in a policy written by an administrator, whether you're using the Console or the REST API with an SDK, CLI or other tool. The policy must give you access to the monitoring services as well as the resources being monitored. If you try to perform an action and get a message that you don’t have permission or are unauthorized, confirm with your administrator the type of access you've been granted and which compartment you should work in. For more information on user authorizations for monitoring, see the Authentication and Authorization section for the related service: Monitoring or Notifications. For information on policies for management agents, see Set Up Oracle Cloud Infrastructure for Management Agent Service.
The metrics listed on this page are automatically available for any management agent you install. You do not need to enable monitoring on the resource to get these metrics.

Available Metrics: oci_managementagent

The Management Agent service emits metrics to oci_managementagent metric namespace in Monitoring service.

The metrics listed in the following tables are automatically available for management agents that you install. You do not need to enable monitoring to get these metrics. You also can use the Monitoring service to create custom queries.

Dimensions

Management Agent metrics include the following dimensions:

AGENTHOSTNAME: The hostname in which the management agent is installed.
RESOURCEID: The OCID of the management agent resource.

Metrics

Metric Name	Metric Display Name	Unit	Frequency	Description	Dimensions
`agentHealthStatus`	Agent Health Status	Not Applicable	1 minute	Agent is alive and it's communicating with the Monitoring service. The status value is a constant number (1) to represent that the agent is alive and is able to upload to Monitoring service. The absence of data points in Monitoring service implies that the agent cannot communicate with Monitoring service. It should not imply that the agent is down.	`agentHostName` `resourceId`
`diskUsageUsed`	Agent Disk Usage	Megabytes	1 hour	Total amount of disk used under "agent_inst".	`agentHostName` `resourceId`
`diskUsageFree`	Agent Disk Space Available	Megabytes	1 hour	Amount of disk space available on the volume.	`agentHostName` `resourceId`
`diskUsageInstallSize`	Disk Usage for Install	Megabytes	1 hour	Total amount of disk used under agentBaseDir.	`agentHostName` `resourceId`
`uploadBufferPctusedofmaxbuffer`	Percentage of Temporary Storage Used	Percent	15 minutes	Amount of disk space used by the agent to temporarily store content while waiting for Monitoring service to be available. This number is expressed as a percentage of the maximum limit at the agent (Default limit: 100 MB).	`agentHostName` `resourceId`
`usageCpu`	Agent CPU Utilization	Percent	1 minute	Percentage CPU usage of the agent.	`agentHostName` `resourceId`
`usageRss`	Agent Memory Usage	Megabytes	1 minute	Amount of memory consumed by the agent's JVM. This number represents the resident set size in main memory.	`agentHostName` `resourceId`

Note

The alarm interval must be greater than the frequency at which the metrics are emitted. Be sure to set alarm intervals greater than the frequency at which the metrics are emitted.

Set Up Default Alarms Using Marketplace Application

To set up default alarms with pre-defined metrics threshold automatically, you can use the Default Alarms for Management Agent application from Oracle Cloud Infrastructure (OCI) Marketplace. This Marketplace application allows you to configure several out-of-the-box alarms for Management Agents and Logging Analytics metrics automatically.

See Overview of Marketplace for a full description of how to use Marketplace.

Using the Marketplace application to set up default alarms automatically provides the following:

An easy way to automate the configuration and monitoring of several recommended metrics by default.
Create default alarms for all the Management Agents under the tenancy (root compartment) in one step.
Alarm definitions are pre-created before Management Agents are installed.
The usage of Resource Manager and stacks (Terraform configurations) to simplify the set up and configuration (An automatic set up performed by Resource Manager which allows to customize the out-of-the-box alarm configurations and start monitoring immediately).

Default Alarms for Management Agent Marketplace Application

The Marketplace application sets up alarms and their pre-defined thresholds for the following:

Management Agent alarms
- Agent availability.
- CPU usage.
- Disk space.
- JVM memory usage.
Logging Analytics alarms
- Logging analytics upload data size.
- Logging analytics upload failure.

The Marketplace application creates the following resources:

Notification Destination topic (if the create new option selected) without a subscription. This is part of the OCI Notifications service which enables you to set up communication channels for publishing messages using topics and subscriptions. For more information, see Notifications.
Alarm definitions for the selected metrics. By default, four Management Agent metrics alarms (availability, CPU, disk and memory alarms) and two Logging Analytics metrics alarms (logging analytics upload data size and logging analytics upload failure alarms) are created.

Architecture

Management Agents is fully integrated with other Oracle Cloud Infrastructure services which makes it easy to automate the metric alarms configuration for Logging Analytics using Resource Manager and stacks. After the alarms are defined, they are monitored using the Monitoring service and notified when the metrics meet alarm-specified triggers using Notifications service.

Prerequisites

User must have permissions or policies to manage management agents, metrics, and notification topics.

User must be part of the Admin group or have permission to invoke the Resource Manager stack.

If the user is not part of the Admin group, they should have the following permissions to invoke the Resource Manager stack:

# Policies required to manage stacks and jobs in OCI Resource Manager
ALLOW GROUP <User-Group> TO MANAGE orm-stacks IN TENANCY
ALLOW GROUP <User-Group> TO MANAGE orm-jobs IN TENANCY
 
# Policies required to read and manage IAM resources
ALLOW GROUP <User-Group> TO READ compartments IN TENANCY
ALLOW GROUP <User-Group> TO READ tenancies IN TENANCY
ALLOW GROUP <User-Group> TO MANAGE dynamic-groups IN TENANCY
ALLOW GROUP <User-Group> TO MANAGE policies IN TENANCY
 
# Policies required to manage Management Agents
ALLOW GROUP <User-Group> TO MANAGE management-agents IN TENANCY
ALLOW GROUP <User-Group> TO READ metrics IN TENANCY
 
# Policies required to create and manage alarms and notification topics
ALLOW GROUP <User-Group> TO MANAGE alarms IN TENANCY
ALLOW GROUP <User-Group> TO USE ons-topics IN TENANCY

Note

To restrict policies explicitly to specific compartments, use <COMPARTMENT_NAME> instead of TENANCY. For example:

ALLOW GROUP <User-Group> TO MANAGE ALARMS IN COMPARTMENT <COMPARTMENT_NAME>.

For details about creating and managing user groups, see Managing Groups.

Deploy Default Alarms for Management Agent Using Marketplace Application

Sign in to the Oracle Cloud.
Open the navigation menu. Under Marketplace, click Marketplace to view all the applications.
Go to the search box, type in Management Agents and select the Default Alarms for Management Agents stack listing.

The details for the listing appear.
Select Stack Version and Compartment.
Review the Oracle terms of use and check the I have reviewed and accept checkbox.
Click Launch Stack.

The Resource Manager page appears, displaying the Create stack process.
Under step 1 Stack information, review the stack information.
- Under Name, review autopopulated name to identify the stack.
- Under Description, add a description for the stack deployment optionally.
- Under Create in compartment, select a compartment from the drop-down list.
- Optionally specify namespace tags to organize and track resources.
- Click Next to continue to the next step.
Under step 2 Configure variables, review and update default settings if needed.
- Under Required Configuration, review the following:
  - Under Management Agent Compartment, select the compartment where the Management Agents are installed.
    - If the root compartment/tenancy is selected, the Enable nested management agent compartments checkbox is displayed. Uncheck this checkbox if you do not want to monitor the agents in the subcompartments.
    - Check the Create new topic checkbox to create a new notification destination topic without any subscriptions. If a notification destination topic was already created, uncheck this checkbox.
  - Under Stack identifier, enter a unique name to identify the alarms and notification topics that are going to be created.
  - Under Prerequisite, review and confirm the required policies.
- Under Management Agent Alarms Configuration, review the four alarm definitions about to get created by default.
  - Check Customize management agent alarms to disable or modify the settings of any alarm.
    Note
    
    Uncheck an alarm checkbox, if you don't want to monitor it (remove it from the stack creation).
  - Check Availability alarm to set up an alarm to check the Management Agent availability.
  - Check CPU alarm to set up an alarm for the Management Agent's CPU percentage.
  - Check Disk alarm to set up an alarm for the Management Agent disk space.
  - Check Memory alarm to set up an alarm to check the amount of memory consumed by the Management Agent.
    Note
    
    Modify the alarm interval, threshold and severity of any alarm if needed.
- Under Logging Analytics Alarm Configuration, review the two alarm definitions about to get created by default.
  - Check Add logging analytics metrics to set up the alarms. If the alarms are not needed, uncheck this checkbox.
  - Check Customize logging analytics alarms to remove or modify the settings of any of the two alarms.
  - Check Logging analytics upload data size alarm to set up an alarm for the log collection upload data size.
  - Check Logging analytics upload failure alarm to set up an alarm when there's log collection upload failures.
Under step 3 Review, review all the information is accurate and complete.
- Under Run apply on the created stack?, uncheck the Run apply checkbox.
  This ensure that this process only creates the stack. The apply stack will be done in another step.
Click Create.

Wait until the page indicates that the stack is created before continuing to the next step.
Under the Stack details page, click Plan to create a plan job.

Provide a unique name for the Plan job and click Plan.
Review the logs section on the same page to ensure there are no issues.
Select Stack Details from the navigation menu (Resource Manager > Stacks > Stack details). Under the created stack, go to Job information and confirm that State displays Succeeded.
Go to the Stack Details tab and click Apply.
- Provide a unique name to identify the Apply job.
- Keep the Apply Job Plan Resolution to Automatically approve the selected.
- Click Apply to execute the Apply job.
  
  This process creates the alarm definitions and bind them to the notification topic.
  
  Review the Logs section to ensure there are no issues.
- Under Job Information tab, confirm that State displays Succeeded.
- From the Resources left menu, click Outputs to view the execution output information.

Deploy Default Alarms for Management Agent Using Github

You can also deploy the Default Alarms for Management Agent stack using Github.

Go to https://github.com/oracle-quickstart/oci-observability-and-management/o11y-ingestion-monitoring
Click Deploy to Oracle Cloud from the README.md section.
Accept the Oracle terms of use and click Next.
Apply the stack using Resource Maneger.

Go to the Deploy Default Alarms for Management Agent Using Marketplace Application section. Skip step 1 through 6 and follow the instructions starting at step 7.

Using the Console

To view the default metrics for a single Management Agent, you can use Service Metrics.

Open the navigation menu. Under Observability & Management, go to Monitoring and click Service Metrics.
For Metric Namespace, select oci_managementagent.
For Dimensions, specify resourceId and select the OCID of the management agent for which you want to display service metrics.

The page displays charts for the metrics that are emitted by the Management Agent.

Using the API

For information about using the API and signing requests, see REST APIs and Security Credentials. For information about SDKs, see Software Development Kits and Command Line Interface.

Use the following APIs for monitoring:

Monitoring API for metrics and alarms
Notifications API for notifications (used with alarms)