Configuring Oracle-Managed Infrastructure Maintenance
Oracle performs the updates to all of the Oracle-managed infrastructure components on Exadata Cloud@Customer.
You may manage contacts who are notified regarding infrastructure maintenance, set a maintenance window to determine the time your quarterly infrastructure maintenance will begin, and also view scheduled maintenance runs and the maintenance history of your Exadata Cloud@Customer in the Oracle Cloud Infrastructure Console. For details regarding the infrastructure maintenance process and configuring the maintenance controls refer to the following:
- About Oracle Managed Oracle Exadata Database Service on Cloud@Customer Infrastructure Maintenance Updates
Oracle performs patches and updates to all of the Oracle-managed system components on Oracle Exadata Database Service on Cloud@Customer. - Infrastructure Maintenance Contacts
Maintenance contacts are required for service request based communications for hardware replacement and other maintenance events. - Using the Console to Configure Oracle-Managed Infrastructure Updates
Full Exadata infrastructure software updates are scheduled on a quarterly basis. In addition, important security updates are scheduled monthly. While you cannot opt-out of these infrastructure updates, Oracle alerts you in advance through the Cloud Notification Portal and allows scheduling flexibility to help you plan for them. - Monitor Infrastructure Maintenance Using Lifecycle State Information
The lifecycle state of your Exadata Infrastructure resource enables you to monitor when the maintenance of your infrastructure resource begins and ends. - Receive Notifications about Your Infrastructure Maintenance Updates
There are two ways to receive notifications. One is through email to infrastructure maintenance contacts and the other one is to subscribe to the maintenance events and get notified.
About Oracle Managed Oracle Exadata Database Service on Cloud@Customer Infrastructure Maintenance Updates
Oracle performs patches and updates to all of the Oracle-managed system components on Oracle Exadata Database Service on Cloud@Customer.
Oracle patches and updates include the physical database server hosts, Exadata Storage Servers, Network Fabric Switches, management switch, power distribution units (PDUs), integrated lights-out management (ILOM) interfaces, and Control Plane Servers. This is referred to as Exadata Cloud@Customer infrastructure maintenance.
In all but rare exceptional circumstances, you receive advance communication about these updates to help you plan for them. If there are corresponding recommended updates for your VM cluster virtual machines (VMs), then Oracle provides notifications about them.
Wherever possible, scheduled updates are performed in a manner that preserves service availability throughout the update process. However, there can be some noticeable impact on performance and throughput while individual system components are unavailable during the update process.
For example, database server patching typically requires a reboot. In such cases, wherever possible, the database servers are restarted in a rolling manner, one at a time, to ensure that the service remains available throughout the process. However, each database server is unavailable for a short time while it restarts, and the overall service capacity diminishes accordingly. If your applications cannot tolerate the restarts, then take mitigating action as needed. For example, shut down an application while database server patching occurs.
- Overview of the Quarterly Infrastructure Maintenance Process
By default, infrastructure maintenance updates the Exadata database server hosts in a rolling fashion, followed by updating the storage servers.
Parent topic: Configuring Oracle-Managed Infrastructure Maintenance
Overview of the Quarterly Infrastructure Maintenance Process
By default, infrastructure maintenance updates the Exadata database server hosts in a rolling fashion, followed by updating the storage servers.
You can also choose non-rolling maintenance to update database and storage servers. The non-rolling maintenance method first updates your storage servers at the same time, then your database servers at the same time. Although non-rolling maintenance minimizes maintenance time, it incurs full system downtime while the storage servers and database servers are being updated.
Rolling infrastructure maintenance begins with the Exadata database server hosts. For the rolling maintenance method, database servers are updated one at a time. Each of the database server host's VMs is shut down, the host is updated, restarted, and then the VMs are started, while other database servers remain operational. This rolling maintenance impact older applications not written to handle a rolling instance outage. This process continues until all servers are updated.
After database server maintenance is complete, storage server maintenance begins. For the rolling maintenance method, storage servers are updated one at a time and do not impact VM cluster VM's availability. However, the rolling storage server maintenance can result in reduced IO performance as storage servers are taken offline (reducing available IO capacity) and resynced when brought back online (small overhead on database servers). Properly sizing the database and storage infrastructure to accommodate increased work distributed to database and storage servers not under maintenance will minimize (or eliminate) any performance impact.
Note that while databases are expected to be available during the rolling maintenance process, the automated maintenance verifies Oracle Clusterware is running but does not verify that all database services and pluggable databases (PDBs) are available after a server is brought back online. The availability of database services and PDBs after maintenance can depend on the application service definition. For example, a database service, configured with certain preferred and available nodes, may be relocated during the maintenance and wouldn't automatically be relocated back to its original node after the maintenance completes. Oracle recommends reviewing the documentation on Achieving Continuous Availability for Your Applications on Exadata Cloud Systems to reduce the potential for impact to your applications. By following the documentation's guidelines, the impact of infrastructure maintenance will be only minor service degradation as database servers are sequentially updated.
Oracle recommends that you follow the Maximum Availability Architecture (MAA) best practices and use Data Guard to ensure the highest availability for your critical applications. For databases with Data Guard enabled, Oracle recommends that you separate the maintenance windows for the infrastructure instances running the primary and standby databases. You may also perform a switchover prior to the maintenance operations for the infrastructure instance hosting the primary database. This allows you to avoid any impact on your primary database during infrastructure maintenance.
Prechecks are performed on the Exadata Cloud@Customer infrastructure components prior to the start of the maintenance window. The goal of the prechecks is to identify issues that may prevent the infrastructure maintenance from succeeding. The Exadata infrastructure and all components remain online during the prechecks. An initial precheck is run approximately two weeks prior to the maintenance start and another precheck is run approximately 24 hours prior to maintenance start. If the prechecks identify an issue that requires rescheduling the maintenance notification is sent to the maintenance contacts.
The time taken to update infrastructure components varies depending on the number of database servers and storage servers in the Exadata infrastructure, the maintenance method, and whether custom action has been enabled. The approximate times provided are estimates. Time for custom action, if configured, is not included in the estimates below. Database server maintenance time may vary depending on the time required to shutdown each VM before the update and then start each VM and associated resources after the update of each node before proceeding to the next node. The storage server maintenance time will vary depending on the time required for the ASM rebalance, which is not included in the estimates below. If issues are encountered during maintenance this may also delay completion beyond the approximate time listed. In such a situation, if Oracle cloud operations determine resolution would extend beyond the expected window, they will send a notification and may reschedule the maintenance.
The timeframes mentioned below can change if Oracle cloud operations determine that additional maintenance work is needed. If additional time is necessary, Oracle will send a customer notification in advance to inform customers that additional time will be required for the next quarterly maintenance window.
- Rolling:
- Each database server takes 90 minutes on average.
- Each storage server takes 60 minutes on average.
- Each InfiniBand or RoCE fabric switch takes 30 minutes on average.
-
The approximate total time for infrastructure maintenance is as follows:
-
Base and Quarter Rack (2 Database Servers/3 Storage Servers): Approximately 7 hours
2 Database Servers X 90 = 180 minutes
3 Storage Servers X 60 = 180 minutes
2 InfiniBand or RoCE Fabric Switch X 30 = 60 minutes
-
Half Rack (4 Database Servers/6 Storage Servers): Approximately 13 hours
4 Database Servers X 90 = 360 minutes
6 Storage Servers X 60 = 360 minutes
2 InfiniBand or RoCE Fabric Switch X 30 = 60 minutes
-
Full Rack (8 Database Servers/12 Storage Servers): Approximately 26 hours
8 Database Servers X 90 = 720 minutes
12 Storage Servers X 60 = 720 minutes
2 InfiniBand or RoCE Fabric Switch X 30 = 60 minutes
-
- Non-Rolling:
- All database servers take 180 minutes on average.
- All storage servers take 60 minutes on average.
- Storage Servers and Database servers are brought back online prior to starting fabric switch maintenance.
- Network fabric switches are still updated in a rolling method and take 30 minutes each on average.
- The approximate total time for infrastructure maintenance is 5 hours
regardless of shape:
- All Database Servers = 180 minutes
- All Storage Servers = 60 minutes
- 2 InfiniBand or RoCE Fabric Switch = 60 minutes
Infrastructure Maintenance Contacts
Maintenance contacts are required for service request based communications for hardware replacement and other maintenance events.
Add a primary maintenance contact and optionally add a maximum of nine secondary contacts. Both the primary and secondary contacts receive all notifications about hardware replacement, network issues, and software maintenance runs.
You can promote any secondary contacts as the primary anytime you want. When you promote a secondary contact to primary, the current primary contact will be demoted automatically to secondary.
For more information, see: Using the Console to Create Infrastructure and Managing Infrastructure Maintenance Contacts.
Using the Console to Configure Oracle-Managed Infrastructure Updates
Full Exadata infrastructure software updates are scheduled on a quarterly basis. In addition, important security updates are scheduled monthly. While you cannot opt-out of these infrastructure updates, Oracle alerts you in advance through the Cloud Notification Portal and allows scheduling flexibility to help you plan for them.
For quarterly infrastructure maintenance, you can set a maintenance window to determine when the maintenance will begin. You can also edit the maintenance method, enable custom action, and view the scheduled maintenance runs and the maintenance history of your Exadata Cloud@Customer in the Oracle Cloud Infrastructure Console. For security maintenance, you may edit the scheduled start time within the 21-day window.
- View or Edit Quarterly Infrastructure Maintenance Preferences for Exadata Cloud@Customer Infrastructure
- View or Edit a Scheduled Quarterly Maintenance for Exadata Cloud@Customer Infrastructure
- View or Edit a Scheduled Security Maintenance for Exadata Cloud@Customer Infrastructure
- View the Maintenance History of Exadata Cloud@Customer Infrastructure
- View and Edit Quarterly Maintenance While Maintenance is In Progress or Waiting for Custom Action
- View or Edit a Scheduled Security Maintenance for Exadata Cloud@Customer Infrastructure
- View or Edit Quarterly Infrastructure Maintenance Preferences for Oracle Exadata Database Service on Cloud@Customer Infrastructure
To edit your Oracle Exadata Database Service on Cloud@Customer infrastructure quarterly maintenance preferences, be prepared to provide values for the infrastructure configuration. The changes you make will only apply to future maintenance runs, not those already scheduled. - View or Edit a Scheduled Quarterly Maintenance for Oracle Exadata Database Service on Cloud@Customer Infrastructure
Learn how to view and edit the time of the next scheduled maintenance. - View the Maintenance History of Oracle Exadata Database Service on Cloud@Customer Infrastructure
Learn how to view the maintenance history for an Oracle Exadata Database Service on Cloud@Customer Infrastructure.
Parent topic: Configuring Oracle-Managed Infrastructure Maintenance
View or Edit Quarterly Infrastructure Maintenance Preferences for Oracle Exadata Database Service on Cloud@Customer Infrastructure
To edit your Oracle Exadata Database Service on Cloud@Customer infrastructure quarterly maintenance preferences, be prepared to provide values for the infrastructure configuration. The changes you make will only apply to future maintenance runs, not those already scheduled.
- Open the navigation menu. Under Oracle Database, click Exadata Database Service on Cloud@Customer.
- Select Region and Compartment, and provide the region and the compartment where the Oracle Exadata infrastructure you want to edit is located.
- Click Exadata Infrastructure.
- Click the name of the Exadata infrastructure that you want to edit.
The Infrastructure Details page displays information about the selected Oracle Exadata infrastructure.
- Click Edit Maintenance Preferences.
Edit Maintenance Preferences page is displayed.
Note
Changes made to maintenance preferences apply only to future maintenance, not the maintenance that has already been scheduled. To modify scheduled maintenance, see View or Edit a Scheduled Maintenance for Exadata Cloud@Customer Infrastructure.
- On the Edit Maintenance Preferences page, configure the following:
- Choose a maintenance method:
- Rolling: By default, Exadata Infrastructure is updated in a rolling fashion, one server at a time with no downtime.
- Non-rolling: Update database and storage servers at the same time. The non-rolling maintenance method minimizes maintenance time but incurs full system downtime.
- Enable custom action before performing maintenance on DB
servers: Enable custom action only if you want to perform
additional actions outside of Oracle’s purview. For maintenance
configured with a rolling software update, enabling this option will
force the maintenance run to wait for a custom action with a configured
timeout before starting maintenance on each DB server. For maintenance
configured with non-rolling software updates, the maintenance run will
wait for a custom action with a configured timeout before starting
maintenance across all DB servers. The maintenance run, while waiting
for the custom action, may also be resumed prior to the timeout.
-
Custom action timeout (in minutes): Timeout available to perform custom action before starting maintenance on the DB Servers.
Default: 30 minutes
Maximum: 120 minutes
-
- Maintenance schedule:
- No preference: The system assigns a date and start time for infrastructure maintenance.
- Specify a schedule: Choose your preferred month, week,
weekday, start time, and lead time for infrastructure
maintenance.
- Under Maintenance months, specify at least one month for each quarter during which Exadata infrastructure maintenance will take place. You can select more than one month per quarter. If you specify a long lead time for advanced notification (for example, 4 weeks), you may wish to specify 2 or 3 months per quarter during which maintenance runs can occur. This will ensure that your maintenance updates are applied in a timely manner after accounting for your required lead time. Lead time is discussed in the following steps.
- Optional. Under Week of the month, specify which week of the month, maintenance will take place. Weeks start on the 1st, 8th, 15th, and 22nd days of the month, and have a duration of 7 days. Weeks start and end based on calendar dates, not days of the week. Maintenance cannot be scheduled for the fifth week of months that contain more than 28 days. If you do not specify a week of the month, Oracle will run the maintenance update in a week to minimize disruption.
- Optional. Under Day of the week, specify the day of the week on which the maintenance will occur. If you do not specify a day of the week, Oracle will run the maintenance update on a weekend day to minimize disruption.
- Optional. Under Start hour, specify the hour during which the maintenance run will begin. If you do not specify a start hour, Oracle will pick the least disruptive time to run the maintenance update.
- Under Lead Time, specify the minimum number of weeks ahead of the maintenance event you would like to receive a notification message. Your lead time ensures that a newly released maintenance update is scheduled to account for your required minimum period of advanced notification.
- Choose a maintenance method:
- Click Save Changes.
If you switch from rolling to non-rolling maintenance method, then Confirm Non-rolling Maintenance Method dialog is displayed.
- Enter the name of the infrastructure in the field provided to confirm the changes.
- Click Save Changes.
View or Edit a Scheduled Quarterly Maintenance for Oracle Exadata Database Service on Cloud@Customer Infrastructure
Learn how to view and edit the time of the next scheduled maintenance.
Monitor Infrastructure Maintenance Using Lifecycle State Information
The lifecycle state of your Exadata Infrastructure resource enables you to monitor when the maintenance of your infrastructure resource begins and ends.
In the Oracle Cloud Infrastructure Console, you can see lifecycle state
details messages on the Exadata Infrastructure Details page when a tooltip is
displayed beside the Status field. You can also access these messages using the
ListExadataInfrastructures
API, and using tools based on the API,
including SDKs and the OCI CLI.
-
If you specify a maintenance window, then patching begins at your specified start time. The infrastructure resource's lifecycle state changes from Available to Maintenance in Progress.Note
The prechecks are now done prior to the start of the maintenance. - When Exadata database server maintenance starts, the infrastructure resource's lifecycle state is Maintenance in Progress, and the associated lifecycle state message is, The underlying infrastructure of this system (dbnodes) is being updated.
- When storage server maintenance starts, the infrastructure resource's lifecycle state is Maintenance in Progress, and the associated lifecycle state message is, The underlying infrastructure of this system (cell storage) is being updated and this will not impact Database availability.
- After storage server maintenance is complete, the networking switches are updated one at a time, in a rolling fashion.
- When maintenance is complete, the infrastructure resource's lifecycle state is Available, and the Console and API-based tools do not provide a lifecycle state message.
Receive Notifications about Your Infrastructure Maintenance Updates
There are two ways to receive notifications. One is through email to infrastructure maintenance contacts and the other one is to subscribe to the maintenance events and get notified.
Oracle schedules maintenance run of your infrastructure based on your scheduling preferences and sends email notifications to all your infrastructure maintenance contacts. You can login to the console and view details of the schedule maintenance run. Appropriate maintenance related events will be generated as Oracle prepares for your scheduled maintenance run, for example, precheck, patching started, patching end, and so on. For more information about all maintenance related events, see Oracle Exadata Cloud@Customer Events. In case, if there are any failures, then Oracle reschedules your maintenance run, generates related notification, and notifies your infrastructure maintenance contacts.
For more information about Oracle Cloud Infrastructure Events, see Overview of Events. To receive additional notifications other than the ones sent to infrastructure maintenance contacts, you can subscribe to infrastructure maintenance events and get notified using the Oracle Notification service, see Notifications Overview.