Oracle Cloud Infrastructure Documentation

Troubleshoot Shape Change Failures

This article helps you troubleshoot and fix the issues that might occur when you change the shape of your DB system.

For multi-node RAC DB systems, shape change operations proceed in a rolling fashion. Depending on where in the shape change operation the failure occurs, you may be able to re-try the operation using the console.

This article includes information to help you determine the cause of the failure and fix the problem. The information is organized into several sections, based on the error condition.

The following topics are covered in this article:

Note

You can also create serial console connections to troubleshoot your DB system in single-user mode. For information on creating a serial console connection in the Console, see Manage Serial Console Connection to the DB System.

Use the OCI Console to Troubleshoot

If your shape change operation fails, a message banner appears on the DB system details page to provide details about the failure. If the failure happens on the first node of a multi-node system, and the operation is rolled back successfully, the Change shape button remains available and the system remains online, in the available state. Contact Oracle Support to get additional details about the failure. You can also use the dbcli to learn more about the failure and what issues need to be resolved. After determining that no issues remain, you can try the operation again.

If the failure leaves the system in a state where the operation cannot be rolled back, the system state is Needs Attention. In this case, contact Oracle Support as soon as you are aware of the issue so Oracle can help you resolve the issue and complete the shape change operation.

Use dbcli to Troubleshoot

The following topics are covered:

Determine What Stage of the Shape Change Operation Failed

  1. Login to the DB system as the root user.

  2. Navigate to /opt/oracle/dcs/bin: 

    cd /opt/oracle/dcs/bin

  3. Update the CLI tool:

    ./cliadm update-dbcli

  4. List the failed jobs:

    dbcli list-jobs | grep -i failed
Note

If the failed job (or jobs) occurred during the pre_action, action, or post_action stage. Also note the job_id value of the failed job, which you will need to resolve the issue.

Troubleshooting Failures That Occur in the Pre_action Stage

  1. Use the job_id value to get more information about the failure: 

    dbcli describe-job -i <job_id>

  2. Search for the error in the dcs-agent.log and dcs-agent-debug.log files (which are located in the /opt/oracle/dcs/log/ directory):

    For example:

    cd /opt/oracle/dcs/log
grep -ir "DCS-10063:Failed to get node names from olsnodes." *
  3. Using the information about the error recorded in the log file, correct the system configuration if possible.
  4. Re-try the shape change operation. If the operation is still not successful, follow the instructions in Get Additional Help.

Get Additional Help

If your failure occurs in the post_action stage, or if you are unable to resolve failures that occur in the pre_action stage, do the following:

  1. Gather the information listed in the document Diagnostic Data Collection For Oracle Database Cloud Service Instance.
  2. Open a Support Request with My Oracle Support. Contact Oracle Support.