Cluster Backups Using the Snapshot API

OCI Search with OpenSearch lets you use the OpenSearch Snapshot API to create a backup of your cluster.

This method is different than the managed backups that Search with OpenSearch automatically generates for your cluster. For more information about the cluster backups managed by Search with OpenSearch, see Automated Cluster Backups.

Using the OpenSearch Snapshot API provides you with more flexibility when creating or restoring backups on OpenSearch cluster, so you can customize the behavior. Some example customizations you could make:

Rename indexes in the scenario where there's an index name collision between the snapshot and the target cluster you're restoring the snapshot to.
Specify certain indexes to include or exclude.
Specify index aliases to include or exclude.

Snapshots are stored in an Object Storage bucket that you specify in your tenancy. Your bill will include storage costs for the cluster snapshots, see Cloud Storage Pricing for details.

This topic describes the prerequisites and the process for using the OpenSearch Snapshot API with your clusters, and includes the following tasks:

Create the Object Storage bucket that will be used as the repository for the cluster snapshot.
Configure a dynamic policy with the required permissions.
Register your Object Storage bucket as the repository.
Take the snapshot.
Restore the snapshot.

Create Object Storage Bucket for Storing Snapshot Files

The repository for snapshots is an Object Storage bucket in your tenancy. If you do not have an Object Storage bucket to register as the repository, you need to create a bucket. For a tutorial that walks you through how to create a bucket, see Putting Data into Object Storage.

Make note of the following information for the bucket you'll register as the repository. You can find this information on the Bucket Details page in the Console.

Namespace
Bucket name
Compartment OCID

To get the compartment OCID:

On the Bucket Details page click the link for Compartment.
Click Copy for OCID in Compartment Information.

Note

You can skip this step and have Search with OpenSearch create the bucket when the repository is registered by specifying true for the forceBucketCreation attribute, as follows:

forceBucketCreation: true

If you use this approach, you need to ensure that you grant sufficient access in the IAM policy for Search with OpenSearch to create an Object Storage bucket.

IAM Policy

You need to configure permissions to provide the OpenSearch cluster access to the Object Storage bucket in your tenancy for storing snapshot files.

The following policy example includes the required permissions:

DEFINE tenancy opensearch-tenancy as <OpenSearch_Tenancy_ID>
ADMIT resource opensearch opensearchsnapshots of tenancy opensearch-tenancy to manage object-family in compartment <snapshot_bucket_compartment> where ALL {request.principal.clusterid='<cluster_OCID>', request.principal.ownertenant='<customer_tenancy_OCID>', request.principal.ownercompartment='<customer_compartment_OCID>', target.bucket.name='<snapshot_bucket_name>'}

The preceding policy only allows you to register the specified bucket as the snapshot repository.

<OpenSearch_Tenancy_ID> is the OCID for the OCI Search with OpenSearch tenancy. You can find this value on the Cluster details page for your cluster in the Opensearch Tenancy OCID field under the Additional Info section.

<customer_tenancy_OCID> is the OCID for your tenancy where you created the OpenSearch cluster.

<customer_compartment_OCID> is the OCID for the compartment where the OpenSearch cluster is located.

If you specify forceBucketCreation : true in the register operation, you need to ensure that the specified policy includes bucket create access in your tenancy, otherwise the operation to register the repository will fail.

Register the Repository

Before you can take the snapshot, you need to register the repository, as shown in the following example:

PUT _snapshot/<repository_name>
{
  "type": "oci",
  "settings": {
    "client": "default",
    "endpoint": "<objectstorage_endpoint>",
    "bucket": "<bucket_name>",
    "namespace": "<namespace>",
    "authType": "RESOURCE_PRINCIPAL",
    "bucket_compartment_id": "<bucket_compartment_OCID>",
    "forceBucketCreation": true
  }
}

The following table describes the parameters to specify when you register the repository.


Parameter	Description
repository name	The name you assign to the repository for the snapshot.
endpoint	The endpoint for the Object Storage bucket. For example, the US East (Ashburn) region Object Storage endpoint is https://objectstorage.us-ashburn-1.oraclecloud.com.
namepace	The Object Storage namespace for the bucket.
authType	Specify to use the resource principal which allows Search with OpenSearch to work on your behalf.
bucket_compartment_id	The OCID of the compartment where the bucket is located.
forceBucketCreation	Specifies to create the Object Storage bucket if it does not exist already. Default is false.

Take Snapshot

After you have registered the repository, you can take a snapshot, as shown in the following example:

POST _snapshot/<repository_name>/<snapshot_name>?wait_for_completion=true

The preceding example includes all indexes and the cluster's state. The next example shows how to include specific indexes and other settings:

POST _snapshot/<repository_name>/<snapshot_name>?wait_for_completion=true
{
"indices": "<index_name>,-<index_name>",
  "ignore_unavailable": true,
  "include_global_state": false
}

You can specify one or more indices to include, <index_name_to_include>, and one or more indices to exclude, -<index_name_to_exclude>.

For more information about this API operation, see Take snapshots.

Restore Snapshot

Before you restore a snapshot, you can retrieve all the existing snapshots for a cluster. First, retrieve all the repositories registered with the cluster, as shown in the following example:

GET _snapshot/_all?pretty

Then, retrieve all the snapshots in repository, as shown in the following example:

GET _snapshot/<repository_name>?pretty

Now, restore the snapshot, as shown in the following example:

POST _snapshot/<repository_name>/<snapshot_name>/_restore
{
     "indices": "<index_name_to_include>,-<index_name_to_exclude>",
     "ignore_unavailable": true,
     "include_global_state": false,
     "rename_pattern": "(.+)",
     "rename_replacement": "restored_$1",
     "include_aliases": true
}