To create a notification-based live table feed:

• Step 1: Configure your object store bucket to emit notifications
• Step 2: Create a Notifications service subscription topic
• Step 3: Create an Events service rule
• Step 4: Create and configure a live table feed to use notifications, and copy the notification URL
• Step 5: Create a Notifications service subscription
• Step 6: Confirm that the endpoint can receive notifications

Step 1: Configure your object store bucket to emit notifications

Where: Oracle Cloud Infrastructure Console: Object Storage & Archive Storage - Buckets page

Configure the bucket containing your source data so that it will emit notifications when the data changes. You can set this option when you create a bucket or you can set it in an existing bucket.

1. Open the Cloud Console navigation menu and click Storage. Under Object Storage and Archive Storage, click Buckets.

2. If you're creating a new bucket:

   1. On the Buckets page, click the Create Bucket button to create a new bucket, as described in Managing Buckets. In the Create Bucket wizard, select the Emit Object Events option, along with the other options for your new bucket.
   2. Click Create.

   If you're using an existing bucket:

   1. On the Buckets page, click the name of the bucket you want to use, as described in Managing Buckets.
   2. On the Bucket Details page, click the Edit link next to Emit Object Events.
   3. Select the Emit Objects Events check box, and then click Save Changes.

Step 2: Create a Notifications service subscription topic

Where: Oracle Cloud Infrastructure Console: Notifications - Topics page

1. Open the Cloud Console navigation menu and click Developer Services. Under Application Integration, click Notifications.
2. Click Create Topic, enter a name and optional description, and then click Create.

Step 3: Create an Events service rule

Where: Oracle Cloud Infrastructure Console: Events - Rules page

1. Open the Cloud Console navigation menu and click Observability & Management. Under Events Service, click Rules.
2. Click Create Rule, and fill out the Create Rule page as described in Managing Rules for Events.
   • Under Rule Conditions, select:
     • Condition: Event Type
     • Service Name: Object Storage
     • Event Type: Object - Create
   • Under Actions, select:
     • Action Type: Notifications
     • Notifications Compartment: Select the compartment to use for the notifications.
     • Topic: Select the name of the topic you created above, in Step 2: Create a Notifications service subscription topic.
3. Click Create Rule.

Step 4: Create and configure a live table feed to use notifications, and copy the notification URL

Where: Database Actions: Live Feeds page

You can configure a new or an existing live table feed to use notifications:

1. Go to the Database Actions Live Feeds page, as described in Feeding Data.
2. Create or edit a live table feed object, as described in Create a Live Table Feed Object or Edit a Live Table Feed Object. Select the Enable for Notification option
3. Click Create or Save.
4. Click the Actions (three vertical dots) icon on the card for your live feed, and select Show Confirmation URL.
5. In the Notification URL dialog box, click the Copy icon to copy the URL to the clipboard. You may want to copy it to a temporary file, so you can retrieve it later. You'll use this URL in the next step, Step 5: Create a Notifications service subscription.

Step 5: Create a Notifications service subscription

Where: Oracle Cloud Infrastructure Console: Notifications - Subscriptions page

1. Return to the Oracle Cloud Infrastructure Console. Open the navigation menu and click Developer Services. Under Application Integration, click Notifications.
2. On the Notifications page, click the Subscriptions tab (on the left side of the page), the status will be Active.
3. Click Create Subscription and fill in the Create Subscription page:
   • Subscription topic: Select the subscription topic you created in Step 2: Create a Notifications service subscription topic.
   • Protocol: HTTPS (Custom URL)
   • URL: Paste in the URL you copied in Step 4: Create and configure a live table feed to use notifications, and copy the notification URL.
   • Click Create. The subscription will be listed in the Subscriptions table in a state of "Pending."

Step 6: Confirm that the endpoint can receive notifications

Where: Database Actions: Live Feeds page

1. Return to the Database Actions Live Feeds page and find the card for the live table feed you are configuring for a notification-based feed.
2. Click the Actions (three vertical dots) icon on the card, and select Show Confirmation URL.
3. In the Confirmation URL dialog box, click the link to confirm the URL. This does not close this dialog box. If the link is successful, a message is displayed that confirms the subscription is active.
4. Return to the Confirmation URL dialog box and select the Check only when the cloud store confirmation process is complete check box, and click OK.

Once you finish the above steps, any new files uploaded to the bucket will automatically be loaded into the live table feed table.

Step 1: Create your object store bucket in Amazon S3

Where: Amazon Web Services (AWS) Management console

Step 2: Create Access Keys

Where: AWS Management console

To access Amazon Simple Notification Service (SNS), you must have credentials that Amazon Web Services (AWS) can use to validate your requests. These credentials must have permissions to access Amazon SNS topics. The following steps provide you details on steps to create access keys using AWS Identity and Access Management (IAM) for security purposes.

1. Log in to AWS Management console and open Amazon Identity and Access Management (IAM) console.
2. On the navigation menu, select Users.
3. Select your user name.
4. In the Security Credentials tab, select Create access key.
5. Copy the Access key ID and Secret access key in the display. Paste them in a clipboard.
6. To download the keys, select Download.csv file icon. This way you can store the file in a secure location.

Step 3: Add an Amazon S3 Cloud Storage Link

Where: Database Actions: Manage Cloud page

Before you create a live table feed, you must establish a connection to the cloud store you want to use.

1. Click the Manage Cloud Store button at the top of the page to go to the Manage Cloud page. For further instructions on adding source files residing in cloud storage provided by Amazon S3, refer to Create an Amazon S3 Cloud Storage Link topic in Managing Connections.
   Note
   
   Paste the Access key ID and Secret access key generated in the previous step (Step 2: Create Access Keys) to their respective text fields in the Add Cloud Storage page.

Step 4: Create and configure a live table feed to use notifications, and copy the notification URL

Where: Database Actions: Live Feeds page

Creating a live table feed enables you to load data in real time from external storage sources to your table in ADB. External storage you use include as Oracle Object Store, AWS S3 or Microsoft Azure containers.

You can configure a new or an existing live table feed to use notifications:

1. Go to the Database Actions Live Feeds page, as described in Feeding Data.
2. Create or edit a live table feed object, as described in Create a Live Table Feed Object or Edit a Live Table Feed Object. Select the Enable for Notification option
3. Click Create or Save.
4. Click the Actions (three vertical dots) icon on the card for your live feed, and select Show Notification URL.
5. In the Notification URL dialog box, click the Copy icon to copy the URL to the clipboard. You may want to copy it to a temporary file, so you can retrieve it later. You will use this URL in the subsequent step (Step 7: Create a notifications service subscription).

Step 5: Create a notifications service subscription topic

Where: Amazon Simple Notification Service (SNS) console

You receive Amazon S3 notifications using Amazon Simple Notification Service (Amazon SNS) topic. You need to add a notification configuration to your bucket using an Amazon SNS topic. SNS topics are shared locations which are used to send notifications of various events that happen in AWS buckets.

During creation, you select a topic name and topic type. After creating a topic, you cannot change the topic type or name. All other configuration choices are optional during topic creation, which you can edit later.

To access any AWS service, you must first create an AWS account.

Navigate to the AWS Management console, and then select Create an AWS Account.

Follow the instructions as provided in the Amazon SNS link to create your first IAM administrator user and group. Now you can log in to any of the AWS services as an IAM user.

1. Log in to Amazon SNS console as an IAM user.
2. On the Topics page, select Create topic.
3. Specify the following fields on the Create topic page, in the Details section.
   • Type:Standard (Standard or FIFO)
   • Name: notify-topic. For a FIFO topic, add fifo to the end of the name.
   • Display Name: This field is optional.
4. Expand the Encryption section and select Disable encryption.
5. Expand the Access policy section and configure additional access permissions, if required. By default, only the topic owner can publish or subscribe to the topic. This step is optional. Edit the JSON format of the policy based on the topic details you enter. Here is a sample of Access policy in JSON format.

   { "Version": "2008-10-17",
   "Id": "__default_policy_ID",
   "Statement":[
   {"Sid": "__default_statement_ID",
   "Effect": "Allow",
   "Principal": {"AWS": "*"
   },"Action": [
           "SNS:Publish",
           "SNS:RemovePermission",
           "SNS:SetTopicAttributes",
           "SNS:DeleteTopic",
           "SNS:ListSubscriptionsByTopic",
           "SNS:GetTopicAttributes",
           "SNS:AddPermission",
           "SNS:Subscribe"
         ],
         "Resource": "arn:aws:sns:us-west-2:555555555555:notify-topic", //us-west-2 is the region
         "Condition": {
           "StringEquals": {
             "AWS:SourceOwner": "555555555555"
           }
         }
       },
       {
         "Sid": "s3_policy", //This field accepts string values
         "Effect": "Allow",
         "Principal": {
           "Service": "s3.amazonaws.com"
         },
         "Action": [
           "SNS:Publish"
         ],
         "Resource": "arn:aws:sns:us-west-2:555555555555:notify-topic", //notify-topic is the topic name
         "Condition": {
           "StringEquals": {
             "aws:SourceAccount": "555555555555" //This is the Account ID
           },
           "ArnLike": {
             "aws:SourceArn": "arn:aws:s3:*:*:testbucket /*testbucket is the s3 bucket name. You will get notifications only when file is uploaded to this
       bucket.*/
   "
           }
         }
       }
     ]
   }
   

6. Expand the Delivery retry policy (HTTP/S) section to configure how Amazon SNS retries failed message delivery attempts. This step is optional.
7. Expand the Delivery status logging section to configure how Amazon SNS logs the delivery of messages to CloudWatch. This step is optional.
8. Expand Tags section to add metadata tags to the topic. This step is optional.
9. Select Create topic.
10. The topic's Name, ARN (Amazon Resource Name), and Topic owner's AWS account ID are displayed in the Details section.
11. Copy the topic ARN to the clipboard.

Step 6: Enable and configure event notifications using the Amazon S3 console

Where: Amazon S3 Management console

You can enable Amazon S3 bucket events to send a notification message to a destination whenever those events occur. You configure event notifications for your S3 bucket to notify OCI when there is an update or new data available to load. The following steps explain the procedure to be followed in Amazon S3 console to enable event notifications.

1. Log in to Amazon S3 Management console and sign in as an IAM (Amazon Identity and Access Management) user.
2. In the Buckets list, select the name of the bucket i.e. testbucket. This is the bucket that you had created in Step 1: Create your object store bucket in Amazon S3.
3. Select Properties icon.
4. Navigate to the Event Notifications section and select Create event notification icon.
5. In the General configuration section, specify the following values for event notification.
   • Event name: bucket-notification
   • Prefix: This value is to filter event notifications by prefix. It is an optional value. This is added to filter event activity.
   • Suffix: This value is to filter event notifications by suffix. It is an optional value. This is added to filter event activity.
6. In the Event types section, select one or more event types that you want to receive notifications for. If you are unsure of what event types to pick, then select the All object create events option.
7. In the Destination section, select SNS Topic as the event notification destination.
   Note
   
   Before you can publish event notifications, you must grant the Amazon S3 the necessary permissions to call the relevant API. This is so that it can publish notifications to a Lambda function or an SNS topic.
8. After you select SNS topic as the event notification destination, select the SNS topic i.e. notify-topic from dropdown. This is the topic you created in Step 5: Create a notifications service subscription topic.
9. Select Save changes.

Step 7: Create a notifications service subscription

Where: Amazon SNS console

Every Amazon SNS topic has a set of subscriptions. Once a message is published to a topic, SNS handles distributing the message to all its subscribers. The subscribers can be AWS Lambda functions, HTTP(S) endpoints, email addresses and mobile phone numbers capable of receiving SMS messages.

Amazon SNS matches the topic to a list of subscribers who have subscribed to that topic and delivers the message to each of those subscribers.

1. Log in to Amazon SNS console.
2. In the left navigation pane, select Subscriptions.
3. Select Create subscription on the subscriptions page.
4. In the Details section of the Create subscription page, specify the following values.
   • Topic ARN: Paste the ARN value copied from previous step (Step 5: Create a notifications service subscription topic).
   • Protocol: HTTPS
   • Endpoint: Paste the endpoint value you copied while creating the live table feed in previous step (Step 4: Create and configure a live table feed to use notifications and copy the notification URL).
5. Expand the Subscription filter policy section to configure a filter policy. This step is optional.
6. Expand the Redrive policy (dead-letter queue) section to configure a dead-letter queue for the subscription. This step is optional.
7. Select Create subscription.

Step 8: Confirm that the endpoint can receive notifications

Where: Database Actions: Live Feeds page

1. Return to the Database Actions Live Feeds page and find the card for the live table feed you are configuring for a notification-based feed.
2. Click the Actions (three vertical dots) icon on the card, and select Show Confirmation URL.
3. In the Confirmation URL dialog box, click the link to confirm the URL. This does not close this dialog box. If the link is successful, a message is displayed that confirms the subscription is active.
4. Return to the Confirmation URL dialog box and select the Check only when the cloud store confirmation process is complete check box, and click OK.

Once you finish the above steps, any new files uploaded to the bucket will automatically be loaded into the live table feed table.

For more information on how to enable and configure event notifications using the Amazon S3 console, refer Enabling and configuring event notifications using the Amazon S3 console.

Step 1: Create a resource group in Microsoft Azure

Where: Microsoft Azure Portal

Step 2: Create a storage account in Microsoft Azure

Where: Microsoft Azure Portal

An Azure storage account contains all your storage data objects like blobs, tables, disks etc. You can create a storage account inside the resource group. It provides a unique namespace for your data. To create a storage account, do the following:

1. From the left portal menu, select Storage accounts to display a list of your storage accounts.
2. On the Storage accounts page, click the Create icon.
3. On the Basic tab, provide the following information for your storage account.
   • Subscription: Microsoft Azure Enterprise
   • Resource group: resource-group
   • Storage account name: teststorage
   • Region: Select your location, such as US west.
   • Redundancy: Locally-redundant storage (LRS)
4. You can select Review+create to accept the default options and proceed to validate the account.
5. After the validation passes, you can proceed to click on Create storage account. In case the validation fails, the portal indicates which settings must be modified.

Step 3: Create access Keys

Where: Microsoft Azure Portal

You must grant Microsoft Azure the permissions necessary to obtain access keys on your storage locations. The access keys specific to the storage account are generated automatically after the storage account is created in the previous step. The following steps describes the procedure to create access keys.

1. In Security+Networking , select Access keys. Your account access keys appear with the complete connection string for each key.
2. Select Show keys to show your access keys and connection string for each key and to copy values.
3. Copy the connection string value under key1. This value will be pasted in Azure storage account access key text field of next step (Step 5: Add cloud storage using Microsoft Azure cloud store).
4. Copy the storage account name i.e. teststorage and paste it in Azure storage account name text field of next step (Step 5: Add cloud storage using Microsoft Azure cloud store).
5. Test the credentials to see if it works or not.

Step 4: Create a container

Where: Microsoft Azure Portal

A container is a location (also known as buckets in Amazon S3 and OCI) which holds Azure Blob (Binary large object) storage. Follow these steps to create a container.

1. Navigate to your new storage account in the Azure portal.
2. In the left menu for the storage account, scroll to the Data storage section, then select Containers.
3. Click the +Container icon.
4. Enter the name for your new container. The container name must be lowercase, must start with a letter or number, and can include only letters, numbers, and the dash character.
5. Set the level of Public Access Level to Private. The default level is Private.
6. Select Create to create the container.

Step 5: Add cloud storage using Microsoft Azure cloud store

Where: Database Actions: Manage Cloud page

1. Click the Manage Cloud Store button at the top of the page to go to the Manage Cloud page. For further instructions on adding source files residing in cloud storage provided by Microsoft Azure cloud storage, refer to Create an Microsoft Azure Cloud Storage Link topic in Managing Connections section.

Step 6: Create and configure a live table feed to use notifications and copy the notification URL

Where: Database Actions: Live Feeds page

The Live table feed object enables data to be loaded from Microsoft Azure cloud storage with no polling delay. This object creates an integration between Oracle Cloud Interface and Microsoft Azure.

You can configure a new or an existing live table feed to use notifications:

1. Go to the Database Actions Live Feeds page, as described in Feeding Data.
2. Create or edit a live table feed object, as described in Create a Live Table Feed Object or Edit a Live Table Feed Object. Select the Enable for Notification option
3. Click Create or Save.
4. Click the Actions (three vertical dots) icon on the card for your live feed, and select Show Confirmation URL.
5. In the Notification URL dialog box, click the Copy icon to copy the URL to the clipboard. You may want to copy it to a temporary file, so you can retrieve it later. You will use this URL in the subsequent step, (Step 8: Create Event subscription).

Step 7: Enable Event Resource Provider

Where: Microsoft Azure Portal

If this is the first time you are using Event Grid, you must enable Event Grid resource provider.

1. Select Subscriptions on the left menu.
2. Select the subscription you are using for Event Grid i.e. Microsoft Azure Enterprise.
3. On the left menu, under Settings, select Resource Providers.
4. Search Microsoft.EventGrid.
5. Select Register.

It takes a minute for the registration to finish.

Step 8: Create Event subscription

Where: Microsoft Azure Portal

You create an Event Subscription by configuring the subscription and specifying the endpoint that will receive the notifications.

1. Select the storage account you created in Step 2: Create a storage account in Microsoft Azure.
2. Select the Events icon in the left navigation pane.
3. Click on +Event Subscription.

The Create Event Subscription window appears.

1. Specify the following fields in Event Subscription details section:
   • Name: Eventssub. This is the name of the Event Subscription we create.
   • Event Schema: Event Grid Schema
2. Specify the following fields in Topic Details section:
   • Topic Type: Storage account
   • System Topic Name: eventtopic.
3. Specify the following fields in Event Types section:
   • Event Type: MicrosoftStorage.BlobCreated
4. Specify the following fields in Endpoint Details section:
   • Endpoint Type: Web Hook
   • Endpoint: Paste the Notification URL copied in Step 6: Create and configure a live table feed to use notifications and copy the notification URL.
5. Select Create.

This way Microsoft Azure creates a system topic first and then the Event Subscription for the topic.

Step 9: Confirm that the endpoint can receive notifications

Where: Database Actions: Live Feeds page

1. Return to the Database Actions Live Feeds page and find the card for the live table feed you are configuring for a notification-based feed as created in Step 6: Create and configure a live table feed to use notifications and copy the notification URL.
2. Click the Actions (three vertical dots) icon on the card, and select Show Confirmation URL.
3. In the Confirmation URL dialog box, click the link to confirm the URL. This does not close this dialog box. If the link is successful, a message is displayed that confirms the subscription is active.
   Note
   
   The Confirmation URL link expires after a few minutes. You must make sure that you click the link before it expires.
4. Return to the Confirmation URL dialog box and select the Check only when the cloud store confirmation process is complete check box, and click OK.

You should be able to view logs for the blob uploaded to the Live Feed table from the Microsoft Azure storage in Live table feed Run Details window.

For more details on how to create a topic and subscription in Azure portal, refer to Azure Event Grid Notifications.