Creating a Connector with a Queue Source
Create a connector in Connector Hub to transfer messages from a queue in the Queue service to a target service.
For more information about the Queue service, see Queue.
A connector that's defined with a Queue source and (optional) function task supports the following targets: Functions, Notifications, Object Storage, and Streaming. For an example of the Connector Hub workflow, see Overview of Connector Hub.
Long Polling
By default, connectors use long polling to read messages from queues. Long polling is useful to prevent tight loop retries when the queue is empty. The read timeout for reading from a queue is 30 seconds. In each move operation, the connector waits for consumable messages to be available in the queue. If the read timeout passes and no messages are available for consumption, then the connector ends the request and retries with another request.
Filters and Consumed Messages
You can filter a queue for specific channels. For API and CLI parameters to filter a queue, see Consuming Messages from a Channel. A message that has been transferred to the connector's target is considered "consumed." To meet requirements of the Queue service, the connector deletes transferred messages from the source queue. For more information, see Consuming Messages.
Retention Period: Queue Source
The retention period for the Queue source in Connector Hub depends on the queue configuration. See Creating a Queue. For more information about delivery, see Delivery Details.
The creation process begins, and its progress is displayed. On completion, the connector's details page opens. Use the oci sch service-connector create command and required parameters to create a connector with a Queue source:
oci sch service-connector create [...] --source <queue_json_input>
To pass source details to CLI using a JSON file:
oci sch service-connector create [...] --source file:queue_source.json
Example JSON file contents:
{ "kind": "plugin", "pluginName": "QueueSource", "configMap": { "queueId": "<queue_json_input>" } }
For a complete list of parameters and values for CLI commands, see the CLI Command Reference.
Run the CreateServiceConnector operation to create a connector.
To create a connector with a Queue source, populate
source
in the request (CreateServiceConnectorDetails) with a connector plugin that uses Queue (pluginName
QueueSource
, with each queue OCID inconfigMap
). For an example, see PluginSourceDetails.
Confirm That the New Connector Moves Data
After you create the connector, confirm that it's moving data.
- Enable logs for the connector to get details on data flow.
- Check for expected results at the target service.
Confirming that data is moved helps you avoid automatic deactivation, which happens when a connector fails for a long time.