Listing Streams and Stream Pools
This information describes how to list streams and stream pools.
Required IAM Policy
To use Oracle Cloud Infrastructure, you must be granted security access in a policy by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don't have permission or are unauthorized, verify with your administrator what type of access you have and which compartment to work in.
For administrators: The policy in Let streaming admins manage streaming resources lets the specified group do everything with streaming and related Streaming service resources.
If you're new to policies, see Getting Started with Policies and Common Policies. If you want to dig deeper into writing policies for the Streaming service, see Details for the Streaming service in the IAM policy reference and Accessing Streaming Resources Across Tenancies.
To Get a List of Streams
- Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
A list of existing streams is displayed.
To View Stream Details
Stream details include the Messages endpoint and the stream OCID.
- Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
A list of existing streams is displayed.
- Click a stream to display the stream details page.
To Get a List of Stream Pools
- Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
- Click on Stream Pools on the left side of the screen.
A list of existing stream pools is displayed.
To View Stream Pool Details
Stream pool details include encryption and Kafka Connect settings.
- Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
- Click on Stream Pools on the left side of the screen.
A list of existing stream pools is displayed.
- Click a stream pool to display the stream pool details page.
- Click on Kafka Connection Settings on the left side of
the screen.
The Kafka Connect configuration values are displayed.
- Open the navigation menu and click Analytics & AI. Under Messaging, click Streaming.
For information about using the CLI, see Command Line Interface (CLI). For a complete list of flags and options available for CLI commands, see the Command Line Reference.
Note
The examples in this section use the full syntax for all parameters, for example--compartment-id
. For some parameters, there are shortened versions that you can use instead, like-c
. See the CLI online help for instances of a shortened parameter associated with a command.Tip
Provide input for--custom-encryption-key-details
,--private-endpoint-details
, and--kafka-settings
as valid formatted JSON. See Passing Complex Input and Using a JSON File for Complex Input for information about JSON formatting.To get a list of streams in a compartment
oci streaming admin stream list --compartment-id <compartment_OCID>
For example:
oci streaming admin stream list --compartment-id ocid1.tenancy.oc1..exampleuniqueID { "data": [ { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.stream.oc1.phx.exampleuniqueID", "lifecycle-state": "ACTIVE", "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com", "name": "example_stream_2", "partitions": 1, "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID", "time-created": "2020-08-21T21:19:35.707000+00:00" }, { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.stream.oc1.phx.exampleuniqueID", "lifecycle-state": "DELETED", "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com", "name": "example_stream_1", "partitions": 5, "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID", "time-created": "2020-07-16T20:59:32.904000+00:00" } ] }
By default, getting a list of streams returns up to the first 10 streams in the compartment.
To get a list of streams in a stream pool
oci streaming admin stream list --stream-pool-id <stream_pool_OCID>
For example:
oci streaming admin stream list --stream-pool-id ocid1.streampool.oc1.phx.exampleuniqueID { "data": [ { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.stream.oc1.phx.exampleuniqueID", "lifecycle-state": "ACTIVE", "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com", "name": "example_stream_2", "partitions": 1, "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID", "time-created": "2020-08-21T21:19:35.707000+00:00" }, { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.stream.oc1.phx.exampleuniqueID", "lifecycle-state": "DELETED", "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com", "name": "example_stream_1", "partitions": 5, "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID", "time-created": "2020-07-16T20:59:32.904000+00:00" } ] }
By default, getting a list of streams returns up to the first 10 streams in the stream pool.
To get a list of stream pools
oci streaming admin stream-pool list --compartment-id <compartment_OCID>
For example:
oci streaming admin stream-pool list --compartment-id ocid1.tenancy.oc1..exampleuniqueID { "data": [ { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.streampool.oc1.phx.exampleuniqueID", "is-private": false, "lifecycle-state": "ACTIVE", "name": "MyStreamPool", "time-created": "2020-10-09T22:21:16.931000+00:00" }, { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.streampool.oc1.phx.exampleuniqueID", "is-private": true, "lifecycle-state": "ACTIVE", "name": "example-stream-pool-02", "time-created": "2020-04-03T07:00:56.196000+00:00" }, { "compartment-id": "ocid1.tenancy.oc1..exampleCompartmentOCID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.streampool.oc1.phx.exampleStreamPoolOCID", "is-private": false, "lifecycle-state": "ACTIVE", "name": "example-stream-pool-01, "time-created": "2020-04-03T05:28:26.025000+00:00" } ] }
By default, getting a list of streams returns up to the first 10 streams in the compartment.
To view stream details
oci streaming admin stream get --stream-id <stream_OCID>
For example:
oci streaming admin stream get --stream-id ocid1.stream.oc1.phx.exampleuniqueID { "data": { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "defined-tags": {}, "freeform-tags": {}, "id": "ocid1.stream.oc1.phx.exampleuniqueID", "lifecycle-state": "ACTIVE", "lifecycle-state-details": null, "messages-endpoint": "https://cell-1.streaming.us-phoenix-1.oci.oraclecloud.com", "name": "MyStream", "partitions": 5, "retention-in-hours": 24, "stream-pool-id": "ocid1.streampool.oc1.phx.exampleuniqueID", "time-created": "2020-11-02T19:12:22.385000+00:00" }, "etag": "\"0613d634-86ab-4446-973f-268d175313d4-12e9725e-5574-4f6b-995b-7dcc80271666\"" }
To view stream pool details
oci streaming admin stream-pool get --stream-pool-id <stream_pool_OCID>
For example:
oci streaming admin stream-pool get --stream-pool-id ocid1.streampool.oc1.phx.exampleuniqueID { "data": { "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID", "custom-encryption-key": { "key-state": "NONE", "kms-key-id": null }, "defined-tags": {}, "endpoint-fqdn": "cell-1.streaming.us-phoenix-1.oci.oraclecloud.com", "freeform-tags": {}, "id": "ocid1.streampool.oc1.phx.exampleuniqueID", "is-private": false, "kafka-settings": { "auto-create-topics-enable": false, "bootstrap-servers": "cell-1.streaming.us-phoenix-1.oci.oraclecloud.com:9092", "log-retention-hours": 24, "num-partitions": 1 }, "lifecycle-state": "ACTIVE", "lifecycle-state-details": null, "name": "MyStreamPool", "private-endpoint-settings": { "nsg-ids": null, "private-endpoint-ip": null, "subnet-id": null }, "time-created": "2020-11-02T23:01:59.429000+00:00" }, "etag": "\"6934531c-efaa-40ba-b083-94eb2350d737-a8b10bda-09cc-45e1-800b-b66b4bc29353\"" }
Use the following API operations to list streams and stream pools and retrieve details:
For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.
Using OCI SDKs
Use the listStreams
method to return a list of streams for a given
compartment or stream pool. To get details about a stream, use the
getStream
method and then examine the properties of the stream.
See the Developer Guide to Streaming for detailed SDK examples.
Using Resource Manager
Resource Manager is an Oracle Cloud Infrastructure (OCI) service that allows you to automate the process of provisioning your OCI resources. Using Terraform, Resource Manager helps you install, configure, and manage resources through the "infrastructure-as-code" model.
A Terraform configuration codifies your infrastructure in declarative configuration files. The configuration defines the resources you intend to provision, variables, and specific instructions for provisioning the resources
You can use Resource Manager or the Terraform CLI with the OCI Terraform provider to see how your streams and stream pools are represented in Terraform configuration files.
For more information about writing configurations for use with Resource Manager, see Terraform Configurations for Resource Manager and Terraform Configuration.
Streams
If you're using Resource Manager to manage streams in a
stack, the streams are represented as oci_streaming_stream
resources.
For example:
resource oci_streaming_stream export_example_stream {
compartment_id = var.compartment_ocid
defined_tags = {
}
freeform_tags = {
}
name = "example_stream"
partitions = "1"
retention_in_hours = "24"
#stream_pool_id = <<Optional value not found in discovery>>
}
Listing Streams
You can use the oci_streaming_streams
data source to retrieve the streams
in your compartment if you're using the Terraform provider.
For example:
data "oci_streaming_streams" "test_streams" {
#Optional
compartment_id = var.compartment_id
id = var.stream_id
name = var.stream_name
state = var.stream_state
stream_pool_id = oci_streaming_stream_pool.test_stream_pool.id
}
Stream Pools
If you're using Resource Manager to manage streams in a
stack, the streams are represented as oci_streaming_stream_pool
resources.
For example:
resource oci_streaming_stream_pool export_test_stream_pool {
compartment_id = var.compartment_ocid
name = "test_stream_pool"
#Optional
private_endpoint_settings {
nsg_ids = [oci_core_network_security_group.test_nsg.id]
private_endpoint_ip = "10.0.0.5"
subnet_id = oci_core_subnet.test_subnet.id
}
kafka_settings {
#Optional
auto_create_topics_enable = true
log_retention_hours = 24
num_partitions = 1
}
}
resource "oci_core_vcn" "test_vcn" {
cidr_block = "10.0.0.0/16"
compartment_id = var.compartment_ocid
display_name = "testvcn"
dns_label = "dnslabel"
}
resource "oci_core_subnet" "test_subnet" {
cidr_block = "10.0.0.0/24"
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.test_vcn.id
}
resource "oci_core_network_security_group" "test_nsg" {
compartment_id = var.compartment_ocid
vcn_id = oci_core_vcn.test_vcn.id
}
Stream Details
Terraform provider users can use the oci_streaming_stream
data source to
retrieve the details of a stream.
For example:
data "oci_streaming_stream" "test_stream" {
#Required
stream_id = oci_streaming_stream.stream.id
}
Stream Pool Details
Terraform provider users can use the oci_streaming_stream_pool
data source
to retrieve the details of a stream pool.
For example:
data "oci_streaming_stream_pool" "test_stream_pool" {
#Required
stream_pool_id = oci_streaming_stream_pool.test_stream_pool.id
}