Class: OCI::GenerativeAiInference::GenerativeAiInferenceClient

Inherits:
Object
  • Object
show all
Defined in:
lib/oci/generative_ai_inference/generative_ai_inference_client.rb

Overview

OCI Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation, summarization, and text embeddings.

Use the Generative AI service inference API to access your custom model endpoints, or to try the out-of-the-box models to chat, generate text, summarize, and create text embeddings.

To use a Generative AI custom model for inference, you must first create an endpoint for that model. Use the Generative AI service management API to create a custom model by fine-tuning an out-of-the-box model, or a previous version of a custom model, using your own data. Fine-tune the custom model on a fine-tuning dedicated AI cluster. Then, create a hosting dedicated AI cluster with an endpoint to host your custom model. For resource management in the Generative AI service, use the Generative AI service management API.

To learn more about the service, see the Generative AI documentation.

Important: The IP addresses behind each DNS endpoint might change over time. Always use the DNS hostname listed under the following API Endpoints section and avoid using hard-coded fixed IP addresses.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(config: nil, region: nil, endpoint: nil, signer: nil, proxy_settings: nil, retry_config: nil) ⇒ GenerativeAiInferenceClient

Creates a new GenerativeAiInferenceClient. Notes: If a config is not specified, then the global OCI.config will be used.

This client is not thread-safe

Either a region or an endpoint must be specified. If an endpoint is specified, it will be used instead of the region. A region may be specified in the config or via or the region parameter. If specified in both, then the region parameter will be used.

Parameters:

  • config (Config) (defaults to: nil) —

    A Config object.

  • region (String) (defaults to: nil) —

    A region used to determine the service endpoint. This will usually correspond to a value in Regions::REGION_ENUM, but may be an arbitrary string.

  • endpoint (String) (defaults to: nil) —

    The fully qualified endpoint URL

  • signer (OCI::BaseSigner) (defaults to: nil) —

    A signer implementation which can be used by this client. If this is not provided then a signer will be constructed via the provided config. One use case of this parameter is instance principals authentication, so that the instance principals signer can be provided to the client

  • proxy_settings (OCI::ApiClientProxySettings) (defaults to: nil) —

    If your environment requires you to use a proxy server for outgoing HTTP requests the details for the proxy can be provided in this parameter

  • retry_config (OCI::Retry::RetryConfig) (defaults to: nil) —

    The retry configuration for this service client. This represents the default retry configuration to apply across all operations. This can be overridden on a per-operation basis. The default retry configuration value is nil, which means that an operation will not perform any retries



63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 63

def initialize(config: nil, region: nil, endpoint: nil, signer: nil, proxy_settings: nil, retry_config: nil)
  # If the signer is an InstancePrincipalsSecurityTokenSigner or SecurityTokenSigner and no config was supplied (they are self-sufficient signers)
  # then create a dummy config to pass to the ApiClient constructor. If customers wish to create a client which uses instance principals
  # and has config (either populated programmatically or loaded from a file), they must construct that config themselves and then
  # pass it to this constructor.
  #
  # If there is no signer (or the signer is not an instance principals signer) and no config was supplied, this is not valid
  # so try and load the config from the default file.
  config = OCI::Config.validate_and_build_config_with_signer(config, signer)

  signer = OCI::Signer.config_file_auth_builder(config) if signer.nil?

  @api_client = OCI::ApiClient.new(config, signer, proxy_settings: proxy_settings)
  @retry_config = retry_config

  if endpoint
    @endpoint = endpoint + '/20231130'
  else
    region ||= config.region
    region ||= signer.region if signer.respond_to?(:region)
    self.region = region
  end
  logger.info "GenerativeAiInferenceClient endpoint set to '#{@endpoint}'." if logger
end

Instance Attribute Details

#api_client ⇒ OCI::ApiClient (readonly)

Client used to make HTTP requests.

Returns:



23
24
25
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 23

def api_client
  @api_client
end

#endpoint ⇒ String (readonly)

Fully qualified endpoint URL

Returns:

  • (String) 


27
28
29
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 27

def endpoint
  @endpoint
end

#region ⇒ String

The region, which will usually correspond to a value in Regions::REGION_ENUM.

Returns:

  • (String) 


37
38
39
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 37

def region
  @region
end

#retry_config ⇒ OCI::Retry::RetryConfig (readonly)

The default retry configuration to apply to all operations in this service client. This can be overridden on a per-operation basis. The default retry configuration value is nil, which means that an operation will not perform any retries

Returns:



33
34
35
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 33

def retry_config
  @retry_config
end

Instance Method Details

#apply_guardrails(apply_guardrails_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use apply_guardrails API.

Applies guardrails to the input content, including content moderation, PII detection, and prompt injection protection. Case 1: Use input when the customer wants simple single-text moderation. Existing customers can continue to use this field without changing their current integration. Case 2: Use multimodalInput when the customer wants moderation over text, image, or a combination of both. multimodalInput supports a single text item, an array of text items only, an array of images only, or a mixed ordered combination of text and image items. Clients may provide input, multimodalInput, or both. At least one of these fields must be provided. If both input and multimodalInput are provided, the service will process input and discard multimodalInput.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 134

def apply_guardrails(apply_guardrails_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#apply_guardrails.' if logger

  raise "Missing the required parameter 'apply_guardrails_details' when calling apply_guardrails." if apply_guardrails_details.nil?

  path = '/actions/applyGuardrails'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(apply_guardrails_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#apply_guardrails') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::ApplyGuardrailsResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#chat(chat_details, opts = {}, &block) ⇒ Response

Note:

Click here to see an example of how to use chat API.

Creates a response for the given conversation.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 195

def chat(chat_details, opts = {}, &block)
  logger.debug 'Calling operation GenerativeAiInferenceClient#chat.' if logger

  raise "Missing the required parameter 'chat_details' when calling chat." if chat_details.nil?

  path = '/actions/chat'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json, text/event-stream'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(chat_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#chat') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::ChatResult',
      &block
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#embed_text(embed_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use embed_text API.

Produces embeddings for the inputs.

An embedding is numeric representation of a piece of text. This text can be a phrase, a sentence, or one or more paragraphs. The Generative AI embedding model transforms each phrase, sentence, or paragraph that you input, into an array with 1024 numbers. You can use these embeddings for finding similarity in your input text such as finding phrases that are similar in context or category. Embeddings are mostly used for semantic searches where the search function focuses on the meaning of the text that it's searching through rather than finding results based on keywords.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 259

def embed_text(embed_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#embed_text.' if logger

  raise "Missing the required parameter 'embed_text_details' when calling embed_text." if embed_text_details.nil?

  path = '/actions/embedText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(embed_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#embed_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::EmbedTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#generate_text(generate_text_details, opts = {}, &block) ⇒ Response

Note:

Click here to see an example of how to use generate_text API.

Generates a text response based on the user prompt.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 320

def generate_text(generate_text_details, opts = {}, &block)
  logger.debug 'Calling operation GenerativeAiInferenceClient#generate_text.' if logger

  raise "Missing the required parameter 'generate_text_details' when calling generate_text." if generate_text_details.nil?

  path = '/actions/generateText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json, text/event-stream'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(generate_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#generate_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::GenerateTextResult',
      &block
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#list_guardrail_versions(opc_compartment_id, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use list_guardrail_versions API.

List the available guardrail system versions.

Parameters:

  • opc_compartment_id (String) —

    The client compartment ID.

  • opts (Hash) (defaults to: {}) —

    the optional parameters

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_request_id (String) —

    The client request ID for tracing.

  • :state (String) —

    A filter to return only the guardrail versions whose state matches the given value.

  • :limit (Integer) —

    For list pagination. The maximum number of results per page, or items to return in a paginated "List" call. For important details about how pagination works, see List Pagination. (default to 10)

  • :page (String) —

    For list pagination. The value of the opc-next-page response header from the previous "List" call. For important details about how pagination works, see List Pagination.

Returns:



386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 386

def list_guardrail_versions(opc_compartment_id, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#list_guardrail_versions.' if logger

  raise "Missing the required parameter 'opc_compartment_id' when calling list_guardrail_versions." if opc_compartment_id.nil?

  if opts[:state] && !OCI::GenerativeAiInference::Models::GuardrailVersion::STATE_ENUM.include?(opts[:state])
    raise 'Invalid value for "state", must be one of the values in OCI::GenerativeAiInference::Models::GuardrailVersion::STATE_ENUM.'
  end

  path = '/guardrailVersions'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}
  query_params[:state] = opts[:state] if opts[:state]
  query_params[:limit] = opts[:limit] if opts[:limit]
  query_params[:page] = opts[:page] if opts[:page]

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-compartment-id'] = opc_compartment_id
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf

  post_body = nil

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#list_guardrail_versions') do
    @api_client.call_api(
      :GET,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::GuardrailVersionCollection'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#logger ⇒ Logger

Returns The logger for this client. May be nil.

Returns:

  • (Logger) —

    The logger for this client. May be nil.



102
103
104
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 102

def logger
  @api_client.config.logger
end

#rerank_text(rerank_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use rerank_text API.

Reranks the text responses based on the input documents and a prompt.

Rerank assigns an index and a relevance score to each document, indicating which document is most related to the prompt.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 455

def rerank_text(rerank_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#rerank_text.' if logger

  raise "Missing the required parameter 'rerank_text_details' when calling rerank_text." if rerank_text_details.nil?

  path = '/actions/rerankText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(rerank_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#rerank_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::RerankTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end

#summarize_text(summarize_text_details, opts = {}) ⇒ Response

Note:

Click here to see an example of how to use summarize_text API.

Summarizes the input text.

Parameters:

Options Hash (opts):

  • :retry_config (OCI::Retry::RetryConfig) —

    The retry configuration to apply to this operation. If no key is provided then the service-level retry configuration defined by #retry_config will be used. If an explicit nil value is provided then the operation will not retry

  • :opc_retry_token (String) —

    A token that uniquely identifies a request so it can be retried in case of a timeout or server error without risk of executing that same action again. Retry tokens expire after 24 hours, but can be invalidated before that, in case of conflicting operations. For example, if a resource is deleted and purged from the system, then a retry of the original creation request is rejected.

  • :opc_request_id (String) —

    The client request ID for tracing.

Returns:



516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
 
# File 'lib/oci/generative_ai_inference/generative_ai_inference_client.rb', line 516

def summarize_text(summarize_text_details, opts = {})
  logger.debug 'Calling operation GenerativeAiInferenceClient#summarize_text.' if logger

  raise "Missing the required parameter 'summarize_text_details' when calling summarize_text." if summarize_text_details.nil?

  path = '/actions/summarizeText'
  operation_signing_strategy = :standard

  # rubocop:disable Style/NegatedIf
  # Query Params
  query_params = {}

  # Header Params
  header_params = {}
  header_params[:accept] = 'application/json'
  header_params[:'content-type'] = 'application/json'
  header_params[:'opc-retry-token'] = opts[:opc_retry_token] if opts[:opc_retry_token]
  header_params[:'opc-request-id'] = opts[:opc_request_id] if opts[:opc_request_id]
  # rubocop:enable Style/NegatedIf
  header_params[:'opc-retry-token'] ||= OCI::Retry.generate_opc_retry_token

  post_body = @api_client.object_to_http_body(summarize_text_details)

  # rubocop:disable Metrics/BlockLength
  OCI::Retry.make_retrying_call(applicable_retry_config(opts), call_name: 'GenerativeAiInferenceClient#summarize_text') do
    @api_client.call_api(
      :POST,
      path,
      endpoint,
      header_params: header_params,
      query_params: query_params,
      operation_signing_strategy: operation_signing_strategy,
      body: post_body,
      return_type: 'OCI::GenerativeAiInference::Models::SummarizeTextResult'
    )
  end
  # rubocop:enable Metrics/BlockLength
end