Class: OCI::ObjectStorage::Transfer::UploadManager

Inherits:
Object
  • Object
show all
Defined in:
lib/oci/object_storage/transfer/upload_manager.rb

Overview

UploadManager simplifies interaction with the Object Storage service by abstracting away the method used to upload objects. Depending on the configuration parameters, UploadManager may choose to do a single put_object request, or break up the upload into multiple parts and utilize multi-part uploads.

An advantage of using multi-part uploads is the ability to retry individual failed parts, as well as being able to upload parts in parallel to reduce upload time.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(object_storage_client:, upload_manager_config: OCI::ObjectStorage::Transfer::UploadManagerConfig.new) ⇒ UploadManager

Creates a new UploadManager

then default configuration values are used

Parameters:



30
31
32
33
 
# File 'lib/oci/object_storage/transfer/upload_manager.rb', line 30

def initialize(object_storage_client:, upload_manager_config: OCI::ObjectStorage::Transfer::UploadManagerConfig.new)
  @object_storage_client = object_storage_client
  @upload_manager_config = upload_manager_config
end

Instance Attribute Details

#object_storage_client ⇒ OCI::ObjectStorage::ObjectStorageClient

The client used to interact with the Object Storage service

Returns:



19
20
21
 
# File 'lib/oci/object_storage/transfer/upload_manager.rb', line 19

def object_storage_client
  @object_storage_client
end

#upload_manager_config ⇒ OCI::ObjectStorage::Transfer::UploadManagerConfig

Configuration for the UploadManager

Returns:



23
24
25
 
# File 'lib/oci/object_storage/transfer/upload_manager.rb', line 23

def upload_manager_config
  @upload_manager_config
end

Instance Method Details

#abort(upload_id, namespace_name, bucket_name, object_name, opts = {}) ⇒ Response

Aborts a multipart upload.

Parameters:

  • upload_id (String) —

    The ID of the multipart upload to abort.

  • namespace_name (String) —

    The namespace containing the bucket in which to store the object

  • bucket_name (String) —

    The bucket where we'll upload the object

  • object_name (String) —

    The name of the object in Object Storage

  • opts (Hash) (defaults to: {}) —

    the optional parameters

Options Hash (opts):

  • :opc_client_request_id (String) —

    The client request ID for tracing.

Returns:

  • (Response) —

    A Response object with data of type nil



141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
 
# File 'lib/oci/object_storage/transfer/upload_manager.rb', line 141

def abort(upload_id, namespace_name, bucket_name, object_name, opts = {})
  raise 'An upload_id must be provided' if upload_id.nil?
  raise 'A namespace_name must be provided' if namespace_name.nil?
  raise 'A bucket_name must be provided' if bucket_name.nil?
  raise 'An object_name must be provided' if object_name.nil?

  assembler = OCI::ObjectStorage::Transfer::Multipart::MultipartObjectAssembler.new(
    object_storage_client: @object_storage_client,
    namespace: namespace_name,
    bucket_name: bucket_name,
    object_name: object_name,
    multipart_upload_opts: opts
  )

  assembler.abort(upload_id)
end

#resume(upload_id, namespace_name, bucket_name, object_name, object_io_or_file_path, opts = {}) ⇒ Response

Resumes a multipart upload. Note that the multipart upload part size which has been configured for this UploadManager needs to match the part size of any parts which have previously been uploaded for the given multipart upload.

an opc-multipart-md5 key. For scenarios where the object was uploaded in a single part, the opc-content-md5 key will be present in the headers of the Response.

the exception to check the underlying errors which caused the failure

Parameters:

  • upload_id (String) —

    The ID of the multipart upload to resume.

  • namespace_name (String) —

    The namespace containing the bucket in which to store the object

  • bucket_name (String) —

    The bucket where we'll upload the object

  • object_name (String) —

    The name of the object in Object Storage

  • object_io_or_file_path (String, IO) —

    Either a path to the file to upload, an IO-like object containing the data to upload or $stdin.

  • opts (Hash) (defaults to: {}) —

    the optional parameters

Options Hash (opts):

  • :opc_client_request_id (String) —

    The client request ID for tracing.

Returns:

  • (Response) —

    A Response object with data of type nil. For a multipart upload, the headers of the response will contain

Raises:



104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
 
# File 'lib/oci/object_storage/transfer/upload_manager.rb', line 104

def resume(upload_id, namespace_name, bucket_name, object_name, object_io_or_file_path, opts = {})
  raise 'An upload_id must be provided' if upload_id.nil?
  raise 'A namespace_name must be provided' if namespace_name.nil?
  raise 'A bucket_name must be provided' if bucket_name.nil?
  raise 'An object_name must be provided' if object_name.nil?
  raise 'An object_io_or_file_path must be provided' if object_io_or_file_path.nil?

  assembler = OCI::ObjectStorage::Transfer::Multipart::MultipartObjectAssembler.new(
    object_storage_client: @object_storage_client,
    namespace: namespace_name,
    bucket_name: bucket_name,
    object_name: object_name,
    multipart_part_size: @upload_manager_config.multipart_part_size,
    non_file_io_multipart_part_size: @upload_manager_config.non_file_io_multipart_part_size,
    parallel_process_count: multipart_parallel_process_count,
    multipart_upload_opts: opts
  )
  assembler.io_for_transfer = object_io_or_file_path

  errors = assembler.resume(upload_id)
  raise OCI::Errors::MultipartUploadError('Errors occurred while resuming the multipart upload', errors, upload_id) \
    unless errors.empty?

  assembler.commit
end

#upload(namespace_name, bucket_name, object_name, object_io_or_file_path, opts = {}) ⇒ Response

Uploads an object to Object Storage. The object can be a path to a file, an IO-like object, or $stdin. Depending on the configuration of the UploadManager and the object provided, it may be uploaded in multiple parts. Note that input from $stdin will always be uploaded via a multipart upload.

an opc-multipart-md5 key. For scenarios where the object was uploaded in a single part, the opc-content-md5 key will be present in the headers of the Response.

the exception to check the underlying errors which caused the failure.

Parameters:

  • namespace_name (String) —

    The namespace containing the bucket in which to store the object

  • bucket_name (String) —

    The bucket where we'll upload the object

  • object_name (String) —

    The name of the object in Object Storage

  • object_io_or_file_path (String, IO) —

    Either a path to the file to upload, an IO-like object containing the data to upload or $stdin.

  • opts (Hash) (defaults to: {}) —

    the optional parameters

Options Hash (opts):

  • :if_match (String) —

    The entity tag to match

  • :if_none_match (String) —

    The entity tag to avoid matching. The only valid value is *, which indicates that the request should fail if the object already exists.

  • :opc_client_request_id (String) —

    The client request ID for tracing.

  • :content_length (Integer) —

    The content length of the body. This will be ignored for multipart uploads.

  • :content_type (String) —

    The content type of the object. Defaults to 'application/octet-stream' if not overridden.

  • :content_language (String) —

    The content language of the object.

  • :content_encoding (String) —

    The content encoding of the object.

  • :content_md5 (String) —

    The base-64 encoded MD5 hash of the body. This will be ignored for multipart uploads.

  • :metadata (Hash<String, String>) —

    A hash of string keys to string values representing any custom metadata to be applied to the object.

  • :storage_tier (String) —

    :The storage tier that the object should be stored in. If not specified, the object will be stored in the same storage tier as the bucket.

Returns:

  • (Response) —

    A Response object with data of type nil. For a multipart upload, the headers of the response will contain

Raises:



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
 
# File 'lib/oci/object_storage/transfer/upload_manager.rb', line 62

def upload(namespace_name, bucket_name, object_name, object_io_or_file_path, opts = {})
  raise 'A namespace_name must be provided' if namespace_name.nil?
  raise 'A bucket_name must be provided' if bucket_name.nil?
  raise 'An object_name must be provided' if object_name.nil?
  raise 'An object_io_or_file_path must be provided' if object_io_or_file_path.nil?

  content_length = File.size(object_io_or_file_path) if object_io_or_file_path.is_a?(String)

  unless object_io_or_file_path.is_a?(String)
    content_length = object_io_or_file_path.size if object_io_or_file_path.respond_to?(:size)
    content_length = object_io_or_file_path.stat.size unless object_io_or_file_path.respond_to?(:size)
  end

  # $stdin is always multipart uploaded (since we can't always make a size guarantee). Otherwise, it depends on the UploadManager settings
  # and the size of the object identified by object_io_or_file_path
  return upload_multipart(namespace_name, bucket_name, object_name, object_io_or_file_path, opts) \
    if object_io_or_file_path.eql?($stdin)

  return upload_single(namespace_name, bucket_name, object_name, object_io_or_file_path, opts) \
    unless use_multipart?(content_length)

  upload_multipart(namespace_name, bucket_name, object_name, object_io_or_file_path, opts)
end