Overview of File Storage with Lustre
Oracle Cloud Infrastructure (OCI) File Storage with Lustre is a fully managed storage service designed to meet the demands of AI/ML training and inference, and high performance computing needs.
File Storage with Lustre automates deployment, scaling, and maintenance tasks of open source Lustre, enabling you to focus on applications rather than Lustre infrastructure management. OCI deploys and maintains all Lustre server components, including metadata servers, management servers, and storage servers.
For more information about open source Lustre, see Introduction to Lustre.
Features
Key features for File Storage with Lustre include:
- Parallel distributed architecture
- File Storage with Lustre is built on a parallel distributed architecture, enabling efficient to handling of massive data loads. File systems can scale and deliver high aggregate speeds, ensuring rapid data processing and high throughput.
- Fully managed service
- Automated deployment and scaling simplifies management for users, eliminating the complexity traditionally associated with setting up and maintaining Lustre infrastructure components.
- Production-ready for AI and ML
- File Storage with Lustre is capable of handling large quantities of data, making it an ideal choice for organizations looking to leverage big data analytics and sophisticated AI models.
Concepts
- Performance
-
File Storage with Lustre offers four performance tiers:
- 125 MBps (1Gbps) per provisioned TB
- 250 MBps (2 Gbps) per provisioned TB
- 500 MBps (4 Gbps) per provisioned TB
- 1000 MBps (8 Gbps) per provisioned TB
Administrators are responsible for any client-side performance optimizations. They can use Lustre
lfsutilities for performance optimization features such as file striping and progressive file layout (PFL). - Lustre clients
-
Administrators are responsible for deploying Lustre clients in the GPU hosts or other Compute hosts that need access to the file system.
File Storage with Lustre uses Lustre version 2.15.5 and you can use 2.15.5 clients for the following OS:
- Ubuntu 22.04 with 5.15.x kernel
- Oracle Linux 8 with 4.8 Red Hat compatible kernel (RHCK)
- Oracle Linux 9 with 5.14 Red Hat compatible kernel (RHCK)
If you need help with Lustre clients, Contact support.
Availability
Oracle hosts its OCI services in regions and availability domains. A region is a localized geographic area, and an availability domain is one or more data centers in that region. OCI File Storage with Lustre is hosted in the following regions:
| Region Name | Region Identifier | Region Key |
|---|---|---|
| Australia Southeast (Melbourne) | ap-melbourne-1 | MEL |
| Japan Central (Osaka) | ap-osaka-1 | KIX |
| Germany Central (Frankfurt) | eu-frankfurt-1 | FRA |
| UK South (London) | uk-london-1 | LHR |
| US East (Ashburn) | us-ashburn-1 | IAD |
| US West (Phoenix) | us-phoenix-1 | PHX |
| US West (San Jose) | us-sanjose-1 | SJC |
Some regions don't offer File Storage with Lustre.
See About Regions and Availability Domains for a complete list of available OCI regions, along with associated locations, region identifiers, region keys, and availability domains.
Authentication and Authorization
Each service in Oracle Cloud Infrastructure integrates with IAM for authentication and authorization for management of OCI resources, for all interfaces (the Console, SDK or CLI, and REST API).
An administrator in your organization needs to set up groups, compartments , and policies that control which users can access which services, and which resources, and the type of access they have. For example, policies control who can create users, groups, and compartments, or who can create and manage virtual deployments.
- If you're a new administrator, see Getting Started with Policies.
- For details about writing policies for this service, see File Storage with Lustre Policies.
- For details about writing policies for resources in other services, see Policy Reference.
Security
For security information and recommendations, see Securing Oracle Cloud Infrastructure File Storage with Lustre.
Ways to Access Oracle Cloud Infrastructure
You can access File Storage with Lustre resources by using the Console (a browser-based interface), Oracle Cloud Infrastructure CLI, or REST APIs. To access the Console, you must use a supported browser.
Instructions for all three methods are included throughout this guide.
- The OCI Console is an easy-to-use, browser-based interface. To access the Console, you must use a supported browser.
- The REST APIs provide the most functionality, but require programming expertise. API Reference and Endpoints provide endpoint details and links to the available API reference documents including the File Storage with Lustre APIs.
- OCI provides SDKs that interact with File Storage with Lustre.
- The Command Line Interface (CLI) provides both quick access and full functionality without the need for programming.
- To use the OCI CLI or REST APIs, you can either set up your environment, or use Oracle Cloud Infrastructure Cloud Shell.
- To use the CLI or REST APIs in Cloud Shell, sign in to the Console. See Using Cloud Shell and the CLI Command Reference.
- To install the OCI CLI in your environment, follow the steps in the Install CLI Quickstart.
- When using REST APIs, refer to REST API documentation and API Reference and Endpoints.
Limits on File Storage with Lustre Resources
See File Storage with Lustre Limits and Service Limits for a list of applicable limits and instructions for requesting a limit increase.
To set compartment-specific limits on file systems, administrators can use Overview of Compartment Quotas.