Static S3 Provisioning on EKS: What I Learned

Only static provisioning is supported for S3 storage on EKS, requiring manual PersistentVolume mapping. The Mountpoint for S3 CSI driver bridges object storage and Kubernetes by presenting Amazon S3 buckets as standard POSIX volumes. This architecture lets Nginx containers serve static content directly from S3 buckets, bypassing traditional block storage entirely. We focus on the hard constraints of StorageClass definitions where flexible provisioning remains unavailable, forcing administrators to pre-define PersistentVolumeClaim bindings. IRSA remains the mandatory identity mechanism since Pod Identity support is not yet available for this driver.

We deploy these resources using Terraform modules within a dedicated VPC to ensure network isolation. The CSI driver handles file system-like access while you manage resource quotas like open file descriptors. This approach balances the scalability of Amazon Simple Storage Service with the operational realities of current Kubernetes storage limitations.

The Role of Mountpoint for S3 in Cloud-Native Storage Architectures

The Mountpoint for S3 CSI Driver translates POSIX file operations into S3 object API calls, presenting buckets as filesystems. This mechanism relies on Mountpoint for Amazon S3 technology, which transparently converts standard `open` and `read` commands into `GET` requests against the backend store. Because the driver maps existing objects rather than creating new storage volumes on demand, it strictly supports static provisioning only. Administrators must manually define a PersistentVolume that references a specific, pre-existing bucket before any workload can attach.

Block storage systems allocate capacity dynamically, yet this architecture requires explicit mapping between the PersistentVolume and the target S3 bucket. A PersistentVolumeClaim subsequently binds to this static volume to grant pod access. Security contexts depend entirely on IRSA (IAM Roles for Service Accounts) to authorize these transactions, as Pod Identity remains unsupported for this specific integration.

Automated volume creation workflows common in other StorageClass definitions simply do not exist here. While this enables massive scalability for read-heavy datasets, the lack of flexible provisioning means infrastructure-as-code templates must explicitly declare every bucket-to-volume relationship. You gain strict governance over data access, but initial configuration overhead increases for ephemeral environments. Organizations adopting this pattern should anticipate additional Terraform logic to manage these static bindings across namespaces.

Deploying ReadWriteMany Access for Shared Logs and Backups

Concurrent access to shared S3 buckets by multiple pods occurs using the ReadWriteMany access mode supported by the driver. Organizations adopt this integration to replace traditional NFS for specific workloads, citing lower costs associated with object storage's elastic consumption model compared to pre-provisioned capacity. Unlike block storage, S3 stores data as objects inside buckets, making it ideal for static content, logs, and backups where aggregate throughput matters more than low-latency random writes. The Mountpoint CSI Driver enables this by allowing multiple pods to simultaneously read from and write to the same S3 bucket without changing application code. Architectural decisions often involve weighing AWS S3 egress fees against competitors like the provider, which offers S3-compatible APIs with zero egress costs, particularly for data-intensive backup scenarios. This tension defines the economic boundary where Rabata.io recommends evaluating total cost of ownership beyond simple per-GB pricing.

Atomic rename operations are absent, a factor that impacts applications relying on strict file locking mechanisms despite virtually unlimited scalability.

POSIX Limitations: Missing Atomic Rename and DNS Dependencies

Support for atomic file renaming is fundamentally absent in the Mountpoint for S3 architecture, creating a hard failure mode for applications depending on this logic. This constraint stems from the object storage model where moving data requires copying the object and deleting the source, a process that cannot guarantee atomicity. Developers migrating stateful services must audit code paths for rename operations, as standard POSIX semantics do not map cleanly to the underlying API. The limitation on renaming means workflows relying on atomic swaps for data consistency will fail without significant refactoring.

Network dependencies introduce a second risk factor distinct from block storage protocols. Nodes strictly require unhindered HTTPS traffic on port 443 and reliable DNS resolution to reach the S3 service endpoint. If security groups block this specific port or DNS fails, the entire mount becomes inaccessible, halting pod startup. Unlike local disk failures, this dependency extends the failure domain to the network edge. Operators must verify that DNS resolution remains stable across all cluster nodes to prevent intermittent outages.

Application logic requires validation against these constraints before production deployment, according to Rabata.io.

Inside the Architecture of S3 CSI Driver and IRSA Security Flows

Static Provisioning Workflow for S3 Buckets in EKS

Manual PersistentVolume creation mapped to pre-existing buckets becomes necessary because flexible allocation remains unsupported. Administrators define a PersistentVolume specifying the target bucket name, then bind it via a PersistentVolumeClaim where the `storageClassName` is explicitly set to an empty string `""` to distinguish the workflow from flexible patterns. This configuration sequence ensures the S3 CSI driver attaches the correct object store without attempting automated bucket creation. The process involves defining the storage resources manually:

1. Create an S3 bucket using infrastructure code or the console.
2. Define a PersistentVolume manifest referencing the specific bucket.
3. Set the PersistentVolumeClaim storage class name to `""`.
4. Deploy pods that reference the claim to trigger mounting.

Network security groups must allow outbound HTTPS traffic on port 443, or the Mountpoint for S3 filesystem will fail to initialize due to connectivity timeouts. Static binding offers precise control over data access yet introduces deployment friction where application scaling requires manual PersistentVolume updates rather than automatic expansion.

Configuring IRSA Security Flows for Pod-to-S3 Access

Secure pod communication requires replacing node-level credentials with fine-grained pod-level authentication to eliminate lateral movement risks. This configuration enforces a zero-trust model where individual workloads manage their own access permissions rather than inheriting broad node identity. Implementing this security flow involves binding IAM roles to Kubernetes service accounts through the following steps:

1. Define an IAM role with a trust policy targeting the EKS OIDC provider.
2. Annotate the Kubernetes service account with the created IAM role ARN.
3. Ensure the pod identity is correctly associated with the IAM role for access.

Nodes running the driver must maintain network access to the S3 service via DNS and port 443; security groups and firewall rules must permit this traffic. IRSA is required for pod identity support, as native Pod Identity is not yet supported for this driver.

Atomic Rename Failures in POSIX Translation Layers

Applications expecting atomic file swaps may encounter issues because the Mountpoint for S3 layer cannot execute true rename operations on object storage backends. A process writing to a temporary file and renaming it to production may leave readers with partial data if the client crashes mid-operation.

The limitation on renaming forces developers to audit application logic for patterns assuming instantaneous file replacement. Administrators should watch resource quotas like open file descriptors and network throughput when scaling workloads that rely on these translation layers.

Deploying Encrypted S3 Storage for EKS Using Terraform Modules

Defining IRSA and OIDC Providers for S3 CSI Driver

The `sts:AssumeRoleWithWebIdentity` trust policy functions as the mandatory bridge between Kubernetes service accounts and AWS IAM roles. This mechanism replaces long-lived credentials with flexible tokens issued by the cluster's OIDC provider. The Mountpoint for Amazon S3 CSI driver uses this identity to present an S3 bucket as a storage volume using standard POSIX interfaces. Operators must configure the IAM role trust relationship to explicitly validate the service account `system:serviceaccount:kube-system:s3-csi-driver-sa` against the cluster issuer URL. The first step involves provisioning a Kubernetes cluster inside a dedicated VPC using widely adopted AWS Terraform community modules.

1. Create an IAM role with a trust policy restricting `sts:AssumeRoleWithWebIdentity` to the specific OIDC provider.
2. Attach a policy granting `s3:GetObject` and `s3:ListBucket` permissions scoped to the target bucket prefix.
3. Annotate the Kubernetes service account with the new IAM role ARN to enable automatic token injection.
4. Deploy the Terraform module to orchestrate these resources consistently across environments.

Security granularity often clashes with operational overhead in these configurations. Overly broad prefixes simplify management but increase blast radius if a pod is compromised. This architecture eliminates static key rotation schedules but introduces dependency on cluster DNS resolution for token validation. Pod identity does not support flexible bucket creation, requiring pre-provisioned storage assets.

Provisioning Encrypted S3 Buckets with Versioning via Terraform

Defining the S3 Bucket resource in Terraform establishes the core security posture before any Kubernetes workload attempts mounting. This configuration enforces AES256 encryption at rest and enables versioning to protect against accidental deletion or corruption during high-frequency AI training runs. The following code block demonstrates the mandatory resources required to block all public access and ensure data durability:

The Mountpoint for Amazon S3 CSI driver requires specific permissions to function correctly within a zero-trust network. Versioning adds durability yet increases storage costs if lifecycle policies are not set to expire old object versions. The cost is clear: without versioning, a single corrupted dataset during model training could halt production pipelines entirely. Static provisioning remains the only supported method, requiring the `storageClassName` to be set to an empty string `""` in the claim specification. This manual binding ensures that the persistent volume maps exactly to the pre-secured bucket created here.

Validating IAM Policies and Network Connectivity for Mountpoint

Verify the IAM Policy includes `s3:ListBucket`, `s3:GetObject`, `s3:PutObject`, `s3:AbortMultipartUpload`, and `s3:DeleteObject` before applying Terraform changes. Insufficient permissions will prevent the driver from accessing the required objects.

Confirm the Mountpoint for S3 Add-on deploys as `aws-mountpoint-s3-csi-driver`. Infrastructure as Code via Terraform has become the standard for deploying these complex storage integrations, yet dependency ordering often breaks without explicit module waits.

1. Inspect node security groups to ensure outbound traffic to port 443 remains open for DNS resolution.
2. Validate that firewall rules permit unhindered HTTPS traffic to the S3 service endpoint.
3. Check that network connectivity via DNS functions correctly from within the worker nodes.

Nodes running the driver must have network access to the S3 service, or the POSIX translation layer will hang indefinitely on simple directory listings. The driver requires network connectivity to the S3 service via DNS and port 443, meaning nodes must have security rules that do not block traffic on this specific port. This constraint means blocking traffic on port 443 renders the storage volume inaccessible to applications. Testing connectivity from within the cluster environment is recommended before deploying production workloads to avoid silent data path failures.

Validating Static Provisioning Patterns with Nginx Workloads

Static Provisioning Manifests for s3.csi.aws.com

Defining the StorageClass named s3-csi-sc with the s3.csi.aws.com provisioner establishes the foundation for mounting object storage. This manifest sets volumeBindingMode to Immediate and explicitly disables allowVolumeExpansion, reflecting the immutable nature of bucket capacity. Operators must populate the ${S3_BUCKET_NAME} placeholder to link the Kubernetes resource to the correct AWS backend. Because flexible creation is unsupported, a PersistentVolume requires manual definition with a fixed capacity reference, typically set to 1Gi for metadata tracking rather than actual limit enforcement.

The associated PersistentVolumeClaim must specify an empty string for `storageClassName` to successfully bind to the pre-existing volume, a strict requirement for static provisioning workflows. This configuration enables an Nginx deployment to serve content directly from the bucket, effectively decoupling compute from storage layers. The pattern supports ReadWriteMany access for shared datasets. Applications relying on move-based persistence logic will fail due to a lack of atomic rename operations. Operators gain massive scalability for static assets but lose the file-locking guarantees present in traditional block storage systems.rabata.io recommends this architecture specifically for high-throughput media streaming where read-heavy loads outweigh complex write semantics.

Deploying Nginx with S3-Mounted Content via kubectl

Executing the deployment workflow begins by retrieving the S3 bucket name from Terraform outputs to update manifest placeholders. Operators must apply YAML files in a strict sequence: `storage-class`, `persistent-volume`, `persistent-volume-claim`, `nginx-pod`, and `nginx-service`. This order prevents binding errors where the PersistentVolumeClaim fails to locate the manually set PersistentVolume. The nginx-pod.yaml specification runs the `nginx:latest` image as user 0 and group 0, a requirement for the driver to mount the filesystem correctly at `/usr/share/nginx/html`.

Verification proceeds by forwarding local port 8084 to the cluster service, allowing immediate validation that the pod serves content directly from the bucket. This pattern effectively decouples content storage from the compute layer, enabling static website hosting on EKS without managing complex synchronization pipelines. If access denied errors occur, check that the IRSA configuration explicitly trusts the OIDC provider and that security groups allow outbound HTTPS traffic on port.

Operational simplicity conflicts with update latency in this design. Modifying content requires waiting for S3 consistency propagation or restarting the pod to refresh the mount point while the architecture scales read throughput infinitely. Unlike block storage, the Mountpoint for Amazon S3 CSI driver presents objects as files, meaning applications expecting atomic rename operations may encounter unexpected failures during write-heavy workloads.

Rabata.io recommends testing this configuration with read-only workloads first to validate network paths before introducing write operations. Ensuring the StorageClass named `s3-csi-sc` uses the `s3.csi.aws.com` provisioner guarantees compatibility with the installed driver version.

Verification Commands for S3 CSI Driver and Pod Status

Confirm the driver pod in the `kube-system` namespace reaches a `Running` state before troubleshooting access errors. Operators must verify the Mountpoint for Amazon S3 CSI driver installation status using `kubectl get pods -n kube-system` to ensure the controller is active. A common failure mode involves `CrashLoopBackOff` states caused by missing IRSA annotations rather than network blocks.

Execute `aws s3 ls s3://${BUCKET_NAME}` to validate that data written by the pod persists to the backend object store. This step confirms the driver successfully translates POSIX writes into S3 API calls, distinguishing it from legacy tools like s3fs which often struggle with high-throughput container workloads. If the file `index.html` appears in the bucket output, the integration is functional. Teams at Rabata.io recommend automating these checks within CI/CD pipelines to catch configuration drift early. The constraint here is strict: without successful bucket verification, the application serves stale or missing content despite a healthy pod status.