Multipart Uploads

For very large objects you must use multipart upload. Under standard S3 behaviour the single-PUT ceiling is 5 GB — this is an AWS S3 limit that Fil One's storage gateway is expected to follow but that we have not independently confirmed (see Limits). Multipart upload is also recommended for any file over 100 MB for improved reliability -- if a part fails, you retry only that part instead of the entire file.

note

The Fil One dashboard uploads objects with a single PUT, so multipart applies when you use your own S3 SDK or the AWS CLI, not the console upload flow.

How it works

1. Initiate the upload and receive an upload ID.
2. Upload parts individually (minimum 5 MB each, except the last part).
3. Complete the upload by listing all parts. The service assembles the final object.

If something goes wrong, abort the upload to clean up incomplete parts.

Example: AWS CLI

The AWS CLI handles multipart uploads automatically for large files:

# The AWS CLI switches to multipart above its own client-side threshold (8 MB by default)
aws s3 cp large-file.zip s3://my-bucket/large-file.zip \
  --endpoint-url https://eu-west-1.s3.filonecontent.com

To control thresholds:

aws configure set s3.multipart_threshold 100MB
aws configure set s3.multipart_chunksize 50MB

Example: Python (boto3)

boto3 also handles multipart automatically via upload_file:

from boto3.s3.transfer import TransferConfig

config = TransferConfig(
    multipart_threshold=100 * 1024 * 1024,  # 100 MB
    multipart_chunksize=50 * 1024 * 1024,   # 50 MB
    max_concurrency=10,
)

s3.upload_file(
    "large-file.zip",
    "my-bucket",
    "large-file.zip",
    Config=config,
)

Limits

Parameter	Value
Maximum object size	5 TB
Maximum parts	10,000
Minimum part size	5 MB (except last part)
Maximum part size	5 GB

note

These are the standard AWS S3 multipart limits. Fil One's storage gateway is expected to follow them, but we have not independently confirmed the exact values — treat them as guidance and test against your own workload before hard-coding a part size. See Limits.

Cleaning up incomplete uploads

Incomplete multipart uploads are generally expected to consume storage until aborted. Fil One has no lifecycle-rule mechanism to expire them automatically (there is no AbortIncompleteMultipartUpload lifecycle support), so you must list and abort stale uploads yourself:

aws s3api list-multipart-uploads \
  --bucket my-bucket \
  --endpoint-url https://eu-west-1.s3.filonecontent.com

aws s3api abort-multipart-upload \
  --bucket my-bucket \
  --key large-file.zip \
  --upload-id YOUR_UPLOAD_ID \
  --endpoint-url https://eu-west-1.s3.filonecontent.com