Wiring managed object storage (S3/R2) into a Python app

Looking for persistent disk (named-volume mounts, e.g. a cache or on-disk index)? That's deploymill://guides/storage. This page is about object storage (a managed S3-compatible bucket).

You are an agent updating a DeployMill-managed Python app that has just had a managed object-storage bucket provisioned. Object storage is a managed S3-compatible bucket for blobs, distinct from a persistent volume mount (single-host disk) and from the managed database (structured/relational data).

Precondition: the bucket is already provisioned, so these env vars are set in the app's environment and present in the container at boot. Done looks like: the app reads/writes objects through one module-level client and never logs the credentials.

• S3_ENDPOINT: the S3-compatible API endpoint (Cloudflare R2 today).
• S3_REGION: the region (auto for R2).
• S3_BUCKET: the bucket name to read/write.
• S3_ACCESS_KEY_ID / S3_SECRET_ACCESS_KEY: bucket-scoped credentials. Treat the secret as sensitive: never log it or echo it back to a client.
• S3_PREFIX: present only when several apps share one bring-your-own bucket (prefix isolation). It ends in / and is the key prefix your app's objects must stay under. It is absent on the managed R2 default and on a bucket-per-app connection, where the app owns the whole bucket. When it is set, prepend it to every key you read or write — that prefix is the isolation boundary. (The put_object / get_object / delete_object MCP tools below take keys relative to the prefix and add it for you.)

Use object storage for large or long-lived blobs: user-uploaded images/video, datasets, anything you serve to clients. Do not use a volume mount for that. A volume is single-host, fixed-size, and can't be served directly. Use Postgres (database) for structured/relational data, not blobs.

Follow these steps in order. Each step is a concrete file edit, not advice.

1. Add the S3 client

Add boto3 to your dependencies (pyproject.toml):

dependencies = [
  # ...existing...
  "boto3>=1.34",
]

boto3 speaks the S3 API, which R2 implements. There is no R2-specific SDK.

2. Create a single client module

Create app/storage.py:

import os
import boto3
from botocore.config import Config

_required = ("S3_ENDPOINT", "S3_BUCKET", "S3_ACCESS_KEY_ID", "S3_SECRET_ACCESS_KEY")
for _k in _required:
    if not os.environ.get(_k):
        raise RuntimeError(
            f'{_k} is not set. Add storage: {{ provider: "r2" }} to '
            ".deploymill/project.json and run reconcile_project."
        )

BUCKET = os.environ["S3_BUCKET"]

# Empty unless this app shares a bucket with others (prefix isolation). Route
# every key through object_key() so the same code works either way.
PREFIX = os.environ.get("S3_PREFIX", "")


def object_key(key: str) -> str:
    return f"{PREFIX}{key}"


s3 = boto3.client(
    "s3",
    endpoint_url=os.environ["S3_ENDPOINT"],
    region_name=os.environ.get("S3_REGION", "auto"),
    aws_access_key_id=os.environ["S3_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["S3_SECRET_ACCESS_KEY"],
    # R2 (and most non-AWS S3 endpoints) require path-style addressing.
    config=Config(s3={"addressing_style": "path"}),
)

Build the client once at import time, a module-level singleton, not per request.

3. Upload, fetch, and list objects

from app.storage import s3, BUCKET, PREFIX, object_key

def put_object(key: str, body: bytes, content_type: str) -> None:
    s3.put_object(Bucket=BUCKET, Key=object_key(key), Body=body, ContentType=content_type)

def get_object(key: str) -> bytes:
    resp = s3.get_object(Bucket=BUCKET, Key=object_key(key))
    return resp["Body"].read()

def list_objects(prefix: str = "") -> list[str]:
    resp = s3.list_objects_v2(Bucket=BUCKET, Prefix=object_key(prefix))
    # S3 returns full keys; strip PREFIX back off so callers keep app-relative keys.
    return [o["Key"][len(PREFIX):] for o in resp.get("Contents", [])]

4. Serve blobs without leaking credentials

Stream objects back through your own route, or, for large/public media, hand the client a pre-signed GET URL so it fetches directly from the bucket:

def presign_get(key: str, expires_in: int = 300) -> str:
    return s3.generate_presigned_url(
        "get_object", Params={"Bucket": BUCKET, "Key": object_key(key)}, ExpiresIn=expires_in
    )

Managing the bucket from outside the app (list_objects / put_object / get_object / delete_object)

To inspect or manage the bucket without going through the app's own code, an agent can call these MCP tools (all ownership-checked, all deriving the bucket name from the app record (the injected S3_BUCKET env only gates whether storage is provisioned, it's never trusted as the name, DET-823), and none ever returning credentials):

• list_objects({ applicationId, prefix?, limit?, cursor? }) → { ok: true, bucket, objects: [{ key, size }], count, truncated, nextCursor, limit }. Read-only. prefix narrows the listing; limit sizes the page (default 200, max 1000). This is ONE page: truncated: true means more objects exist, and nextCursor is how you reach them — pass it back as cursor unmodified, keeping the same prefix, and loop until nextCursor is null. (The MCP tool, distinct from the in-app list_objects() helper above.)
• put_object({ applicationId, key, content, encoding?, contentType? }) → { ok: true, bucket, key, size, contentType }. Uploads content to key (replacing any existing object), plain text by default, encoding:"base64" for binary. Decoded size ≤ 10 MB. Push larger blobs through the boto3 client above.
• get_object({ applicationId, key }) → { ok: true, bucket, key, size, encoding:"base64", content }. Decode content from base64. Objects over 10 MB return { ok:false, errorCode:"object_too_large" }. Fetch those via boto3.
• delete_object({ applicationId, key }) → { ok: true, bucket, key, deleted }. Deletes the single object at the exact key (never a prefix match). Idempotent (deleted:false when already gone).

Each returns { ok:false, errorCode } on a recoverable precondition. Object storage is a paid capability, so a free Explore org gets upgrade_required (with the upgrade-guide pointer), and an org whose subscription lapsed gets object_storage_locked (data retained 30 days, then deleted). The provisioning/runtime codes are object_storage_not_enabled (reconcile first), object_store_not_configured, storage_connection_unavailable (the app's bring-your-own storage connection was disconnected — re-register it), object_too_large, or an object_store_* provider code. These are for occasional management (seed a fixture, inspect an upload, clear a test blob). High-volume or large-blob traffic still belongs in app code through the boto3 client above.

What NOT to do

• Don't commit credentials. They live only in the app env, injected by reconcile. Never hardcode S3_* values into the repo.
• Don't park blobs on a volume mount. Volumes are single-host and fixed-size. Object storage is the right home for media/datasets.
• Don't rebuild the client per request. One module-level singleton.
• Don't drop the path-style config. R2 needs addressing_style: "path".

After wiring this up, commit via push_files and call deploy.