Storage quotas

Every user has a bounded, tier-derived allowance for the files they keep in their own resources — Free = 10 GiB — that they genuinely cannot exceed by hammering uploads, shown back to them as a "X of Y used" meter in the Resource Explorer's header. The bar and the limit shipped together: a usage number nobody can act on was the reason the display was deferred on its own.

Nothing is scheduled. The counter moves only on a write it is told about — the server charging bytes it wrote itself, Azure telling us a client's blob landed, or one of our own deletions removing one — and an abandoned upload needs no cleanup at all because its hold expires as a predicate rather than as a state some job flips.

Why a plain pre-flight check is not enough

The obvious design — "read storageBytesUsed, if under quota issue the SAS" — does not stop abuse, for two structural reasons:

1. It races. Read-then-check is not atomic. A client firing many upload requests concurrently has them all read the same low counter, all pass, and all upload — overshooting by an unbounded amount.
2. The SAS carries no byte cap. Uploads are a two-step flow (file uploads): the server mints a scoped write SAS, the client PUTs bytes directly to Azure Blob. The server is never in the data path, so it cannot stop a client that declares 1 MB and PUTs 1 GB, and it cannot abort a transfer mid-stream.

This is the difference from Gmail and Drive: their uploads flow through Google's servers, so the server counts bytes as they stream and hard-aborts the instant you cross the line. We structurally cannot abort a direct-to-blob upload — so the check is made part of the write, and what the client declared is never what it is ultimately charged.

How it works

flowchart TD
  client[Client upload] -->|"generateUploadFileSasEntities (declared size)"| sign[Sign write SAS locally]
  sign --> reserve{"Lock user row<br/>used + live holds + declared &lt;= quota?"}
  reserve -->|no| reject["FORBIDDEN — You have run out of storage"]
  reserve -->|yes| ledger[("storageLedger hold<br/>countedBytes = 0<br/>expiresAt = SAS ttl")]
  ledger --> respond[SAS returned to client]
  respond --> put[Client PUTs bytes to Azure Blob]
  put --> created["Storage emits BlobCreated<br/>with contentLength"]
  created --> reconcile["ReconcileStorageLedgerEntry<br/>counter += actual - countedBytes"]
  reconcile --> counter[("users.storageBytesUsed")]
  del["Blob deletion / purge"] -->|"counter -= countedBytes, row dropped"| counter
  save["Server-side writes, no SAS<br/>content · version objects · cloned assets"] -->|"chargeStorageLedgerEntry<br/>row + counter += actual - countedBytes"| counter
  ledger -.->|"expiresAt passes with no BlobCreated —<br/>the hold simply stops counting"| gone(["Nothing runs"])
  reconcile -->|"whose counter moved"| pubsub["Storage Web PubSub hub<br/>group = owner"]
  del -->|"whose counter moved"| pubsub
  pubsub -->|"re-read"| ui
  save -->|"emitStorageUsage — in-process"| ui
  counter -->|storage.readUsage| ui["Usage bar — 3.2 GB of 10 GB"]
  tier[("users.storageTier")] -->|StorageTierQuotaMap| quota[Quota bytes]
  quota --> reserve
  quota --> ui

The counter holds stored bytes and nothing else. users.storageBytesUsed is the sum of what is actually in blob storage for that user. A declaration the client has not yet made good on is not in it — that lives in the ledger and is summed at reserve time.

Reserve. The SAS is signed first — signing is local and touches nothing in Azure — and the reserve runs before the response is returned, so a rejection is a write target the client never receives. The resource upload chokepoint goes through generateReservedUploadFileSasEntities, which mints and reserves in one call, and the one write target whose name the server fixes rather than mints — a resource's staging blob — goes through generateReservedWriteSasUrl, which does the same for that one name: a new upload path cannot hand out write targets nothing accounts for. In one transaction: this user's collectable holds are deleted (pure garbage collection — they never entered the counter); the user row is taken FOR UPDATE, which is what makes concurrent reserves serialize instead of race; the live holds are summed; and storageBytesUsed + pending + declared <= quota decides. Then one storageLedger row per write target, with countedBytes = 0.

A reserve over a name that already has a row takes the row over. A resource's staging blob is one fixed name, so its next large save reserves a name whose row the last one left — settled by BlobCreated, or still a hold. The reserve turns it back into a hold: the new declaredBytes and expiresAt, reconciledAt cleared, and countedBytes kept, because that is what the counter still carries for the blob on that name. The next BlobCreated then corrects the counter by the difference, and a release hands back whatever the row holds. The row it takes over is left out of the pending sum rather than counted beside the new hold, and it is locked with the collection, ahead of the user row, so the takeover keeps the lock order below.

Lock order: storageLedger before users, on every path that touches both. A release deletes the ledger rows and then decrements their owners; a reconcile locks the ledger row and then moves the counter; so the reserve collects expired holds before it takes the user row rather than after, even though it is the one path that could hold the user lock the whole way. Taking the user row first would close a cycle with a concurrent release or reconcile of that user's rows, and Postgres resolves that by aborting one side — a 500 on an upload the user could retry, or a dead-lettered BlobCreated whose bytes are never charged. Nothing decides anything before the user lock: the collection moves no bytes, and the sum and the gate that read it both sit behind it. Within a release, the owners are locked in sorted order for the same reason.

A staged content save charges the content blob alone. A save too large for one request body is uploaded to the resource's staging blob through a reserved write SAS, like any upload, and its commit deletes that blob through deleteStorageBlobs once the content blob is written, so its row is released in the same call and the meter moves by one content blob. A staging BlobCreated that arrives after the release finds no row and charges nothing (file uploads).

The server's own writes charge themselves. A resource's {id}/content.json, a version's object (resource version store), and every asset cloneContentAssets copies for a publish, a duplicate or a restore have no SAS and so no reserve — the server holds the bytes, knows their length, and the client was never in the data path, so there is nothing to gate and nothing to overshoot. chargeStorageLedgerEntry writes the ledger row and applies the same delta a BlobCreated would, which is what makes a save that rewrites the same blob correct the counter instead of growing it. The content blob is charged at its stored length, the zstd frame rather than the JSON it decodes to, which is also the length its BlobCreated reports (compressed JSON blobs). Two consequences worth stating: a save is never rejected for being over quota (the bytes are already stored, and refusing the charge would only make the counter lie — the next upload is what gets refused), and the row is what lets the existing releases hand these bytes back, so purge needs no path of its own. Each clone charges for the source blob's own contentLength — the properties read that already had to say whether the source exists — before the copy rather than after it, and releases that row if the copy then fails. It is the one charge whose figure is a guess about a blob the server did not write, so it is the one that has to be on record before the copy's own BlobCreated can arrive to correct it; a row minted per clone that nothing ever writes or deletes again is also the one place where a charge for a blob that never landed would never be corrected. A version's object is charged after the transaction that took it, and content after the save's, never inside either: the charge locks the ledger row and then the user's, and a transaction held open across that is a second connection waiting on locks the first will not release until it returns.

A charge is provisional, and the container's own event is what settles it. Every write to resource-assets raises BlobCreated — a server-side copy included — and now that the server's writes leave a ledger row, those events find one instead of falling through. So the byte count a charge supplies is a starting position, not the last word: whatever actually landed is what the reconcile writes seconds later, by the same difference-against-countedBytes that makes a redelivery a no-op. A charge therefore claims no position in the blob's write order and passes no sequencer, which is what marks its figure as a guess.

Carrying no position does not make a charge yield to the last event that spoke, though — and a rule that had it yield is what froze the meter. {id}/content.json is rewritten under one name on every save, and each save's event settles that name seconds later from the Functions host. So a charge that stood down once the blob had an event moved the counter on a resource's first save and never again: every save after it was counted only by its own BlobCreated, which is right in postgres, late on screen, and invisible in the request the owner is watching. A charge now lands whatever the row has already seen, and only an event is ranked. What that trades is a charge delayed past a newer write's event putting its own size back — corrected by the next save's charge rather than stranded, because the same name is written again. The one charge whose figure is a guess about a write the server did not make is the clone, and it charges before its copy for exactly this reason, so its own event always lands behind it and measures over it. A future charge path that guesses about a write it makes first would be the one to re-open the question.

Which event is last is a question the events themselves answer, not the order they arrive in. Event Grid delivers at-least-once and in no order, so two writes to one blob name can have their events delivered reversed, and the older size would otherwise be the one left on the counter — permanently, since nothing revisits a blob that is not written again. Being idempotent does not cover this and is a different property: replaying the earlier event is a perfectly well-behaved no-op that still leaves the counter wrong. So the reconcile carries the event's sequencer, Storage's own per-blob ordering value, on the ledger row and drops any event not newer than the one already applied — the general rule in conditional writes. Both delivery orders then converge on the size of the write that actually happened last.

Charge. Storage's own Microsoft.Storage.BlobCreated event carries the stored object's contentLength, and arrives seconds after the PUT. The handler moves the counter by actual − countedBytes and sets countedBytes = actual. That one expression is correct in all three cases at once: the first event adds the whole object, a redelivery computes a zero delta instead of double-counting, and a second upload to the same still-valid write target corrects the counter rather than stranding the old size on it.

Release. Deleting a blob drops its ledger row and decrements its owner by whatever that row was holding — zero if the blob never landed. The row is what carries both the amount and the owner, which makes a redelivered deletion event a no-op and is also the only way a blob can be attributed to an owner at all: a blob name carries the resource, not the uploader.

The delete and the release are one operation (deleteStorageBlobs), waved in chunks of MAX_CONCURRENT_BLOB_DELETIONS, and each wave releases exactly the names it removed — not the set it was handed. That is what makes a partial failure safe: a prefix deletion re-resolves its set from what is still in the container, so a blob an earlier attempt removed is one the redelivery can never name again, and a release keyed on the retry's smaller listing would hold those bytes against their owners forever. Waving also bounds the release statement, whose IN (…) would otherwise expand past what a single Postgres bind message may carry on a directory of tens of thousands of blobs. The failure is rethrown after the wave's release lands, never before it.

Expiry. Past expiresAt — the SAS's own TTL — the write target is dead, so the hold cannot become real: the sum and the in-flight count both read expiresAt > now, so it stops counting the moment the predicate turns false. No release, no job, no compensating write.

Collection is a different, later bound, and the difference is load-bearing. A hold that stopped counting is not a hold that may be deleted: a BlobCreated for a blob that did land can still be inside Event Grid's retry window, and a reconcile that finds no ledger row charges nothing and reports nothing — the bytes would be stored, attributed to nobody, with no dead letter to show for it. So a row is kept until no event naming it can still arrive. Storage checks a SAS against the moment it receives a request, not the moment that request finishes, so the last PUT a write SAS authorizes starts at expiresAt and raises its event whenever the upload completes; Event Grid then retries that event for EVENT_GRID_DELIVERY_TTL_MS. The reserve therefore collects only rows whose expiresAt is older than the delivery window plus an upload-completion allowance — and the allowance is WRITE_SAS_DURATION_MS, the SAS's own lifetime reused, which is a bound already in hand and orders of magnitude beyond what a MAX_FILE_REQUEST_SIZE upload can take. The bound is derived from the subscription's own retryPolicy rather than restated beside it, because two hard-coded hours that drift apart fail silently in exactly this direction. The rows are dropped opportunistically by the next reserve that user makes, which already holds their rows under lock. Only a hold that never entered the counter is collected — countedBytes = 0 — since a row a reserve took over keeps the bytes of the blob it replaced until a release gives them back.

The quota itself is resolved from the tier at read time, never copied onto the user row, so moving a user to a new tier changes their limit instantly with nothing to backfill. Only the usage is stored, because recomputing it means enumerating every {resourceId}/ directory and message-attachment blob — exactly what got the display deferred in the first place.

Why there is no scheduled sweep

The obvious shape for settling holds is a timer: wake every 15 minutes, list the expired rows, probe each blob, and charge or release it. It is the wrong one, and the reasoning generalises:

• The event already exists. Storage emits BlobCreated with the exact byte count on a system topic that is already provisioned (the dead-letter replay subscribes to the same event type). Polling for a fact the platform will push is standing compute spent to learn something late.
• The other half was never an event. "This hold expired" is not something that happens — it is something that becomes true. Written as a WHERE clause it needs nothing to run; written as a job it needs a schedule, a batch cap, a retry policy, and an ordering story against the arrival of BlobCreated.
• The race disappears rather than being handled. With two signals (a persistence-time confirm and an unordered, at-least-once reconcile) a whole state machine of conditional transitions exists purely to survive them arriving out of order. With one signal and a predicate there is no second thing to be out of order with.

The cost of doing it with events instead is one Event Grid subscription per environment on a system topic that already exists — no new Azure resource, one event per upload, and reconciliation latency in seconds rather than up to an hour. What it avoids is a timer function, a batch cap, and two blob requests per expired row.

A lost BlobCreated would leave a blob stored but uncharged. The subscription dead-letters like the others, but its dead letters are quarantined rather than replayed, and that is deliberate: this is the only handler fed by a system topic, and the replay gate quarantines a system-topic event rather than replaying it, for the reasons that page gives — it admits only event types it can route. Nothing could consume a republish of one either — that page says why. The consequence, stated rather than mitigated: those bytes stay uncharged until a usage recompute, the same one the uncounted blobs below need. What the dead letter buys is the record — the quarantine copy keeps Event Grid's diagnostics and the quarantine is logged, so the loss is inspectable instead of silent. The operator remediation for a normal quarantine (moving the blob back to the container root) is a no-op here, the next pass refusing it on the same ground.

The residual gap, and why it is acceptable

Between the PUT and the BlobCreated event, a client that under-declared is holding less space than it is about to use. The size of that gap is not bounded: Azure blob SAS has no upload-length option, and the PUT never passes back through Nitro, so MAX_FILE_REQUEST_SIZE constrains the declaration the client sends us and nothing about the bytes it sends Azure.

What is bounded is the gap's shape. MAX_UNRECONCILED_STORAGE_LEDGER_ENTRIES caps how many under-declared uploads one user can have in flight at once, and BlobCreated charges each one's real size within seconds of its PUT completing — after which the counter is truthful and every further reserve is rejected. So abuse is a single burst, not a sustained drain, and it only ever exhausts their own allowance — never another user's.

The cap counts live holds, which is one per write target and so one per upload — the resource upload chokepoint mints no thumbnail targets, so there is nothing riding alongside a file to distort it. The same number bounds each upload request's files array, because a batch larger than the cap can never pass the reserve however long the client waits.

True Gmail-style hard enforcement would require proxying uploads through our own server so it can count bytes and abort mid-stream. That is rejected: it puts our compute in the data path for every upload — cost, latency, and re-architecting the entire SAS flow — to close a self-inflicted gap that only harms its own author.

Data model

• StorageTier — pg enum, Free only for now. The enum exists so a paid tier is a value add rather than a schema change.
• StorageTierQuotaMap — { [StorageTier.Free]: 10 * GIBIBYTE }, as const satisfies Record<StorageTier, number>.
• users.storageTier + users.storageBytesUsed — the tier and the stored-bytes total. bigint in number mode: 10 GiB is ~1e10, far under the 2^53 safe-integer ceiling.
• storageLedger — one row per write target, keyed (containerName, blobName), and a row is in one of two states rather than one. A reserved row is a hold on a blob that may never exist: the reserve writes it with countedBytes = 0, declaredBytes = what the client said it would upload, expiresAt = the SAS TTL and reconciledAt null. It holds space only through the reserve's pending sum — the counter has not moved for it at all, which is why an expired one can simply stop counting. A settled row is stored bytes: countedBytes is what the counter is carrying for this blob right now, and reconciledAt is when that last moved. BlobCreated settles a hold in place and stamps the sequencer it was ordered by; a server-side charge writes its own row already settled, with declaredBytes = 0, expiresAt already past and no sequencer — it never held space, because its bytes were stored before it ran. A charge and a release find their row on the primary key, so the only index is (userId, reconciledAt), for the two questions a reserve asks about one user — their live holds and their collectable ones. Nothing scans the ledger account-wide.

Existing users start at storageBytesUsed = 0, so for them the gate is advisory until their real usage accumulates — a user already over 10 GiB keeps uploading until enough of their blobs are ledgered. Under-counting only ever under-rejects, so the advisory window never wrongly rejects a legitimate upload. New users are accurate from their first upload.

What is not counted

These blobs stay outside the ledger and are deliberately uncounted:

• Blobs uploaded before this shipped — nothing backfills them (persisted data — latest shape only).
• Room attachments — the quota counts what a user keeps in their own resources, and a room's files belong to the room. Charging them to whoever uploaded tied a personal allowance to what someone gives a shared room, and put a number about chat in the resource explorer. The room-scoped replacement is written up as room attachment quota; until it lands, message uploads are bounded by nothing here.
• Everything outside the two charge paths — avatars, room images and the per-user game-save blobs go through their own writes, and the subscription filter does not even deliver most of their events. A server-side write joins the ledger by calling chargeStorageLedgerEntry; one that does not is uncounted by omission rather than by design, which is the trade each new write path has to make deliberately.

Closing these needs a recompute that lists real object sizes per user. Worth doing once the counter has drifted enough to matter, and it is a one-shot backfill rather than a recurring sweep.

Failure and retry semantics

• Client under-declares: the counter is low for the seconds until BlobCreated lands; the next reserve then rejects. Bounded as above.
• SAS issued, upload abandoned: the hold stops counting at expiresAt and is dropped by the next reserve that user makes past the collection bound above. If bytes did land, BlobCreated charges them and the blob is an orphan under {id}/files that nothing reclaims until purgeResource takes the whole directory (blob lifecycle) — that prefix has no unreferenced-asset sweep, by design, because a restore must be able to hand back a whole resource.
• BlobCreated redelivered: the delta is zero.
• BlobCreated dead-lettered: quarantined for an operator, not replayed — see above. The blob stays uncharged.
• BlobCreated for a blob nothing reserved (a clone, a name that will not percent-decode): a no-op, never an error. The handler tries the raw name, then the decoded one, and a name that cannot be decoded keeps its raw form rather than throwing — a throw would make an unaccounted blob a poison event that retries to the dead letter.
• BlobCreated beats the charge that ledgers its blob: the reconcile finds no row, no-ops, and is never retried — a successful delivery — so the charge, not the event, is what the counter ends up carrying. The figure is still right for the two paths this can happen on: content and a version's object are bytes the server serialized itself, so the charge declares a measurement rather than a guess. The clone is the case where it would not have been, which is why that one charges before its copy instead. What the blob loses is its place in the write order, because a charge stamps no sequencer: until some later event lands on that row, a retrying event for an earlier write of the same name is treated as the first to speak and applied over it. The counter then holds a superseded size until the newest write's event arrives and wins on rank. Nothing is built for it — closing it means reserving a ledger row before every server-side write and releasing it if the write fails, which is the whole hold machinery spent on bytes that were never in doubt, to shorten a window that ends on the next event either way.
• A server-side charge fails after its write committed: the blob is stored and the counter has not moved. The same shape as a dead-lettered BlobCreated and accepted on the same terms — under-counting only over-serves that one user. It self-corrects where the same blob name is written again, which is the case for {id}/content.json: the charge writes an absolute size rather than a delta, so the next save charges it in full — which holds only because a charge is never dropped once that blob has an event. A version's object does not self-correct, because it is content-addressed and write-once: nothing rewrites that name, and a later version with the same content is deduplicated and charges nothing — so only a usage recompute closes it. What is deliberately not built for it is a durable outbox: a byte counter that self-corrects on the next write does not earn a second write path, a queue and a retry policy.
• An expired revision is counted until something evicts it: a revision past its 30 days is gone to every read at once, but its bytes are released only when the resource's next revision take deletes its row, or when the resource is purged (resource snapshots). A resource nobody edits keeps charging its owner for history they can no longer open. Accepted, because releasing it on time would take a sweep, and the bytes are bounded by the history the resource already had.
• A release is missed: the counter stays high, which only under-serves that user (rejects slightly early) — the safe direction. Only a whole wave can be missed, never part of one.

Key files

Notes

• The meter lives in the resource shell's header — on the breadcrumb row, whose trail almost never fills a line, so the readout costs no row of its own — not in the app bar and not on user settings. Storage is what this area spends, so the number sits where uploads happen and on no route that cannot spend it; the app bar would make every page pay for a query about resources. It reads as a bar, then "X of Y used", then the plan, all written out at every viewport and with no tooltip: the reading is what the meter is for, so it is never metadata parked behind a hover a touch device does not have. Being in the shell puts it on every resource page, not only Home.
• The number itself lives in useStorageStore, read once on initial mount and updated in real-time by useStorageSubscribables. Each resource page declares the shell layout in its own template, so the meter is remounted on every navigation inside the explorer — reading the stored value immediately while updates stream into it as they land. The subscription is established before the read is issued, so the store drops a read that a pushed value overtook rather than letting it write the older number back. emitStorageUsage announces the total, and it announces it best-effort: everything it reports is durable before it runs, so a failed read there leaves a stale meter rather than turning a committed charge into a failed call. A clone announces once for the whole content rather than once per asset.
• The counter moves in two processes and is read in one, so it is announced twice over. What the app changes it reports over the storage.onUpdateUsage tRPC subscription, carrying the figure, so a save moves the meter within the request. What the Functions host changes — every BlobCreated that settles a charge or an upload, every blob-deletion wave, and the retention sweep's purge — shares no emitter with the app at all, so it publishes to the Storage Web PubSub hub instead, to a group named for the owner rather than the device: a quota belongs to the account, and every device showing it is stale by the same amount. Without that half the number is right in Postgres and stale on screen until the next full page load, which is what a client upload and every autosave after a blob's first event both looked like. The published message says only whose counter moved and the client re-reads — refetch, so it cannot join the cached answer the change just invalidated. Deriving the quota a second time in the Functions host would put a figure computed on the far side of a process boundary next to the one the gate enforces, and two of those disagree eventually. The reconcile and the release each hand back the owners they actually moved, so a redelivery that computes a zero delta costs nobody a re-read. A deletion announces per wave, as the wave lands, for the same reason it releases per wave: a wave that gives bytes back and then throws would otherwise announce nothing, and the redelivery behind it re-resolves a set that can be empty.
• The resource upload input gained a size per file, matching the message path. It is bounded at the Zod boundary (z.int().positive().max(MAX_FILE_REQUEST_SIZE)) because a negative or non-finite declaration would shrink the pending sum and weaken the gate.
• purgeResource releases its directory by prefix. The purge never enumerates blob names, and the resources row it deletes has no foreign key into the ledger, so without this the rows would be dropped by no one and their bytes held forever.
• The subscription filters with a single subjectBeginsWith — getBlobSubjectPrefix(AzureContainer.ResourceAssets), the one container whose uploads pass through a reserve. The handler agrees with that filter rather than trusting it (STORAGE_BLOB_CONTAINERS), so a filter that drifts wider delivers events the handler drops.
• A blob name reaches the handler through a url path and storage's encoding of it depends on the characters in it. Rather than guess, the raw name is looked up first and the decoded form only if that found no row — through getDecodedUriComponent, since a filename holding a lone % is legal and would otherwise throw.
• The reserve is a service called from the upload chokepoint, not a tRPC middleware. A middleware runs before the handler, and the ledger rows are keyed by blob names the handler is what mints — so a middleware would have had to either pre-mint the names or split the atomic transaction in two.