Count it. Check it. Hold it. A person decides.
A refund that follows a scan pays out for boxes that are not complete. ReturnWitness puts a counted record between the scan and the money, so an incomplete Creator Kit ($340) is held at the bench and a person approves what goes out.
Six jobs, in order. A model can add a hold; only a person releases money.
Each number on a step is measured on my two filmed returns (d1, the lens missing; d2, a complete kit) and says where it comes from; a step with nothing measured shows nothing. The props are stand-ins: a real compact film camera for the kit's interchangeable-lens body and a lens-shaped mug for the zoom lens. Every model answer shown is a replay, with its loss lines beside it.
- SAM 3.1 tracksSAM 3.1 · Meta
- App code countsTray counter · app code
- Readers read, and can be wrongSpark · auditor · Glimmer
- A disagreement can hold, never clearMerge · app code
- Muse Wonder draftsMuse Wonder 1.0 · Meta
- A person approves; Stripe refundsA person · Stripe test mode
d2, the complete kit: all three counted → HELD: false hold. The open-weight auditor called the real compact film camera a toy, and that one doubt was enough, because any reader may add a hold.
No model can release it. Only a person's single-use approval can, and nothing refunds until then.
Job 01: SAM 3.1 tracks
Temporal evidence: a mask and a track ID for each item in every frame, so the counter can see where each item goes and when it stops.
It never counts and never decides.
On this public serverThe filmed clips d1 and d2 replay their recorded SAM 3.1 run (masks and text only, no footage):
RECORDED · replay of a recorded real run, not a live callOther clips replay a synthetic scene:
RECORDED FIXTURE · synthetic scene, not a live Meta call- d1 · 21 s clip, lens missing
- 14.8 s
- measured: live SAM 3.1 call, 6 concepts in parallel, 14789 ms
- d2 · 27 s clip, complete kit
- 20.8 s
- measured: live SAM 3.1 call, 6 concepts in parallel, 20765 ms
Job 02: App code counts
An item counts once, when it settles in the inspected tray. It is built not to add a count for hands, kraft paper, lifted-and-replaced items or re-issued track IDs, and passes the synthetic stress sweep; on the recordings this site replays (my two filmed returns, and gate clips g0 and g1) it counted every item right. On g1 it also flagged one mask covering more than one item (a merged-mask hold), and the refund was still recommended for a person to release. The registered gate is incomplete: g2–g8, the clips that test hands, paper and lifts, were not filmed.
A count is presence in the tray, not identity or condition.
On this public serverDETERMINISTIC · app code, no model- d1 · items counted in the tray
- 2 of 3
- Camera body 1/1 · Zoom lens 0/1 · Battery + charger 1/1. Computed by the tray counter on the recorded SAM 3.1 tracks.
- d2 · items counted in the tray
- 3 of 3
- Camera body 1/1 · Zoom lens 1/1 · Battery + charger 1/1. Computed by the tray counter on the recorded SAM 3.1 tracks.
Job 03: Readers read, and can be wrong
Muse Spark 1.3 re-reads the count, crops, slip and policy. The open-weight auditor (Muse Spark Open on Baseten) checks each settled crop for a stand-in or damage. Glimmer (on Modal) transcribes the printed labels. They can disagree with the count and with each other, sometimes wrongly.
On this public serverOn d1 and d2 each reader's recorded answer replays:
RECORDED LIVE RUN · model · req idWith no recorded answer, reconcile runs in app code:
DETERMINISTIC · app code, no modelMuse Spark 1.3 · d1Zoom lens MISSING; every other line OK.
RECORDED LIVE RUN · muse-spark-1.3 · req chatcmpl-01a1a268-95f9-712f-9668-30696f1e5921Muse Spark 1.3 · d2Every line OK.
RECORDED LIVE RUN · muse-spark-1.3 · req chatcmpl-01a1a26b-7840-7286-9061-656b936fbe0aOpen-weight auditor · d1Flagged the camera body as a stand-in: “Label BW-CB-55107 is legible but item is a light-green toy truck, not a camera body.”
Loss misread: the prop is a real compact film camera, not a toy (it stands in for the kit's interchangeable-lens body); a false stand-in flag
RECORDED LIVE RUN · meta-models/muse-spark-open-1026 on BasetenOpen-weight auditor · d2Flagged the camera body as a stand-in: “Label BM-CB-55120 is legible but item is a toy car, not a camera body.”
Loss misread: the prop is a real compact film camera, not a toy (it stands in for the kit's interchangeable-lens body); a false stand-in flag; it held a complete kit (a false hold under the registered scoring)
Loss misread: the label reads BW-CB-55120
RECORDED LIVE RUN · meta-models/muse-spark-open-1026 on BasetenOpen-weight auditor · d2Passed the zoom lens: “Label BW-LN-80311 is clearly legible on the lens with no visible damage.”
Loss missed: the lens prop is a camera-lens-shaped mug; the stand-in check passed it
RECORDED LIVE RUN · meta-models/muse-spark-open-1026 on BasetenGlimmer · d1Read: Camera body serial BW-CB-55107; Battery + charger label unreadable.
RECORDED LIVE RUN · meta-models/Muse-Glimmer-30B on ModalGlimmer · d2No answer in the recorded run, so this replay has no label read from it.
Loss Glimmer timed out on d2 (its 15 s budget): that run lost its label read.
Job 04: A disagreement can hold, never clear
App code merges the count and every reader, most restrictive first. A required reader that fails holds the box; optional readers (the auditor, Glimmer, replays) can only add holds, and one that fails adds nothing: the count and the other readers decide.
A reader can turn OK into HELD. No reader can turn HELD into OK.
On this public serverDETERMINISTIC · app code, no modeld1 · lens missingHELD (zoom lens not counted; Spark: zoom lens MISSING; auditor: camera body stand-in). A correct hold: an item on the slip did not settle in the tray.
Loss misread: the prop is a real compact film camera, not a toy (it stands in for the kit's interchangeable-lens body); a false stand-in flag
DETERMINISTIC · app code, no modeld2 · complete kitHELD (auditor: camera body stand-in). All three counted, so this is a false hold: the hold came from a reader's doubt, not the count.
Loss misread: the prop is a real compact film camera, not a toy (it stands in for the kit's interchangeable-lens body); a false stand-in flag; it held a complete kit (a false hold under the registered scoring)
DETERMINISTIC · app code, no modelJob 05: Muse Wonder drafts
A recommendation that cites a clause from the store's policy. A citation that is not in the policy text is rejected.
A draft, not an approval. It cannot move money.
On this public serverThe draft shown is the app-code template:
DETERMINISTIC · app code, no modelor, in a replay of the recorded run on d1 and d2, the recorded Wonder draft:
RECORDED LIVE RUN · muse-wonder-1.0 · req id- d1 · Muse Wonder 1.0 agent run
- 19.0 s
- measured: live run, tool loop included (4 calls), inside its 60 s deadline
- d2 · Muse Wonder 1.0 agent run
- 19.3 s
- measured: live run, tool loop included (4 calls), inside its 60 s deadline
Muse Wonder draft · d1Drafted HOLD, citing clause 4.2 (“A kit is refunded in full only when every component listed on the packing slip is returned.”). Its reason: “Not all lines are OK: camera_body is HELD because the observed item is described as a light-green toy truck, not a camera body, and lens is MISSING, so the kit is incomplete and must be held for review.”
Loss misread: the prop is a real compact film camera, not a toy (it stands in for the kit's interchangeable-lens body); a false stand-in flag
RECORDED LIVE RUN · muse-wonder-1.0 · req chatcmpl-01a1a268-a0af-7528-9981-743f1b1baf76Muse Wonder draft · d2Drafted HOLD, citing clause 4.2 (“A kit is refunded in full only when every component listed on the packing slip is returned.”). Its reason: “The camera_body line is not OK: the item observed under its legible label was reported as a toy car, not a camera body, so a kit component is effectively missing and the full kit refund cannot proceed.”
Loss misread: the prop is a real compact film camera, not a toy (it stands in for the kit's interchangeable-lens body); a false stand-in flag; it held a complete kit (a false hold under the registered scoring)
RECORDED LIVE RUN · muse-wonder-1.0 · req chatcmpl-01a1a26b-8ee6-723b-8b98-c67225049438Fleet policy agent (Muse Wonder 1.0)The fleet's separate policy check on d1 and d2 returned no answer.
Loss the policy agent timed out at 90 s on both filmed boxes; app code stood in and the holds were kept.
Job 06: A person approves; Stripe refunds
The reviewer reads the lines, the crops and the cited clause, then approves or keeps the hold. Approval issues a single-use token, bound to that RMA, observation and amount, that expires in 10 minutes. Only that token runs the refund.
A hold is not a denial. A complete kit is not an automatic refund: it waits for this step too.
On this public serverA PERSON · app-code authorization[Stripe test mode — no real funds moved]In this prototype the approver is a typed name; staff sign-in and roles are a production step. The refund endpoint refuses any key that is not a Stripe test key.
What runs for real here, what runs on recorded data, what runs only where the key is.
LIVE runs for real on this server. FIXTURE runs on seeded or recorded data, labelled on screen. OFF ON THIS SERVER works with the Meta Model API key, which is kept off this public server on purpose; those steps run live in recorded sessions.
Pipeline modes: what every label means, and the status of each part
Every result on screen carries one of these labels. LIVE · Meta means a real Meta call produced it. LIVE · Meta open weights on a partner GPU means Meta's open-weight model ran live on a partner's GPU (Baseten today), not on Meta's own API. RECORDED FIXTURE · synthetic scene, not a live Meta call means a saved synthetic result is being replayed; RECORDED · replay of a recorded real run, not a live call means a stage of a real run on filmed clips is being replayed; RECORDED LIVE RUN · model · req id means a saved real model call is being replayed. DETERMINISTIC · app code, no model means plain code produced it.
| Feature | What it does | Status | Note |
|---|---|---|---|
| Bench station screen | Pick the RMA, play or record the clip, see stage timers and the per-line result. | LIVE | UI. Next.js, React. |
| Seeded store, customer, RMA, packing slip, policy | The merchant data a real store would connect. | FIXTURE | Deliberate: no store integration. Labelled [Seeded merchant order & RMA fixture]. |
| Item segmentation | Masks and track IDs per item per frame. | OFF ON THIS SERVER | SAM 3.1 has run live on Meta's API in recorded sessions (first on a known-answer synthetic clip, matching the open weights frame for frame). This public server has no Meta key on purpose, so it replays a recorded real run on filmed stand-in props (masks and text only, no footage), labelled RECORDED · replay of a recorded real run, with its losses shown: the complete d2 kit was falsely held. Clips without a recording replay synthetic scenes, labelled RECORDED FIXTURE · synthetic scene. |
| Tray-zone counter | Counts an item when it settles in the tray; ignores hands and paper; flags merged masks and unreadable labels. | LIVE | Deterministic, unit-tested on synthetic tracks with the hard cases. |
| Reconciliation | Expected, observed and missing per line, with reasons. | LIVE | Schema plus deterministic reconciler. |
| Model review of the reconciliation | Muse Spark 1.3 re-reads the clip, counts, crops, slip and policy in one call and can only add a hold. | OFF ON THIS SERVER | Ran live in recorded sessions. No non-Meta fallback. On this public server reconcile runs in app code, labelled DETERMINISTIC; a replay of a real recorded run is labelled RECORDED LIVE RUN with its model and request id. |
| Cross-check | A model answer that disagrees with the counter can only raise a hold. | LIVE | Most restrictive result wins. |
| Decision draft | Recommendation and cited clause, validated against the policy text. | OFF ON THIS SERVER | Muse Wonder 1.0 ran live in recorded sessions. On this public server the draft is the app-code template, labelled DETERMINISTIC, or, in a replay of the recorded run on d1 and d2, the recorded Wonder draft, labelled RECORDED LIVE RUN with its model and request id. |
| State machine + approval token | Refund needs a single-use, expiring token bound to that RMA and observation; any error is a hold. | LIVE | HMAC-SHA256, canonical JSON digest. |
| Refund rail | Seeds a test charge, refunds it on approval, books the $2.00 fee (proposed pricing), shows the re_ id, receipt and webhook payload. | LIVE | Stripe test mode only. |
| Call log + cost | Every model call: input, output, latency, cost, model. | LIVE | Local call log. |
| Gate test harness | Runs SAM 3.1 on test clips and compares IDs per physical item with the tray counter. | OFF ON THIS SERVER | Ran on the filmed bench clips: INCOMPLETE, not run as registered (g2–g8 not filmed); g0 scored correctly, g1 scored correctly (merged_mask hold). |
| Evals | Counter accuracy on labelled clips; every decision and its human outcome logged. | RECORDED | The real gate (INCOMPLETE) and a same-input A/B on 4 filmed clips (Muse Spark alone 4/4, the pipeline 3/4; the pipeline arm ran without the auditor and Glimmer on g0 and g1), next to the synthetic clips. |
A refund can only move forward through these states.
Anything unresolved, any model disagreement, and any failure of a required step (segmentation, the count, a required reader) lands in Held. An optional reader that fails adds nothing. Held is a stop, not a verdict.
- Received
- Counted
- Reconciled
- HeldorAwaitingApprovalApprovedRefunded
The refund endpoint will not run without one.
Single use. Spending it once invalidates it, so one approval cannot trigger two refunds.
10 minutes. It expires. A stale approval cannot be replayed later.
Bound and signed. HMAC-signed and tied to a digest of that exact RMA, that observation and that amount. Change any of them and it fails.
Planning projection: one shift at 500 returns a day
Arithmetic, not observed throughput: no bench has run this load. Model time and cost come from measured live Meta calls on a synthetic preflight clip; the people side is assumption. On the filmed clips SAM 3.1 took 14.8 s and 20.8 s (the flow above), slower than the preflight clip, so the SAM time in this projection is optimistic.
What the projection says
- In the projection the draft is ready about 60 s after the clip ends, inside the 120 s assumed to unpack the next box, so a bench would not wait on a model: the projected bottleneck is people at the bench.
- Two or three boxes in flight and about 19 calls a minute is a small load; each box is independent, so in the projection benches add linearly.
- Wonder is the slowest and the costliest model step per box; SAM grows with clip length (billed per frame, 50-frame minimum per request).
- The fleet's app-code roles cost nothing to scale; see Agents at work for the per-role projection.
When something breaks
- A required model call fails, times out or answers out of shape: the box lands in Held, and nothing refunds by default. An optional reader that fails adds nothing. On the filmed run the fleet's policy agent timed out on both boxes and app code stood in; Glimmer timed out on d2 and added no hold; the boxes were held for other reasons (d1 the missing lens, d2 the false stand-in flag).
- SAM 3.1 has a second path on open weights (Modal); it is switched on for local runs, not on the public site.
- A queue backs up: boxes wait as Held or awaiting approval. The cost of falling behind is a slower refund, never a wrong one.
Projection inputs: every value and where it comes from
Measured: meta-preflight (scripts/meta-preflight.ts): 4 runs on a synthetic 25-frame clip; SAM billed the 50-frame minimum.
| Input | Value | Source |
|---|---|---|
| SAM 3.1, one concept 28 calls measured | 3.2 s p90 · $0.010 | measured |
| Muse Spark 1.3 reader 2 successful of 4 (two returned 400s, since fixed) measured | 15.5 s p90 · $0.017 | measured |
| Muse Wonder 1.0 agent run 4 runs, tool loop included measured | 41.4 s p90 · $0.069 | measured |
| Muse Voice Transcribe runs while the box is unpacked, off the critical path measured | 1.9 s p90 · $0.0001 | measured |
| Returns a day planning volume for one returns centre's kit returns (a target, not a customer's figure) planning target | 500 | planning target |
| Shift one 8-hour shift assumption | 8 h | assumption |
| Busiest hour the busiest hour sees twice the average assumption | 2× average | assumption |
| Clip length bench clips run 10–25 s (gate/CLIP-SPEC.md); 5 fps after preparation configured | 20 s | configured |
| SAM concepts per box 3 kit items + hand + 2 packing concepts (data/kit.config.ts), one SAM 3.1 request each configured | 6 | configured |
| Bench time per box open, unpack into the tray, repack: to be replaced by the filmed clips' timings assumption | 120 s | assumption |
| Boxes held the seeded shift holds 6 of 24 boxes (data/shift.ts) seeded fixture | 25% | seeded fixture |
| Review a hold a person reads the reasons and evidence on a held box assumption | 90 s | assumption |
| Approve a clean box a person approves a complete box assumption | 10 s | assumption |
The measured calls were on a synthetic 25-frame clip, and SAM's cost is scaled by billed frames. On the filmed bench clips SAM 3.1 took 14.8 s (a 21 s clip) and 20.8 s (a 27 s clip), all concepts in parallel, so the SAM time here is optimistic; these figures aren't recomputed from the filmed run yet (scripts/measured-calls-summary.ts).
Explained simply
The analogy
Think of a cashier counting items back onto the belt. Each thing that comes out of the bag is put down once, in the clear space, and the cashier ticks it against the receipt. A hand passing over it, or the bag rustling, does not add a tick.
If the receipt says three and only two are on the belt, the cashier does not accuse anyone. They hold the money, show what they saw, and ask the manager. That is the whole product: a careful counter with a manager in the loop.
The steps
- Pick the RMA on the tablet. The packing slip says what should be in the box.
- Take items out one at a time and put each in the tray.
- The camera follows each one; the counter counts it when it settles.
- The lines show OK, MISSING or HELD against the slip and the policy.
- A draft names the clause. The person approves or keeps the hold.
- On approval, the test refund runs and its receipt appears.
The parts
| Part | What it does |
|---|---|
| Camera | Looks straight down at the bench and films the unboxing. It sees; it decides nothing. |
| Tray zone | The marked right-hand side of the mat. An item only counts once it settles here. |
| Counter | App code that watches the tracks and counts an item once when it settles in the tray. |
| Readers | SAM 3.1 finds and outlines items; it does not count. Muse Spark 1.3, the open-weight auditor (Baseten) and Glimmer (Modal) re-read the counts, crops, labels, slip and policy, and can only add a hold. |
| Drafter | Muse Wonder 1.0 writes the recommendation and names the clause it relied on. |
| The person | Approves or keeps the hold. The only one who can let a refund go. |
| Refund rail | Stripe in test mode. It moves test money only after a valid approval. |
What the numbers mean
- Expected
- How many of this line the packing slip says were shipped.
- Seen
- How many the counter saw settle in the tray in this clip.
- Missing
- Expected minus seen. For RMA-20418 the lens reads 0 of 1, so it is missing.
- Held $
- The refund that waits. Here $340, the whole kit, until a person decides.
- Settle time
- The counter waits for an item to stay in the tray for 3 frames at 5 frames per second, which is 0.6 seconds, so a passing item is not counted.
Who does what
| Who | Does |
|---|---|
| Camera + code | Count what settles in the tray, and enforce the rules. |
| Meta models | Read the clip and the paperwork, and draft the recommendation. |
| The person | Rita Calder decides: approve, or keep holding. |
| Stripe | Moves test money once, after an approval. |
What a HELD shows, and what it doesn't
No accepted arrival of that item in the tray during this clip, or something the camera couldn't resolve: two items merged into one, a label it couldn't read. A hold is a question for a person, not proof the customer kept anything.
That the customer kept the item, that an item it did see is genuine, or that there is no hidden damage. A brick the shape of a lens is out of scope, and so is anything inside a sealed box.
It holds the refund, attaches timestamped evidence and the policy clause, and a person decides. Under clause 5.1 a hold is not a denial.
See the hold happen.
Two seeded returns: one with the lens missing, one complete. Test mode only.