Lab · About this build

How the Lab is built

I built the Lab to show what current planning software does, in a form you can touch, and to show how I build. This page is the part a hiring manager or an engineer would ask about: the architecture, the model choices, the data, and the decisions.

The shape of it

The engine runs in the browser.Merrimack Supply Co. is a TypeScript simulation: three warehouses, thirty-six branches, ninety-six items, seasonal demand with fixed random noise per seed, supplier lead times with variability, minimum orders and case packs, warehouse capacity, and an agent that forecasts every item at every site each week and proposes moves with a risk score. A thirteen-week quarter runs in about a third of a second, which is what makes the reveal possible: the reference desks and one counterfactual run per event are all real runs of the same engine on the same seed, computed on the visitor's machine. They run on worker threads beside the page, several at a time, so the desk never waits on them.

The language models run on the server, through this site. Next.js route handlers hold the provider keys, route each job to a model, cap calls per visitor and per day, and return what the model said along with which model it was and what the call cost. The page shows that. The copilot's tools are declared on the server and executed in the browser against the game: the model asks for the case, the item's history, the branch, the inbox, or a what-if, the browser runs it, and the conversation continues. Nothing a model says changes the game. Every change is the planner's click.

The weekly board is scored on the server.A browser that posts a result sends its decisions, not a number: the entries it made and when, and what it did with each case. A route handler replays the quarter with the same engine, checks that the minutes fit the budget, scores the desk against the reference desks, and keeps that browser's best for the week in Postgres at Neon. Nothing about you is stored beyond the name you type.

The arithmetic stays out of the language models. Forecasts, targets, allocation and costs come from statistics and inventory policy, the way real planning systems keep it. The models read, explain, rank, test and recommend. That split is the design decision I would insist on in any AI plan, and the game is built to make you feel why.

Which model does what

Four providers, picked per job on published price and capability as of September 2026, and routed with fallbacks so a missing key never breaks the desk. The order below is the order of preference; the first provider with a key configured takes the job.

JobPreferenceWhy
Reading the inbox
  1. 1. Gemini 3.8 Flash $0.75/$3.75 per M
  2. 2. Grok 4.3 $1.25/$2.5 per M
  3. 3. GPT-5.6 Luna $0.2/$1.2 per M
  4. 4. Claude Haiku 4.5 $1/$5 per M
Short messages, strict JSON, dozens a quarter, and it has to resist decoys. The cheapest model that reads carefully wins. The eval below decides the order.
Working a case with the planner
  1. 1. Claude Opus 5 $5/$25 per M
  2. 2. GPT-5.6 Terra $2/$12 per M
  3. 3. Gemini 3.8 Flash $0.75/$3.75 per M
  4. 4. Grok 4.6 $2/$6 per M
Several tool calls, a what-if through the engine, and a recommendation that has to be right for a reason. This is the job for the strongest reasoning model with the best tool use.
The morning brief
  1. 1. GPT-5.6 Terra $2/$12 per M
  2. 2. Claude Sonnet 5 $2/$10 per M
  3. 3. Gemini 3.8 Flash $0.75/$3.75 per M
  4. 4. Grok 4.6 $2/$6 per M
Read a queue of up to eighty cases and the inbox, rank them, say what to skip. Mid-tier reasoning with low effort keeps it fast and cheap.
The debrief
  1. 1. Grok 4.6 $2/$6 per M
  2. 2. Claude Sonnet 5 $2/$10 per M
  3. 3. GPT-5.6 Terra $2/$12 per M
  4. 4. Gemini 3.8 Flash $0.75/$3.75 per M
Two hundred words with some voice, once a quarter. Cheap output and a good writer.
The monthly scan
  1. 1. Gemini 3.8 Flash $0.75/$3.75 per M
  2. 2. GPT-5.6 Luna $0.2/$1.2 per M
  3. 3. Claude Haiku 4.5 $1/$5 per M
  4. 4. Grok 4.3 $1.25/$2.5 per M
Summarizing long articles for the state-of-the-art digest. Long context at low cost.

Prices are list prices per million input and output tokens, as published by each provider on 2026-09-16. Cost shown on the page is estimated from token counts at those prices.

The inbox reader, measured

The engine knows which messages are real events, which are decoys and which are noise, so the reader can be scored. The eval runs the same messages through each configured model and counts how often it says enter when it should, how often it names the right kind, how many decoys it catches, what it costs and how long it takes. The routing order above follows the numbers.

ModelEnter or notKindReal caughtDecoys caughtPer 100 msgsPer week
Gemini 3.8 Flash100.0%70.6%100%100%$0.0763.1s
GPT-5.6 Luna99.5%74.5%97%100%$0.0196.3s
Claude Haiku 4.598.7%76.9%92%100%$0.1038.1s
Grok 4.3100.0%77.4%100%100%$0.11510.6s

385 messages across 6 chaos quarters, run September 16, 2026.

What is real and what is not

The company, the branches, the items, the demand and the costs are made up. The map is real: state outlines from the US Census cartographic boundaries, branches placed in real towns. The dynamics are the ones every distributor lives with, and every mechanism in the engine has a name a planner would recognize: order-up-to targets, time-phased requirements, a phantom pipeline when nobody updates the dates, a dead feed read as a demand drop. The model calls are real calls to real providers, named on the page each time. Nothing about a visitor's game is stored on a server; a best score lives in their own browser.

Decisions worth knowing about

  • The planner only edits what crosses the gate, and only enters what has reached the inbox. That is what keeps the score honest: the value of a desk is the value of its inputs and its judgment, not a hidden channel.
  • Entries carry most of a planner's value in this model, and the queue earns its keep on the expensive exceptions. That was a finding, not a design goal, and the game keeps it because it matches what I have seen in real planning systems.
  • Once a supplier notice is entered, the system shows the revised dates and the agent plans with them. Left un-entered, the overdue order shows as arriving next week, week after week. That is the phantom pipeline, and it is the most common way real replenishment systems get fooled.
  • Site-to-site transfers were built and then removed: in one hundred and seventy-six combinations of scenario and gate they never fired, because the agent's own inflated forecast hid the excess.
  • Anomaly flags on slow items are floored at the counting noise of the rate itself, so an item that sells three a week does not throw a flag every time it sells five.

The other build: The Measure of the Machine

The Measure of the Machine started in the Lab and now has its own address. It charts AI from 1950 to 2035 on three measured series: Epoch AI's training compute for 539 systems, METR's task horizons for 26 models, and Epoch's capability index for 266.

It is a static page. A script pulls the three series into JSON files in the repository. Everything else is computed from those files when the site builds: the trend fits, the scenario lines, the date each scenario reaches a level, the estimates for models nobody has measured, and a preview picture for each of the 38stories, so a link to a story shows that story's chart. Nothing on the page calls a model and nothing is stored about a visitor. A birth year, if you enter one, stays in your browser.

The charts are hand-written SVG in React, with no chart library. That is what lets one chart slide between a log and a linear axis, zoom as you drag across it, draw itself when it first comes into view, and carry three lenses with their own scales. The motion outside the chart is plain CSS, so it needs no script and stands still for a reader who asks for less motion.

Two rules shaped it more than any design choice. Every claim about what AI can do now was checked against a live source on the day it was written, and carries that date on the page. And where no measurement exists, the page estimates, shows the arithmetic and the range, and draws the result hollow so it cannot pass for data. When a measurement arrives, it replaces the estimate at the next refresh.

What changed

  1. September 19, 2026

    The labels too

    The first pass at explaining terms covered the sentences and missed the labels. Chips only, the name of a scenario line, sat on the chart, in a readout and at the head of a table with its meaning in a folded method note. Every name of that kind now opens its own explanation where it stands: the nine scenario lines, with a key under the chart that shows which dashes are which, the layer chips, the measured and estimated tags, the table headings, the chess key and the eras on the story rail. The scenario entries are written from the same data that draws the lines, so the two cannot drift apart. Terms and concepts now holds 99 entries.

  2. September 18, 2026

    Finding your way, and every term explained

    The site's menu mixed two kinds of link: jumps within the home page, and other pages. They are apart now. The top row is the site: Home, Lab, AI since 1950, the background PDF marked as a download, and the way to reach me. A second row belongs to the page in view and jumps between its sections, on the home page and on The Measure of the Machine alike, and on a phone it slides sideways under the header. The Measure of the Machine is a long page, so it also has a contents panel: every section with a line on what is in it, opened from the bar, from a button that stays at the bottom of a phone screen, or with the C key. The bracket keys move a section at a time, and each section ends with a link to the next. And every term the page leans on, more than eighty of them, now opens a short explanation in plain words where it stands, on hover, focus or tap. All of them are collected in a Terms and concepts section with a box to find one.

  3. September 18, 2026

    The Measure of the Machine: a review of the page, then ten changes

    The page left the Lab for its own address, with a place in the site's main menu, and then got the same treatment the desk did. The top of the page is the record line itself, drawing from 1950 to today. The five headline figures count up, say whether they are measured or estimated, and open the lens they come from. The chart draws itself the first time it comes into view and marks today's frontier. Drag across it to zoom into a span of years. A rail under the chart lists every story by era, one tap away. A search box finds any of the systems, models, creatures and stories and shows it on the chart. The explorer goes full screen, for showing it to a room. Each story can be shared, with an address and a preview picture of its own. The levels of work sit on one ladder, with how far measurement has come and how far the estimates reach. And a birth year marks the chart and reads the data from that year to now. Around those: a section bar that stays with you and shows where you are, links from every section back into the chart, and keys for all of it.

  4. September 18, 2026

    The Measure of the Machine

    AI from 1950 to 2035 on three lenses. The scale lens plots the computing used to train 539 notable systems from Epoch AI's data, with the record run, brain-scale anchors from a roundworm to a person, and three scenarios to 2035. The ability lens plots METR's measured task lengths, from GPT-2's three seconds to seventeen hours, at even odds and at four in five. The index lens plots Epoch's capability index, the one series that scores a new model within days, with the span of work that goes with a score. Where measurement stops, the page estimates and says so: horizons for the models METR has not reached, worked out from the index, and GPT-6 Astra's training run, worked out from its disclosed hardware, all drawn hollow. The scenario lines can start from the measured trend or from the estimated frontier. A chess strip follows the one skill with a rating for every year, from Mac Hack VI in 1968 to the top of the engine list on the day of writing. Thirty-seven milestones open as stories, each with what it meant on a planning desk where that applies. A script refreshes the three measured series, and the trends, the estimates, the scenario lines and every date on the page recompute from the data.

  5. September 17, 2026

    Room for a whole quarter

    The allowances on the desk's model calls were sized for a look, not a full game. A visitor now gets enough in an hour to play a whole quarter hard and replay it: forty inbox reads, forty briefs, about 160 copilot questions and six debriefs. The site's daily cap used to count calls, which treated a cheap inbox read the same as a copilot round on Claude Opus. It is now a budget in dollars per provider, charged per call with what a typical call costs on the model that serves it, measured on the live site. The session cost shown at the top of the desk also under-counted the copilot: rounds after the first were not counted, so a question that cost nineteen cents showed as two. Every round counts now.

  6. September 17, 2026

    Expedites worth flying, and the Autonomy Dial retired

    Every air freight used to be a reject. A sweep that finished each quarter twice per proposal, flown and rejected, showed why: the agent only ever proposed freight for a warehouse running dry on the two slow critical lines, and the branches always held enough cover to ride that out. Now an air freight can also come from a short projection on the system's numbers, on any line whose margin beats the premium: how long branch stock, the warehouse and what is on the way can serve the forecast, and how much demand nothing can serve before relief lands. The week the freight cannot reach is left out. About one proposal in eight is worth flying now, the flag reads above or below the premium and is right about four times in five, and the misses are slips the system does not know about yet, which is what the phone call is for. Supplier slips can run wider and longer and lean toward the critical suppliers, and cold snaps hit water heaters and compressors the way they do. The copilot's allowance is counted in rounds, one per tool call, so it was reaching its hourly cap after about fifteen questions and reporting that as unavailable; the cap is now about forty questions, the panel says the real reason, and it offers a retry. The Autonomy Dial page is retired: its lesson is the gate inside the game, and the old address now leads here.

  7. September 16, 2026

    A new face for the desk

    A review of the interface, then ten changes. Play starts on the first screen with the difficulty and the gate behind a setup panel. The week shows its four beats, and closing a week brings a short recap: what was reviewed, what was stamped, where the margin went, and one thing to notice. The first week is a walkthrough, and every quarter now opens with a real event and one that only sounds real. Cases lead with the dollars at stake and why they crossed the gate, with an adjust stepper in place of a bare number field, and the unrelated office noise is off the card. The reveal leads with what cost the most and what to try next, with the deep sections folded. The calendar's titles are readable, the map has tap callouts and larger labels, the side panels fold on a phone, the score tile means something from week one, and the filters and map are reachable without a mouse.

  8. September 16, 2026

    The copilot survives a bad tool call

    When the copilot asked for an item's history at a branch that does not exist, which happens on a supplier order or an air freight because those cases have no branch, the browser-side tool threw and the whole conversation ended with an error. The tools now answer a bad id with a readable error and the list of valid ids, the case file names the branches a warehouse serves, and a failed tool can no longer end the chat. The server logs the provider's own error when a step fails.

  9. September 16, 2026

    A board for the week

    Finish this week's quarter and you can post your result. The browser sends the decisions, not a score: which messages you entered and when, and what you did with each case. A route handler replays the quarter with the same engine, checks that the minutes fit the budget, scores it against the reference desks, and keeps your browser's best for the week on a Postgres database at Neon. Nothing about you is stored beyond the name you type.

  10. September 16, 2026

    A shared quarter each week

    The start screen offers this week's quarter: one seed for everyone who plays in the same calendar week, a normal quarter with whatever gate you choose. The seed comes from the week itself, so no server is involved, and your best result on it is kept in your browser. A leaderboard for the week comes when there is somewhere to keep it.

  11. September 16, 2026

    Cleaner counterfactuals

    A sweep of twelve seeds showed real events whose entry was worth less than nothing, and the reasons were in the engine. Supplier lead times were drawn from one random stream per week, so one extra order changed every later order's lead time that week and a counterfactual quarter picked up noise in lines the event never touched; each order now draws its own. The modeled planner priced an air freight against the branches' whole shortfall, ignoring what the warehouse could still ship them; it now nets that out and flies only the uncovered units. Supplier slips no longer show a negative worth, promotions are worth more, and the reveal says plainly when an entry would not have changed much.

  12. September 16, 2026

    The reveal moves off the page thread

    The four reference desks and the one-per-event counterfactual runs now go to a pool of worker threads and run in parallel. Starting a quarter no longer freezes the page for a second, and the reveal fills in as each run lands. If a browser has no workers, the runs fall back to the page thread one per frame.

  13. September 16, 2026

    Four models join the desk

    A cheap model reads the inbox and proposes entries. A reasoning model with tools that run in the browser works cases with the planner, runs the engine forward to test a number, and hands over one-click recommendations. A third ranks the queue into a morning brief. A fourth writes the debrief. Every call names its model and its cost on the page. The models are routed per job, with fallbacks when a provider is not configured.

  14. September 16, 2026

    The network on a map, and a calendar

    New England state outlines from the US Census, branches colored by stock and stockouts, warehouses with capacity, halos for promotions and cold snaps, badges for dead feeds and account changes. A thirteen-week calendar shows what has reached the planner and whether it is in the plan. Clicking a branch filters the queue to it.

  15. September 16, 2026

    The Exception Queue

    A full distributor engine: three warehouses, thirty-six branches, ninety-six items, seasonal demand, lead times, minimum orders and case packs, expedites, a risk-scored gate, event entry into the plan, a modeled planner with an attention budget, decoy messages, and counterfactual reference desks for the reveal. Thirteen weeks in about eight minutes.

  16. September 15, 2026

    The Autonomy Dial

    The first page: one product, six sites, a rules-based agent, and a dial that decides which moves a planner sees. Small on purpose, and the reason the bigger build followed.

If you want to talk about how any of this was built, or what it would take inside your company, I would like that.