How I work with AI

Six steps, the same order every time. I've run this on a hospital scheduling tool, a public sign-up app, a small agency's booking and dispatch system (still on staging), and my own personal agent. The order doesn't change; only the gates get stricter when the stakes are higher.

I write the spec, set the gates, and review every change. AI coding agents write much of the code; tests, evals and my sign-off decide what ships. A model can draft a message to a customer, but a person sends it, and anything that can text or charge a customer ships behind a switch a person turns on.

1. Start with the work

   Before anything gets built, I watch how the work actually happens today: who does it, what they type, what breaks. At National Jewish Health, the staffing schedule was paper before it was a spreadsheet, and a spreadsheet before it was an app.

   National Jewish Health: from paper to spreadsheet to app →

2. Decide what should and shouldn't be automated

   Some steps are routine and safe to hand to a model. Others touch money, a customer, or a person’s data, and stay manual on purpose. At Three Peas in a Pod, the model may only copy a date or time word that already appears in a parent’s text; it never drafts a price or sends a reply.

   Three Peas in a Pod: AI where it’s safe, switches where it isn’t →

3. Build it with agents, under gates

   AI coding agents write most of the code, inside gates that block a bad change before it lands: a shared script that runs the full test suite and a type check before anything merges, and a hook that refuses any raw deploy command an AI agent tries to run and makes a production release stop and ask a person. Nothing reaches main red.

   Three Peas in a Pod: the deploy guard and the land script →

4. Prove it with evals

   SignUpCity's draft feature is checked against 35 written cases, including a prompt-injection case, and a rerun with no code change caught the model quietly getting worse. One clarified prompt rule fixed it. The lesson is written into the plan: a served model drifts, so the suite should be rerun on a schedule, not only after a change.

   SignUpCity: model choice by evidence →

5. Hand it over and train people

   A system nobody uses is not a win. I write down how it works and train people once it works, not before. National Jewish Health’s workbook rollout replaced paper across seven staffing areas, and it needed managers to actually use the new sheet, not just have it.

   National Jewish Health: from paper to spreadsheet to app →

6. Measure

   Status is stated plainly and never inflated. SignUpCity is live with no usage figures claimed; the float-pool app is in IT security review, not yet in use with hospital data. What gets measured is whether the thing works and whether anyone’s week actually got shorter, not vanity counts.

   National Jewish Health: from paper to spreadsheet to app →

• A reply to a family is sent by a person. Three Peas in a Pod’s reply drafts have no send path in the code; the owner sends every one. Anything else that could text a family on its own, like a renewal notice, ships behind a switch that stays off until a person turns it on.
• Anything touching patient data. My own AI use at National Jewish Health is on non-patient work only; no patient data goes into an AI tool.
• A deploy to production. In the Three Peas repository a hook refuses any raw deploy, rollback or secret write an AI agent tries to run, and a production release stops to ask a person first.
• A charge or a price. Three Peas in a Pod’s AI-assisted reply drafts state no price and make no promise; nothing charges a card without a person choosing to.
• Sending a message on my own behalf. Butler, my personal agent, has no send tool at all: outbound messages exist only in transport code the model cannot call.
• Reading untrusted inbound text with a tool-using agent. Butler’s email inbox reaches a single tool-less model call with a strict output schema, never a looping agent that could act on what it reads.

Data boundaries

Patient data never goes into an AI tool. Untrusted inbound text (an email, a customer message) is either handed to a tool-less model call with no ability to act, or wrapped so it can’t be read back as an instruction.

Review points, named

In the Three Peas repository, every merge to main runs through one script that runs the test suite and a type check first; a dirty tree or the wrong branch is refused before anything runs. A hook refuses any raw production deploy or secret write an AI agent attempts, instead of leaving it to memory.

Eval before a model or prompt change

A model swap or prompt edit doesn’t ship on a hunch. SignUpCity picked Haiku 4.5 over Sonnet 5 because Haiku scored as well at a fraction of the cost, and a rerun two days later, with nothing changed, showed that a served model can drift on its own.

Cost caps

SignUpCity’s AI drafting is capped at 200 calls a day, enforced in a quota object the code checks before every call, at roughly $16 a month worst case.

IT security review

The National Jewish Health float-pool app shipped a written submission package answering the hospital’s security team item by item: source code, configuration, and a plain statement of scope (a staff name and free-text availability, no PHI). It is in review, not yet in operational use.

Claims traced to sources

Every factual sentence on this site traces to a line in an evidence file kept with its source. StoneQR keeps the same discipline for its own marketing claims: a table of every claim, its source, and whether it’s verified.

Building the system is half the job. The adoption record covers the other half: training, rollouts, and teaching.