I Build Systems That Fail Safely
The failures that hurt most are the quiet ones. A sync that succeeds with nothing in it. A newsletter that confidently buries a company that’s still alive. All three of these stories end with the same rule: when a system doesn’t know, it has to say so, and then do the safe thing.
RIP · URL · Autonomous publication
An obituary for a website that’s still alive
The pilot wrote obituaries for oracle.com, cranberries.com, and saban.com. All three are still online.
What went wrong
An 11-issue pilot ran unattended from July to September 2026. It chose subjects from Wikidata’s records of dissolved companies and treated a company’s recorded death as proof that its website was dead. Wikidata lists oracle.com as JD Edwards’ website. JD Edwards is gone. Oracle isn’t.
Nothing reported an error. Two settings in the pipeline’s Wikipedia request made the API return only the first paragraph of each article, so every pilot issue was written from one paragraph. One issue ran an Internet Archive “Temporarily Offline” error page as its main image.
What I changed
I retired the pilot and rebuilt it around gates that run before anything is written. A candidate must pass five checks, cheapest first: the domain matches the company’s name, the company has a documented history, it lived on the web, its URL is checked live today, and the Internet Archive holds at least a year of captures.
The writer sees only the evidence the pipeline fetched and must refuse rather than pad. A separate fact-check audits every draft, and on its first real draft it found seven unsupported claims. Flagged claims get a surgical revision, because full rewrites fixed old errors and invented new ones.
Unsure = aliveA bot wall, a server error, or an unreadable page disqualifies the candidate.
Skipping a real dead site costs one candidate. An obituary for a living one costs the publication its credibility. Offline tests fake every kind of response (parked pages, bot walls, redirect loops) on every push, and the pilot’s three mistakes are a permanent regression check.
Userscript · Midjourney · Val Town
A sync that succeeded with nothing in it
After a Midjourney update, my userscript kept syncing like nothing happened. Every row arrived with its IDs and images. The prompt and parameters were empty.
What went wrong
I log my own Midjourney generations to a database on Val Town with a userscript that reads them off the page. An update changed the page’s layout. The selectors for the prompt and parameters matched nothing, but the ones for job links and images still worked, so every sync succeeded.
The rows said raw: false, which looked exactly like a prompt written without --raw. The AI assistant I was debugging with suggested a sturdier selector plus a warning when a card had no parameters. I asked how that warning would tell “no --raw” from “looked in the wrong place.” It couldn’t. A true-or-false field has no room for “I don’t know,” so every false in that data really meant unknown.
What I found
I spent two hours in the web inspector. The update had split each card, so the prompt and its parameters no longer share an element, and no selector can join them again. But the page loads each job as one data object: prompt, parameters, timestamps, and image dimensions together.
Reading that object instead of the page makes absence mean something: if --raw isn’t in the record of my own job, I didn’t use it. I drew a line on how to get it: only data my browser already received for my own account. Midjourney has no public API, and I take that as a reason not to query its servers myself. The fix is designed, not shipped yet.
Unknown ≠ falseA field that can only say yes or no will record “didn’t look” as “no.”
crierbaby · Cloudflare Workers · Bluesky
Never post twice, never post unapproved
A scheduler that posts to a real account has two failures that can’t be undone: a duplicate post, and a post nobody approved.
Never twice
Before posting, the worker claims the post in the database, so two runs, or the timer and a “post now” tap, can’t both send it. If an attempt dies partway, Bluesky may have published the post and lost the reply. So a retry doesn’t just post again. It re-uploads the photos, which produces the same content IDs, and looks for a recent post with a match. It matches photos, not captions, because short captions like ✨ repeat. After three failed tries, the post stops and shows as “didn’t post.”
Never unapproved
Only approved posts are ever due, and auto-posting ships paused. A vision model writes captions in the background, and its result only lands if the post is still waiting for one: if I approved, edited, or deleted it in the meantime, my version wins. Spacing out look-alike photos only ever rearranges unapproved drafts.
Check, then retryWhen you can’t tell whether something happened, find out before doing it again.
claim → post → reply lost → retry: look for it by its photos → post only if missing
Tests fake Bluesky and cover both sides: an older post with the same caption isn’t mistaken for this one, and an earlier attempt that went through is found and not repeated.