Over one day of work on a live web app, we fixed problems in four unrelated areas: how the site deploys, how a design review was interpreted, how a financial calculator's numbers are tested, and a line of text in a privacy policy. None of these share any code. But the cause was the same each time: something looked correct, so it went unchecked, and it turned out to be wrong.
This post walks through those four cases — what we assumed, how we verified it, and what we changed. Every example is a real bug that shipped until an independent check caught it: a failed build, a live HTTP request, a pixel measurement, or a plain text search. In each case, re-reading the original code or copy more carefully would not have caught the problem. Only checking against something outside our own assumption did.
Code that behaved differently at build time and at request time
- Three separate assumptions about how the code would run in production, each wrong in a different way.
- Found through a failed build, a live comparison of two HTTP responses, and a public documentation page.
What a reviewer saw wasn't what actually shipped
- A false claim about how a page rendered, a visual cue that was technically present but not noticeable, and a screenshot taken at the wrong screen size.
- Found by loading the live page directly, measuring actual pixel colors, and checking image dimensions instead of trusting a filename.
Two numbers answering the same question
- A calculator intentionally shows two different results for the same question, and a testing approach keeps them from drifting apart.
- The same testing approach caught an ordering bug in the product that nobody had flagged.
A disclosure that named a tool we never installed
- A privacy policy line described an analytics product that was never actually added to the site.
- Found with a single command-line search against the live site's HTML.
01Code that behaved differently at build time and at request time
The app is built with Next.js and deployed to Cloudflare Workers. The first assumption was that a routing feature called middleware — code that runs before a request reaches a page — would work the same way it always had. It didn't: in this version of the framework, middleware defaults to a runtime that our specific deployment setup doesn't support, and there's no supported way to change that from within the file itself. This wasn't obvious from the release notes. It became obvious the moment we ran an actual production build and it failed outright.
The fix was to stop using middleware and instead set an HTTP header (used to tell search engines not to index a staging site) from the framework's own build configuration file. That created a second, subtler assumption: that the same environment variable controlling that header would also work inside a separate route that generates the site's robots.txt file. It didn't, and the reason is worth spelling out, because it's easy to get backwards:
// Build configuration -- runs ONCE when the site is built. // An environment variable set on your local machine or CI system IS visible here. const isStaging = process.env.DEPLOY_ENV === "staging"; // A route file -- runs on EVERY request, inside the deployed server. // That same environment variable does NOT carry over to here.
Both files looked like they were reading the same setting. Only one of them actually could. We found the gap by requesting the staging site twice in a row and comparing the two results: the response header correctly said "don't index this," but the robots.txt file itself still said the opposite. Same server, same deployment, two different answers to the same question. The fix was to read that setting a different way inside the route — using an API that reads the live server's actual configuration at request time, rather than a variable that only exists during the build.
The third assumption was the most dangerous one, because it looked like extra caution rather than a mistake. The staging site was blocking search engines two different ways at once: a Disallow rule in robots.txt, and a "noindex" instruction in the HTTP header. The idea was that stacking two signals is safer than relying on one. According to Google's own documentation on robots.txt and its guidance on blocking pages from search indexing, that combination backfires: a page blocked by Disallow is never crawled at all, which means the search engine never even sees the "noindex" instruction sitting on it. A blocked URL can still show up in search results with no preview text, discovered through a link from somewhere else. We fixed it by removing the Disallow rule and relying on the header alone, exactly as Google recommends.
A failed build. A side-by-side comparison of two live HTTP responses. A documentation page, read directly instead of assumed from memory. None of these three problems were caught by reviewing the code more carefully — each one needed a check that came from outside the code itself.
02What a reviewer saw wasn't what actually shipped
A round of design review on a staging site produced eight findings, ranked by severity. One of them said a page showing five financial tools across four levels of dependency "rendered flat" — a real usability problem, if true. It wasn't. The reviewer had been looking at a text-only export of the page, and that export collapses a nested layout into one flat list. The live page, viewed directly in a browser, had rendered all four levels correctly the entire time.
Reading the actual rendered page is authoritative. I was reviewing a lossy text export and it looked flat when it wasn't.
— from the review notes, once the live page was checked
A separate round of screenshots caught a real version of the same kind of problem. A data table on mobile was cut off with no visible sign that it scrolled sideways — a genuine bug. The first fix added a soft shadow along the table's edge to hint that more content was there. It looked correct in a screenshot. It still wasn't enough: we sampled the actual pixel colors across that edge and found the shadow rendering exactly as designed — a real gradient from white to light grey over about a dozen pixels — and still too subtle for anyone to notice while scrolling normally.
254,254,254 → 245,245,245 → 225,225,225 → back to white, over ~13 pixels
The shadow was there. It just wasn't something anyone would actually notice. We fixed it by adding a plain text hint — "scroll to see all columns" — instead of relying on a visual effect that only worked in theory.
The same review round included a screenshot labeled "homepage, mobile view." It was actually captured at a leftover desktop browser width from the previous screenshot in the batch — the browser was never resized back down before that last image was taken. Nothing in the image itself looked obviously wrong; a homepage can render fine at almost any width. It was only caught because someone asked directly which file showed what. Going forward, we check a screenshot's actual pixel dimensions before treating it as proof of how something looks on a given device.
03Two numbers answering the same question
One of the calculators shows two figures for the same question: how much money someone needs saved before retiring. The first is a simple estimate based on a standard formula that assumes steady investment returns every year. The second runs a thousand simulated market scenarios that include bad early years, taxes, and medical costs that rise faster than everything else. The two numbers are usually far apart on purpose — the simple estimate quietly understates real risk, so the simulation-based number is the one that gets passed along to the next step in the flow.
Keeping that handoff honest only matters if the numbers actually agree once passed from one calculator to the next. Anywhere one tool's result feeds directly into another tool's input, we run a test that feeds the first tool's real output into the second tool's own calculation and checks that the two agree — exactly, where the math allows it, or within a defined margin where simulation makes exact agreement meaningless. A calculator passing its own tests says nothing about whether it agrees with the next one in the sequence. Only a test that crosses that boundary catches that.
That testing approach found something nobody had gone looking for. The site's own suggested order of steps had a direct link from the retirement calculator straight to a tool comparing two withdrawal strategies — skipping over a withdrawal-planning tool in between. But that comparison tool listed the withdrawal planner as something a user should already have gone through first. Someone following the site's own recommended path could reach a tool before the thing it depended on. No reviewer had flagged it. The step order just didn't hold up once we looked at it as a sequence, and we corrected it the same day.
04A disclosure that named a tool we never installed
One calculator's privacy disclosure included this line: "This site uses Cloudflare Web Analytics to count page visits." Cloudflare Web Analytics is a specific, real product — a small script that loads in the visitor's browser and reports back to Cloudflare. It sounds exactly like the kind of privacy-friendly analytics tool a site like this would use. It had never actually been added to the site.
$ curl -s https://example.com | grep -i "cloudflareinsights\|beacon.min.js" (no output — the script this line describes isn't present on the page)
What the site actually uses is a different, custom-built approach: it records the page path a visitor loaded, with no cookie and no way to identify the same visitor twice, and stores that in Cloudflare Analytics Engine, a general-purpose data store rather than the packaged analytics product named in the disclosure. It's arguably a more private approach than the one described. That didn't make the sentence accurate. We fixed it with a one-line correction, describing the mechanism that's actually running instead of the plausible-sounding one that isn't:
"This site uses Cloudflare Web Analytics to count page visits"
"This site records anonymous, cookie-free page-visit counts via Cloudflare Analytics Engine"
The mistake wasn't caught by proofreading the privacy page — every earlier read of that sentence judged it fine, because it sounded entirely plausible. It took a text search against the live site's actual HTML, and separately against the whole codebase, for the script the sentence claimed was there. Neither search found it.
05The pattern across all four
Plausible and true are different things
A release note, a reviewer's description, a rendered shadow, a line in a privacy policy can all sound exactly right and still be wrong. How confident something sounds is not evidence that it's correct.
The check has to come from outside the assumption
Re-reading your own reasoning rarely surfaces a belief you already hold. A build run, a live request, a pixel measurement, or someone else's documentation does, because none of those already agree with you by default.
Verification is cheap compared to the alternative
Each of these four checks took one command or one direct look at the actual system. Left unverified, each one would have shipped to a real search engine, a real regulator, or a real user reading a privacy page.
All eight problems described here — three in deployment, three in design review, one in product logic, one in a privacy disclosure — were found and fixed in a single working session. They didn't share any code. They shared one pattern: an assumption stood in for a fact until someone checked it against something real.