ONLINEREV · 2026-08-04§ BLOG · 2026-08-03-THE-SYSTEM-THAT-WATCHES-MY-SYSTEMSPUBLISHED · 2026-08-03
All Posts
§ Journal2026-08-03

The System That Watches My Systems

By Corey SlickPublished

This week I shipped a tool that automatically summarizes everything I build across my portfolio — and watched three other automated scans catch bugs and shifts I never would have found by hand.

The problem with running many products solo

At some point this year the question "what did I actually ship this week?" stopped being something I could answer from memory. I'm running measurement instrumentation, diagnostic tooling, location-intelligence apps, and half a dozen other products across iOS, Android, macOS, and the web, each with its own repo, its own release cadence, its own set of open questions. A team would split that across people who each hold their own slice in their head. I don't have that luxury — so this week I built the substitute: a tool whose entire job is to watch what I'm building and tell me, in plain language, what changed.

Building the tool that watches my own portfolio

The feature — I'm calling it a project-status system internally — pulls commit activity and session history across every machine I work from, runs it through an agent that synthesizes a daily digest per project, and surfaces it natively in a Mac and iOS app with a voice interface I can just ask questions of ("what's still open on the diagnostics rewrite?"). It sounds simple, but getting it right took real engineering: the first pass compared raw date strings for "most recent activity" instead of parsed dates, which silently misordered projects. A review pass caught six more issues — things like tab sections that weren't actually mutually exclusive, and a coverage gap where a project could go quiet without the system noticing. All of that got fixed and verified on-device before I trusted the thing enough to rely on it day to day.

The meta-point matters more than the feature: I used AI to build a system whose entire purpose is compressing "did I really understand what happened across 8+ codebases this week" into a two-minute read or a spoken answer. That's the actual lever — not that AI writes code faster, but that it lets me build the instrumentation layer that makes running this many concurrent efforts tractable at all.

Verifying against the reference standard, not against memory

The same instinct — don't trust your own recall, verify against ground truth — showed up in a completely different domain this week: a measurement-validation pass on a signal-processing pipeline. I added a suite of reference-anchored tests that check calculations against the actual published constants (luma coefficients from the BT.709 broadcast standard, threshold values from the IEEE 1789 flicker-safety standard, the standard flicker-index formula) instead of checking outputs against whatever the code already produces.

That process caught something real: a calibration routine was scaling against the mean of an instantaneous-power signal when it should have been scaling against the peak. It's a one-line-looking fix, but it moved a calibration constant by about 3% — the kind of silent, compounding error that's brutal to catch by inspection because the output still "looks reasonable" at a glance. It only surfaced because I had a model write a fix, then ran a second, adversarial review pass whose only job was to look for what the first pass missed. Between the two, seven confirmed calculation bugs got resolved as one coherent set instead of dribbling out over months of "huh, that reading seems slightly off" reports.

The bug that only shows up in production

A subtler version of the same pattern hit a receipt-verification path this week. When I added support for verifying in-app purchase signatures, the code correctly validated against Apple's sandbox environment — and only the sandbox environment. In testing, sandbox is all you ever see, so it passed every check I ran manually. In production, real transactions come from Apple's live environment, which uses a different signing root. Nobody would have caught that by testing the way a human tests — click, buy, confirm it works — because the sandbox path does work. It took an adversarial review pass specifically designed to ask "what environment does this assume, and is that assumption actually true everywhere this code runs?" to surface it. Fixed now, with tests that pin both environments so it can't regress silently.

What the portfolio-wide scanners caught this week

I run a set of scheduled scans across the whole portfolio — one tracks changes across roughly 50 third-party APIs I depend on, another watches competitor moves, another watches the communities my products serve. None of these are things I could do by hand across this many products; the value is entirely in the coverage.

This week's API scan flagged a real, live risk: an AI provider's API now silently rejects keys that don't have an explicit restriction configured, a change that's been live since mid-June — if a key was never restricted, calls could already be failing in production with no changelog alert to tell me. Same scan flagged a location-data provider whose documented endpoint-sunset date reads as already in the past, which either means the docs are stale or an integration has quietly broken — worth a live test call either way, not something to trust from documentation text alone.

The community scan turned up something different but just as useful: a lighting-standards registry showing that a widely-cited flicker-safety standard quietly moved to "inactive, no active revision" status back in the spring. Nobody announced that — it's the kind of change you'd only notice if you were periodically checking a standards body's own registry, which is exactly the kind of low-signal, high-value check that's worth automating and not worth ever doing manually.

The actual workflow lesson

The thread connecting all of this: every one of these findings — the misordered dashboard, the calibration bug, the environment-specific verification hole, the silently-expiring API key, the dormant standard — is a class of error that's nearly invisible to a single pass of manual attention and completely findable by a system built to check systematically. That's the shift AI actually enables for someone running this many parallel efforts solo: not "write the code faster," but "build the layer that watches the code, watches the domain, and watches itself" — and then trust that layer enough to spend your own attention on the parts that still need a human doing the framing.