How do you stay aware of what your AI coding agents are doing?

by

I've been running Claude Code, Cursor, and Codex pretty heavily for the last few months and I keep hitting the same loop:

1. Start a task in one agent

2. Switch to something else (Slack, Twitter, another terminal)

3. Come back 30-40 minutes later

4. Agent finished 35 minutes ago. Or worse, it's been waiting for my approval the entire time.

The more agents I run, the worse it gets. There's no unified way to know what's happening across them.

Curious what other people's setups look like:

- Do you just keep terminals visible and check manually?

- Built any custom notification scripts?

- Use something like ntfy or Pushover?

- Just... accept the wasted time?

I've been building something in this space (push notifications + approval flows for AI agents) and I'm trying to understand if everyone's workflow is as janky as mine, or if some of you have figured out something clever.

Would love to hear what's working and what's not.

5.1K views

Add a comment

Replies

Best

This is a real pain point once you run more than one agent at a time. For me the useful alert is not just “the job finished,” but what state it finished in: completed cleanly, waiting for approval, blocked by an error, or made a change that needs review.

The part I’d care about most is the summary quality. A notification that says “done” is less useful than “changed 4 files, tests failed here, needs approval for X.” That turns the alert from noise into a decision point.

Curious if you’re thinking of notifications as the main product, or as the first layer of a bigger agent activity/audit dashboard?

 

Straight answer to your straight question: notifications are the wedge, not the destination. The ping is just the most acute version of the pain, the thing that gets someone to install it on a Tuesday. Underneath it is the real product, an agent activity and audit layer: what changed, what was reviewed, where a human approved, across whatever tools you're running. The notification is how it earns attention. The trail is why it stays useful.

One caveat I'd add, because smart people in this same thread pushed me on it: "dashboard" is a slightly dangerous word. The audit layer has to stay a boring handoff trail, not a charts-and-vanity-metrics thing you sit and stare at. The goal is something you almost never open until it taps you, and when you do, it hands you the decision instantly. Audit trail, yes. Dashboard you babysit, no.

And you put your finger on the actual moat: summary quality. "Done" is noise. "Changed 4 files, tests failed here, needs approval for X" is a decision point, and that gap is the entire product. Anyone can fire a notification. Making the payload good enough that you can rule on it without opening the terminal is the hard, valuable part, and it's exactly where most "we added notifications" features stop.

Curious which half of a good summary you'd trust an agent to get right, the factual part (files, tests, commands) is mechanical, but "needs approval for X" requires the agent to correctly know X is risky. Do you trust it to flag the right things, or would you rather define the risky paths yourself up front?

Honestly, I think this is becoming a real problem as people run multiple AI agents in parallel. Keeping terminals visible works for one agent, but it breaks down quickly when you're juggling Claude Code, Cursor, Codex, and other tools.

The most effective setups I've seen use notifications (desktop, mobile, Slack, Discord, ntfy, Pushover, etc.) for three events:

  • Task completed

  • Agent needs approval/input

  • Agent encountered an error

Without that, you end up wasting time either waiting on agents or having agents wait on you.

What seems to be missing is a unified "agent inbox" that shows the status of all running agents in one place. If you're building push notifications and approval flows, you're solving a real pain point. The challenge will be making it work seamlessly across different agent ecosystems rather than becoming yet another dashboard people forget to check.

 

"Agent inbox" is the right name, and you also wrote the warning that should be tattooed on the project: don't become another dashboard people forget to check. That's the failure mode that kills this entire category. The moment your tool requires the user to remember to look at it, you've just rebuilt the terminal grid with extra steps. A dashboard you have to check is a dashboard that has already lost.

The escape from that is making it push-first, not pull-first. You never check it. It checks you. The three events you listed, completed, needs-input, error, are exactly the only reasons it should ever interrupt, and the rest of the time it stays dark. The inbox isn't a place you visit, it's a thing that taps your shoulder and is otherwise silent. Get that polarity right and "forgot to check it" stops being possible, because checking was never the job.

The other half, seamless across ecosystems, is the genuinely hard engineering and I won't pretend otherwise. Per-tool notifications already exist, Claude Code pings, Warp pings, the problem is they each ping in their own silo with their own definition of done. The whole value is one inbox speaking one language across Claude Code, Cursor, Codex, and whatever's next, so you're not maintaining three notification systems and a script graveyard. That unification is the moat, and it's also the part that's hard enough to be worth doing.

Curious where you'd want it to draw the line on errors specifically, every error, or only the ones it can't self-recover from? An agent that retries and fixes its own typo shouldn't page you, but I keep wrestling with where "it'll handle it" ends and "you need to know" begins.

My setup ended up being pretty janky too — tmux splits with a watch command tailing logs on each agent session, plus a few custom aliases to grep for 'waiting for input' strings. It works but it's manual polling dressed up as automation.

The part I haven't solved is knowing which agent finished *successfully* vs which one quietly stopped because it hit an ambiguous decision and didn't know what to do. Those look identical from the terminal. You come back, nothing's running, and you have to read through the whole context to figure out why.

The visibility gap feels especially real when you're learning AI coding — you're still building intuition for what the agent will or won't surface on its own.

 

"Manual polling dressed up as automation" is the most honest sentence in this entire thread. The grep-for-'waiting-for-input' aliases are clever and also a confession: you're pattern-matching on the agent's prose, which means the day it phrases a question slightly differently, your automation goes blind and you don't even know it went blind.

But you've put your finger on the genuinely hard one, and it's deeper than it looks. Finished-clean and quietly-stopped-on-ambiguity look identical from the terminal because "stopped" isn't a state the agent declared, it's the absence of output, and you cannot grep for an absence. A silent terminal is the same whether the agent succeeded, got confused, or crashed. That's the root bug: terminal-watching infers state from activity, and the most important state, "I hit a decision I can't make and gave up," produces no activity at all.

The only real fix is flipping it: the agent has to announce its terminal state explicitly, "done, here's the diff" versus "blocked, here's the ambiguous call I couldn't resolve," so you're reading a declaration instead of interpreting a silence. That's exactly the gap Pushary goes after, make the agent say which of those two it is, so the two stop being indistinguishable and you never again come back to a dead terminal and have to autopsy the whole context to learn why it died.

The learning-AI-coding angle is the part I find most motivating, honestly. When you're still building intuition for what an agent will and won't surface, a tool that consistently shows you "here's what it flagged, here's what it silently decided" is also teaching you the agent's blind spots. You stop guessing what it'll handle and start knowing. Curious, in your tmux setup, roughly how often is a stopped agent actually blocked versus actually done, do the silent-ambiguous stops happen enough to be the main tax, or is it the rarer-but-brutal case?

The notification piece you're describing solves half of it — knowing when the agent finished. The half that kept biting us was knowing whether what it did was any good, and being able to see it later. The agent runs, produces output, and the context evaporates — you can't see what it knew going in or whether the result held up.

We ended up wiring memory + execution + review into one layer so the answer doesn't disappear when the terminal closes. The thing that finally made it legible was cost-per-approved-output instead of raw token spend — turned out a handful of people were producing most of what actually shipped, and you can't see that from notifications alone.

How are you thinking about the "was it good" layer in Pushary, or are you keeping it tightly scoped to the notify-on-finish problem?

 

Straight answer: I'm scoping Pushary to make the "was it good" check fast, not to render the verdict itself, at least at the layer I'm building. The notify-on-finish piece is the wedge, the handoff payload, diff, tests, what's risky, is meant to collapse the time-to-evaluate, so you're judging from the first second instead of reconstructing context. But that's compressing the eval, not performing it, and I want to be honest about that line rather than pretend a notification layer secretly grades quality.

That said, you just named the thing I keep circling: context evaporates when the terminal closes, and a notification by definition lives at the exact moment of finish and then dies too. So a pure notify tool inherits the same amnesia it's supposed to fix. Wiring memory plus execution plus review into one durable layer so the answer survives the terminal closing is the correct shape, and it's the part I think notification-only tools quietly fail at.

The cost-per-approved-output metric is the sharpest idea in this whole thread, full stop. Raw token spend measures activity, which is noise. Cost-per-approved-output measures the only thing that matters, output that actually shipped, and the fact that it surfaced a power-law, a few people producing most of what holds up, is exactly the kind of truth that's invisible from notifications and invisible from token dashboards. You measured the denominator everyone else ignores.

So genuine question back, since you've clearly run further down this: how do you define "approved" cleanly enough to be the denominator? Merged-to-main is concrete but lagging, human-thumbs-up is fast but noisy. Where did you land on what counts as approved, and did it survive contact with people trying to game it?

 You're poking exactly where it's still soft, so let me be precise about the line today.

Right now "approved" in NEST is a state-transition event with an actor attached — who resolved it, when, and at what scope (one-shot vs session-wide for agent actions; submitted → approved with an optional rejection reason for time/work logs). That gives us a clean, auditable denominator: a real person moved the artifact forward, and we know who and when. It's concrete, not a vibe.

Where you're pushing is exactly the edge we haven't closed yet, and I won't pretend otherwise: today approval is binary. We don't yet capture "approved-but-heavily-edited" as its own bucket, there's no edit-distance or rewrite-ratio on what happens after the approval, and approval quality isn't paired against cost yet. So the gaming question you asked — does it survive someone rubber-stamping weak output? — the honest answer is the current model wouldn't catch that cleanly. A stamp is a stamp.

That post-approval layer is exactly what we're building right now — capturing what happens to the artifact after the approval, so a rubber-stamp that gets rewritten downstream stops counting as a clean approval. It's close. Happy to show you where it lands once it's running against real usage — you've thought about the noise problem harder than most people in this thread, and I'm curious whether the approach survives your scrutiny.

 

@caro_fogli That's a clean answer and the honesty about the soft edge is exactly why it's worth engaging with. The state-transition-with-actor denominator is the right primitive: a real person moved the artifact forward, you know who and when, it's auditable rather than vibes. That gets you provenance, which is real. What it can't get you, and you said this plainly, is whether the thing they moved forward was any good. A stamp is a stamp. Provenance tells you a decision happened, not that it was correct, same gap that bites everywhere in this space.

The post-approval layer you're building is the right answer to that, and I think it's sharper than even you're pitching it. The insight is that quality is a lagging signal, so you can't measure it at approval time no matter how good your notification is. The truth about whether an output was good is written later, in what survives. If an approved artifact gets quietly rewritten downstream, the approval was hollow regardless of how confident the stamp looked. Measuring the fate of the artifact instead of the moment of approval is the correct move, and the lag is the feature, not a bug to engineer away.

One trap I'd flag before you ship it, because it's the same false-positive shape that came up elsewhere in this thread: a downstream rewrite is ambiguous about why. It could mean the approver rubber-stamped weak output, or it could mean requirements changed and the artifact was correct-at-the-time and simply got superseded. Edit-distance alone conflates "the approval was bad" with "the world moved." If you penalize both as failed approvals, you'll start dinging people for approvals that were right when they made them, which is unfair and also teaches the wrong lesson. The disambiguator I'd reach for is time-proximity plus intent: a rewrite twenty minutes later smells like the approval was wrong, a rewrite three weeks later smells like a pivot. Weight the signal by how soon the rework lands, and you separate "bad call" from "stale call" without asking anyone to hand-label it.

Which connects to the part I'd resist: don't make the human tag "approved-but-heavily-edited" as its own bucket. You'll never get clean self-grading, nobody flags their own rubber-stamp. Infer the grade from what survives rather than asking the approver to confess. Keep approval binary at the moment, and let downstream survival turn it into a distribution after the fact. That respects your "a stamp is a stamp" honesty while still extracting the quality signal, you just stop trusting the stamp and start trusting the artifact's fate.

Where this rhymes with what I'm building: my handoff-and-risk-flag at approval time is the leading indicator, the cheapest preview of a bad call before it lands. Your post-approval survival metric is the lagging confirmation of whether the call held up. They bracket the same decision from opposite ends in time, and honestly the dream is one timeline per artifact that runs from "agent flagged this as risky" through "human approved at this scope" to "survived / got rewritten in twenty minutes." Take the offer, genuinely, show me where it lands against real usage and I'll throw my hardest gaming scenarios at it.

One question back, since you've run furthest on this: when an approved artifact does get rewritten downstream, can you currently tell who rewrote it, the same approver catching their own miss, or someone else cleaning up after them? Because "approver fixes their own stamp fast" and "someone downstream silently eats the cost of a bad approval" are very different organizational signals, and only the second one is the rubber-stamp problem you actually want to surface.

 Verified this against the actual code before answering, because it's a sharper question than it looks and the honest answer is "it depends on the artifact."

For structured work — tasks, docs, admin actions — yes, I can answer your question today. Every post-approval mutation is logged with the actor and a before→after, so I can tell whether the person who approved it is the one who later reworked it, or whether someone downstream quietly cleaned up. That's exactly the "approver fixes their own miss" vs "someone else eats the cost" distinction you drew, and you're right that only the second one is the rubber-stamp signal worth surfacing.

On identity — this is where it's stronger than I'd have guessed before checking: each person gets their own namespace per project, so in normal use the actor on these logs is the individual, not a team blob. The only way that goes fuzzy is if someone deliberately shares a token across people, and at that point you've opted out of attribution yourself. So for the structured artifacts, the "who" is real at the person level.

Where it genuinely breaks down is unstructured output — chat messages, session state, the actual code. Messages are append-only with no editor attribution, and code provenance lives in git, not in us. So the downstream-survival signal isn't uniform across artifact types yet, and I won't pretend otherwise — that's the real gap.

On your edit-distance point — you're completely right and it changed how I'm thinking about this. Conflating "the approval was bad" with "the world moved" is exactly the trap, and time-proximity-plus-intent is the cleaner cut: fast rework smells like a wrong call, slow rework smells like a pivot. I'm taking that. Genuinely — when the post-approval layer is running against real usage I'll show you where it lands, and you can throw your gaming scenarios at it.

Completely relate to the "half your brain monitoring" thing. You think you're doing deep work but part of your attention is always reserved for checking the terminal.

 

That reserved attention is the real tax, and almost nobody prices it correctly. People count the minutes spent checking the terminal, but the actual cost is the background process that never shuts off, you're never fully in the deep work because a thread of you is permanently on terminal-watch duty. It's not the checking that's expensive. It's that you can never stop being ready to check.

And the cruel part is that standby thread is running even when the agent's fine. You pay the attention tax on every minute, not just the minutes something happened. That's the whole pitch for getting it off your shoulders: not "save you the clicks," but let you actually close that background process because you trust something else will tap you when it matters. You can't go deep until you're allowed to forget.

Out of curiosity, do you find that standby thread ruins focus work specifically, or does it leak into everything, like you're half-watching the terminal even during a meeting or lunch? Trying to gauge whether it's a deep-work problem or an always-on one.

The learning angle is underrated here.

When you're new to AI coding agents, the wait-and-miss loop isn't just wasted time — it breaks the feedback you need to actually learn what the agent is doing. Agent gets stuck, waits 30 min, you come back to a half-finished state and have no idea what decision it was waiting on.

I started adding checkpoint comments to my CLAUDE.md: "after each major step, describe what you did and what you're about to do next." Not a notification system, just making the agent narrate. It doesn't solve the async problem, but it means when I do check back, I can actually see where it got stuck and why — which is how I'm learning what these tools can and can't handle autonomously.

What's the most common approval request you see people missing?

 

Direct answer: the most-missed approval isn't the scary one, it's the boring clarifying question. People instinctively watch for the destructive stuff, rm, force-push, dropping a table, those get attention. The one that silently eats 30 minutes is "the requirement was ambiguous so I paused to ask which way you meant," or "should I use library X or Y." It doesn't look dangerous, so nothing flags it, so it doesn't alarm anyone, and the agent just sits there politely waiting on a question that would've taken you ten seconds. The dangerous approvals get caught. The trivial-looking ones are where the time actually dies.

Your narration trick is exactly right and more clever than you're giving it credit for. "After each major step, describe what you did and what you're about to do next" converts the agent's silence into a readable trail, which is half the entire problem solved with a prompt. You built the "what did it do" half by hand. The half you correctly note it doesn't solve is the async one, you still have to come back to read the narration. The narration makes the check fast. It doesn't make the check come to you.

That's basically the seam Pushary sits in: take the trail you're already making the agent write and push the state-change moments out of it, so "it's about to do something / it got stuck and here's the decision it's waiting on" finds you instead of waiting for you to wander back. You've already done the hard prompt-engineering half. The missing piece is just delivery.

And big agree on the learning angle being underrated, when you're new, the narration trail is how you build the model of what these things will and won't handle alone. The wait-and-miss loop doesn't just waste time, it severs the cause-and-effect you need to actually learn. Curious, now that you've been narrating for a while, has it changed what you trust the agent to do unsupervised, or are you still surprised by where it gets stuck?

Same problem here. Running Claude Code, Cursor, and Copilot means I have 3 different tabs just to track what is running. What helped was adding a cost dimension to the awareness problem - we started using aicosts.ai to track spend per-agent. Turns out some agents were burning way more tokens than others on the same task. Now I can see both status and cost in one place.

For just status, have you tried something simple like a shared terminal dashboard? We use tmux with named sessions per agent.

 

Straight answer to your question: yeah, tmux with named sessions is genuinely good and it's where I'd start too, it's the cleanest single-machine version of this. Named, colored, one per agent, you can eyeball the grid. The two places it stops scaling are the ones you'd predict: it's still manual polling (you have to look at the grid for it to tell you anything), and it lives on one machine, so the second you walk away from the desk the dashboard you built doesn't follow you. Great while you're sitting in front of it. Useless the moment you're not, which is exactly when the agent goes quiet on an approval.

The cost dimension is the more interesting thing you said, honestly. "Same task, wildly different token burn across agents" is a real signal hiding in plain sight, it means the expensive agent is either taking a worse path or thrashing, and you'd never catch it from status alone because both finish. Status tells you it's done. Cost tells you whether it was done efficiently or it flailed its way there. Two different truths, and most people only watch one.

That's actually converging with something else in this thread, a few people landed on cost-per-useful-output rather than raw spend as the metric that matters, because raw tokens measure activity and approved output measures value. Pairing your per-agent cost with status in one view is the right instinct, and the next step is tying cost to whether the output actually shipped.

For Pushary I'm scoped on the status/approval half, status, blocked, what-changed, across tools and onto your phone, rather than the cost half, but the two clearly want to live in the same inbox eventually. Curious, when you found an agent burning way more tokens on the same task, was it taking a genuinely worse approach, or just looping and retrying? Trying to figure out if token-burn is a quality signal or just a noise signal.

I use codex and I have a codex hook setup


#!/bin/bash

EVENT="$1"

MESSAGE="$2"

osascript -e "display notification \"$MESSAGE\" with title \"Codex: $EVENT\""

afplay /System/Library/Sounds/Glass.aiff >/dev/null 2>&1 &

 

That's a clean little hook and I respect the minimalism, twelve lines, fires a native notification, plays Glass.aiff so you get the audio cue too. For one tool on one Mac while you're sitting at it, that's honestly hard to beat, no SaaS required.

Three ceilings it hits, all of which you've probably half-noticed: it's Codex-only (your hook doesn't know Claude Code or Cursor exist, so you'd be maintaining three of these), it's osascript so the notification dies at the edge of that machine and never reaches your phone, and the payload is just a $MESSAGE string, it tells you something happened but not whether the agent finished clean, got blocked, or what it changed. It's a doorbell. It rings the same whether it's a package or a fire.

That's basically the gap Pushary fills, same hook idea you've already wired, but the event carries a real handoff (done vs blocked vs what-changed), works across Codex/Claude Code/Cursor instead of one, and lands on your phone instead of dying on the desktop you walked away from. You've built the trigger. The missing parts are reach and payload.

Genuine question since you're clearly comfortable in the hook layer: what EVENTs are you actually firing on right now, just task-complete, or have you got it hooked into the approval/blocked moment too? The blocked one is harder to detect from a hook and it's the one that actually saves time.

 interesting, just took at look at it. I'll give it a spin.

Right now i use only 1 coding agent - given the complexity of my work I often tend not to leave the agent alone or give permissions to making changes. the event hook has worked well for me so far.

i have also added in the to write changes to a file and i have a file hook too with a one-liner on what changed.

 

Appreciate you giving it a spin, and I'll be honest with you about where it will and won't earn its keep for your setup, because credibility matters more to me than an install.

One agent, high supervision, no autonomous change permissions on complex work, that's the correct call, not a limitation. When the work is genuinely intricate, babysitting is cheaper than cleaning up a confident wrong turn, and your event hook plus sitting-right-there is already a tight loop. For that exact mode, Pushary's marginal value is honestly thin, you're not the guy losing 35 minutes to a silent terminal, because you never left. The tool shines when you start trusting the agent to run unattended or you add a second and third, which is the moment your current hook-per-tool approach starts to strain. So spin it up, but the real test is the day you decide to walk away or parallelize.

The part I'd actually flag: your writing changes to a file plus the file hook emitting a one-liner on what changed is a hand-built handoff trail, and it's the good half. You've independently built the "what changed" payload that I keep saying a notification should carry. You're closer to the right architecture than most of the autonomous-six-agents crowd, you just don't need the delivery layer yet because you haven't left the room.

Curious where your line is: is it the complexity specifically that keeps you from granting change permissions, or have you been burned by an agent making an unsupervised change you couldn't easily unwind? Trying to understand whether "stay in the room" is a trust problem that better visibility could eventually loosen, or a fundamental "this work needs a human in the loop, period."

I use codex to help with my projects, and I can review the code every time when it updated something. So I feel that I'm still the owner of the project, and knows where this project will land.

 

"I'm still the owner of the project and I know where it'll land" is the entire point, and you've protected it the most reliable way there is: reviewing every change so nothing enters the codebase you haven't seen. At one agent with review-on-every-update, that ownership feeling is real and earned, not a vibe.

The interesting question is what happens to that feeling when you scale. The whole reason ownership stays intact for you is that you can personally review everything in real time, one agent moves at a pace one human can fully watch. The thread's pain starts the moment that stops being true: six agents move faster than you can review live, and the ownership feeling erodes not because you stopped caring but because you literally can't see everything anymore. You become a bottleneck or you lose the plot, pick one.

That's exactly the feeling Pushary is trying to preserve at scale, let you keep being the owner who knows where it lands, by surfacing what each agent changed and flagging the stuff that actually needs your eyes, so "review everything" becomes "review the things that matter" without losing the thread. Same ownership, just affordable past one agent.

Genuine question: at one agent reviewing every update, would you even want more agents, or is the deliberate single-stream pace part of how you keep that ownership clear? Some people run six to go fast, some run one on purpose, curious which camp you're in.

I think this is pretty much unsolved still but maybe also more of a planning and workflow issue. It almost feels like the moment you start working in the agentic world you turn into a manager. What I've seen work improve my workflow is leaning heavily into planning and documentation sessions. I try to plan out milestones and epochs (similar to what we did in Jira or Linear boards), plan what can be done in parallel and define a solid eval / QA set. Between this and staying active in the hand-off this is where I had the most success building stuff that checks the marks when it comes to reliability systems. Excited to see what you folks are releasing to help with this!

 

"The moment you start working in the agentic world you turn into a manager" is the cleanest one-line diagnosis in this thread, and it reframes the whole problem correctly. People keep trying to solve this as an engineering tooling issue when half of it is just management, and nobody handed us the management training. The skills that suddenly matter, scoping work, defining done, planning what runs in parallel, are exactly the things a good eng manager does, and exactly the things ICs spent years avoiding.

Your planning-heavy approach is the right one and I'll push it one step further: good planning is preventative, it reduces the number of interruptions by removing ambiguity up front, which is genuinely the highest-leverage move. But it caps out, because no plan survives contact with execution perfectly, and the residual, the ambiguous call the plan didn't anticipate, still surfaces mid-run. So planning shrinks the interruptions, it doesn't zero them. The two halves are "plan well so it asks less" and "see clearly when it does ask," and you need both.

Which is why the line you almost threw away, "staying active in the hand-off," is the one I'd underline. That's the half that's still manual for most people, and it's exactly where Pushary lives, making the handoff legible so the manager-you can stay active in it without sitting in the terminal.

Your milestones and eval set define what good looks like, the handoff layer tells you whether each agent hit it. Planning and visibility, same job from two ends.

Genuine question, since you're running this like a Linear board: do you find the agents are good enough at self-reporting against your milestones, or do you still have to translate "what it did" back onto "which epoch this was" yourself? That mapping seems like where the manager tax quietly hides.

 I found that in dev workflows a set of research, specs, plans, tests & PRs works quite well when met with defined coding conventions. But I don't expect the agents to figure everything out by themselves. There are still a lot of decisions that I'd like to be part of the loop and I think this is where something like Pushary will be interesting. I believe a real tough questions to crack is to define when to notify especially in long-horizon tasks.

 

You've handed me the actual hard problem, so let me not pretend it's easy. "When to notify on long-horizon tasks" is the question that makes or breaks this whole category, because the two failure modes are both fatal: over-notify and you're a captcha, people mute the channel, and then the one alert that mattered dies in a silenced inbox. Under-notify and the agent quietly makes a call you'd have wanted in on, which on a long-horizon task is worse than usual, because a wrong-but-unflagged decision at hour one compounds silently through hours two through five before you ever see it. The cost of a missed notification scales with the horizon.

Here's where I've landed, and your own workflow is half the answer. The notify decision shouldn't key off event type ("it finished a step"), it should key off the decision's reversibility, blast radius, and the agent's confidence. Cheap-to-undo, low-blast, high-confidence: stay silent, just log it. Hard-to-undo or touches something load-bearing or the agent's genuinely unsure: escalate. The trap is that this asks the agent to self-assess its own risk and confidence, which is exactly the thing models are worst at. You cannot fully trust the agent to decide when it needs you, because the dangerous case is precisely when it's confidently wrong and therefore won't flag itself.

Which is why the fix is partly to take that judgment away from the agent at runtime and move it to plan time, and that's where your specs-and-plans discipline becomes the secret weapon. The notify-worthy boundaries get declared up front, the same way you'd write "stop before the migration, stop before touching auth" into a plan, so the gates are human-defined checkpoints rather than the agent's in-the-moment guess. Your research/specs/plans/tests workflow is basically already authoring those decision gates. The product's job is just to fire reliably at the ones you marked and stay dark everywhere else. Long-horizon tasks don't need more notifications, they need the right three, placed by you in advance, plus a runtime backstop for the genuinely unanticipated irreversible call.

So my honest answer is it's half a policy problem solved at planning time and half a heuristics problem at runtime, and the people with your kind of spec discipline make the first half almost free. Question back: when you write your plans, do you already annotate which steps are decision-points you want to be pulled into, or is that judgment still living in your head and applied ad hoc? Because if it's already in the plan, that's the exact signal a tool should consume, and if it isn't, I'm curious what it'd take to make writing it down feel worth the friction.

First
Previous
•••
567
•••
Next