Why I built Inkspect: how do I know the AI is really done?
An agent says the task is done. I want to see the evidence before giving it the next one. A real bug and fix from building Inkspect explain why.
I give an AI agent a task. It changes the code, runs some checks, and tells me the work is done. Before I give it the next task, I have one more question: what evidence shows that it is done?
The gap between two tasks
I started asking this because of problems I had faced in my own projects. An agent would finish a task, but more problems would appear when I checked the result. Sometimes a requirement was missing. Sometimes the checks did not cover the behavior I cared about. This repeated problem is why I built Inkspect.
When I work with coding agents, each task often builds on the task before it. If I accept unfinished work, the next task starts from that point. Finding the problem later means going back to something I thought was complete.
So the moment between two tasks matters to me. I want to know what was checked, what problems were found, and what still needs attention. That information helps me decide whether to fix something or continue.
Reading the changed lines is part of this process. Understanding the original task and the surrounding code matters too. I wrote about this idea in A diff is only half the story.
A real example from building Inkspect
One example comes from an earlier Go version of Inkspect’s review engine. During development, a local Codex source review found a problem with saved review data. This was a local development review, rather than an Inkspect production review.
The engine recorded model IDs returned by the AI service. These labels help explain which models were involved in a review. But this information could be lost when a review ended early.
The engine saved progress at different stages. These saved points, called checkpoints, included usage data but missed the model IDs. If a run failed, was cancelled, or went through recovery after an interruption, its final report could lose those IDs.
The information had existed during the run. It did not reach the saved report.
From a finding to a tested fix
The finding gave us a clear question to test: after an interrupted run, does the final report still contain the recorded model IDs? A test checked three cases. All three failed before the correction.
Before the fix
--- FAIL: TestStageCheckpointsPreserveModelIdentity/failed
--- FAIL: TestStageCheckpointsPreserveModelIdentity/cancelled
--- FAIL: TestStageCheckpointsPreserveModelIdentity/recovered
After the fix
--- PASS: TestStageCheckpointsPreserveModelIdentity/failed
--- PASS: TestStageCheckpointsPreserveModelIdentity/cancelled
--- PASS: TestStageCheckpointsPreserveModelIdentity/recoveredThe fix was to include the session’s recorded model IDs in each stage checkpoint. That gave the failure and recovery paths the information they needed to build the final report.
A follow-up source review checked the correction too. The tests ran the cases. The AI reviewer inspected the code; it did not run those tests. The saved review and test records show a specific problem, a test that exposed it, a correction, and a passing result for the same cases.
That gave us a clear reason to close the finding. These records describe the earlier Go engine during local development, rather than a test of the current production engine.
The loop I use now
Today, I use Inkspect as part of my development loop. I give the coding agent a task. When it finishes, Inkspect reviews the result and reports findings with evidence. When needed, fixes and further checks follow. The agent gets a handoff when the work is ready for the next task.
Inkspect can also make fixes when needed and tell the agent when it is ready to continue. This makes review part of the work between tasks. Its findings help decide what happens next.
What I want from a review
- What is the problem?
- When does it happen?
- What evidence supports it?
- What change could fix it?
- How can we check the fix?
The reviewer needs to support its own claims too. If something is unclear, I want to see that uncertainty. The checkpoint bug is a small example, but it explains why I care about this process. A clear finding helped turn a hidden gap into a test and a fix.
That is the kind of workflow I am building Inkspect around. When a coding agent says “done,” what do you check before giving it the next task?
This article was written with AI assistance, using the founder’s experience and saved review and test records from the project.
Every line. Worth a second look.