97% after 8+ compactions
Same GPT model: Vibey still answers what the task, the rules, and the next step are. Claude Code 93%, Codex 53%.
Highest in 8 of 9
Three models, three measures. The exception: the long-term goal on Claude Sonnet 5.5, Claude Code’s native model (99% vs 96%), where Vibey keeps more of your requirements (89% vs 78%).
81% intent depth
Vibey’s memory captures why, not only what. GPT: Claude Code 20%, Codex 14%. Grok: 62% vs Pi 25%. Sonnet: 36% vs Claude Code 14%.
Memory after many compactions
After every compaction, each agent was asked three questions: what is the task, what are the rules, and what comes next. The charts show the share answered correctly, all three answers combined, grouped by how many compactions the session had been through. Every chart uses the same 40–100% scale.- GPT
- Grok
- Sonnet
GPT-6 Astra / GPT-6.1 Sol. The 8+ group has 3 sessions and 14 compaction points.
Head to head
Scores over every compaction point, in percent. The best score for each model is in bold.The exception. On Claude Sonnet 5.5, Claude Code’s native model, Claude Code states the long-term goal more often than Vibey: 99% vs 96% over all points, and 97% vs 93% after 8+ compactions. On the same model, Vibey keeps more of your requirements, prohibitions, preferences, and permissions: 89% vs 78%.
Each group’s size:
Beyond the task: intent depth
A memory can hold the task and still miss why you are doing it. Intent depth is the share of compactions whose memory captures, beyond the task, your grounded deeper intent: why, and what you ultimately want. An inference counts only if it is grounded in the record and never presented as your request or permission.
Exact intent-depth scores
Exact intent-depth scores
Method
- Sessions. 6 real long-running sessions from the Avibe team’s own development, imported as they were: 55 compaction points in all, the longest chain 14 compactions. The transcripts are private and are not published; only aggregate scores are.
- Same conditions. Every comparison uses the same transcript, the same compaction point, the same model, and high reasoning effort.
- Models. GPT-6 Astra, GPT-6.1 Sol, Claude Sonnet 5.5, and Grok 4.6.
- Judges. Two blind AI judges, Claude Opus 5.5 and GPT-6.1 Sol; every score is their average. A few Opus refusals were re-judged by GPT-6 Astra.
- Answer keys. Written beforehand from the raw record only (the long-term goal, the user’s requirements and permissions, and the next step), without seeing any candidate output.
- When. An internal evaluation run on October 21–23, 2026.
Setup notes
- GPT group: Claude Code and Codex reach the same GPT model through a gateway. Codex therefore uses its local compaction, not OpenAI’s server-side compaction. Claude Code requested high reasoning effort; the effort after the gateway’s translation was not separately verified.
- Sonnet group: prior model thinking was removed from the history on both sides, to avoid provider refusals.
- Grok group: two long sessions were cut on both sides from the point where the provider refused Pi’s requests.
- One session was removed from every group before the results were compiled.
- Each agent compacts in its own native way, with its own system prompt. This is a product comparison, not an algorithm ablation.
Limitations
- Scores come from AI judges and have not been human-reviewed. Rerunning the same version can move the Sonnet results by several points.
- The sample is small: 6 sessions and 55 compaction points. The 8+ groups hold 3 sessions and 14 points (GPT and Sonnet), and only 1 session and 4 points (Grok).
- Every session comes from the Avibe team’s own development work. Other kinds of work were not measured.
- Only Claude Code, Codex, and Pi were compared with Vibey. Other agents were not measured.
- The transcripts stay private, so the results cannot be reproduced from published data; only the aggregate scores are published.