Learn From Past Runs to Sharpen the Next Prompt
Opulent examines why a session succeeded or failed, extracts the patterns, and produces revised prompts that avoid dead ends and wasted compute.
Identify the expensive run
Connect GitHub, Notion, Linear, and Amplitude. The run you want to analyze should have a clear outcome and a clear compute cost that is out of line with the task size. Start from a session that succeeded but used more runs, tokens, or actions than expected.
The analysis is a playbook, a reusable, named set of steps that examines a run's history and extracts prompt and knowledge improvements.
Playbook: !prompt-review For a completed run that exceeded expected compute: 1. Read the run history: actions, tool calls, rewrites, backtracks. 2. Identify phase transitions and long exploration loops. 3. List dead ends: wrong approaches, unnecessary searches, repeated installs. 4. Compare the original prompt to the actual task needs. 5. Rewrite the prompt with missing context filled in. 6. Propose Knowledge entries for recurring context. 7. Suggest environment fixes for repeated setup failures. 8. Run the new prompt on a similar task and compare compute use.
Get the phase-by-phase breakdown
Opulent reads the run history and breaks down where time went. Common wastes include unnecessary exploration, picking the wrong implementation approach, and environment issues that force retries.
Run analysis: Task: Add pagination to GET /api/users Expected: ~12 units Actual: 42 units Inefficiencies found: 1. Unnecessary exploration, investigated 23 files looking for a pagination helper. Built pagination from scratch, then found src/utils/paginate.ts and had to refactor. Fix: mention existing utilities in the prompt. 2. Wrong approach, implemented cursor-based pagination. Tests failed because the API uses offset-based. Rewrote the whole implementation. Fix: specify "offset-based" in the prompt. 3. Environment issue, npm install failed twice due to missing .npmrc config for the private registry. Retried with workarounds. Fix: add registry config to repo setup.
Rewrite the prompt and compare
Opulent produces a rewritten prompt that includes the missing context: the pagination type, the existing utility, the reference pattern, the response shape, and explicit test criteria. It then runs the new prompt on a similar task and compares compute use.
Original prompt:
"Add pagination to GET /api/users. Run the tests when done."
Optimized prompt:
"Add offset-based pagination to GET /api/users.
Context:
- Use existing pagination utility at src/utils/paginate.ts.
- Follow the same pattern as GET /api/projects.
- Query params: page (default 1), limit (default 20, max 100).
- Response shape: { data: User[], meta: { page, limit, total, totalPages } }.
Implementation:
1. Add page and limit query params with Zod validation.
2. Use paginate() from src/utils/paginate.ts for the Prisma query.
3. Return the paginated response matching the shape above.
Testing:
- Run existing tests with npm test.
- Verify pagination with 0, 1, and multiple pages.
- Verify invalid page/limit params return 400.
- Do not open a PR until all tests pass."
Comparison:
| Dimension | Original | Optimized |
|-----------|----------|-----------|
| Pagination type | not specified | offset-based |
| Existing utility | not mentioned | src/utils/paginate.ts |
| Reference pattern | none | GET /api/projects |
| Response shape | not specified | explicit |
| Testing criteria | "run tests" | specific edge cases |
| Compute | 42 | ~12 |Turn findings into permanent fixes
A single prompt rewrite saves compute on one run. Turning the findings into permanent configuration saves compute on every future run. Add Knowledge entries for facts Opulent needs repeatedly, and fix environment issues in the repo setup.
Knowledge entries to add: 1. "All API endpoints use offset-based pagination. The shared utility is at src/utils/paginate.ts. See GET /api/projects for the reference implementation." 2. "Run tests with npm test. Run type checks with npm run typecheck." Environment fixes: - Add .npmrc config for the private registry. - Pin common dependencies in the environment image.
Sharpen the prompt-learning loop
When multiple runs are expensive, batch-analyze them together. Ask Opulent to find common patterns of wasted compute and propose Knowledge entries that prevent the issues across the team.
Add evals that measure prompt completeness: does the prompt name the existing utility, the reference pattern, the response shape, and the test criteria? A complete prompt gets a higher score.
The natural chain: when a prompt improvement becomes a standard pattern, update Add Unit Tests to Your Payments Service or Implement Bookings API from OpenAPI Spec guides; when the waste is in multi-strategy optimization, use Fix Checkout Latency with Three Competing Strategies; when the run involves product analytics, use Amplitude event data to validate the outcome.