Correction Memory and Feedback Loops

Prompt engineering gets you 80% of the way. Correction memory gets you the rest. Instead of endlessly rewriting system prompts to handle edge cases, you store human corrections and retrieve them as few-shot examples when similar situations arise. The agent sees what it got wrong before and how a human fixed it.

The Problem with Static Prompts

A well-crafted system prompt handles common cases well. But every deployment surfaces edge cases the prompt author didn't anticipate:

• A customer asks about a pricing exception that isn't in the standard docs
• The agent uses a tone that's technically correct but doesn't match the brand voice
• A product question has a nuanced answer that depends on the customer's specific plan

The traditional fix is to keep editing the system prompt — adding more rules, more examples, more guardrails. This works for a while, but eventually the prompt becomes unwieldy, conflicting, and hard to maintain.

Correction memory is a different approach: keep the system prompt lean and let specific corrections handle the edge cases.

How Few-Shot Correction Retrieval Works

The architecture has three components:

1. Correction Storage

When a human reviews and edits an agent response, the system stores a correction record:

{
  "trigger_message": "Can I get a discount on the annual plan?",
  "agent_draft": "I'm not able to offer discounts. Our pricing is listed on the pricing page.",
  "human_correction": "Thanks for asking about annual pricing! Annual plans already include two months free compared to monthly billing. For teams over 20 seats, we offer custom enterprise pricing — I can connect you with our sales team to discuss.",
  "category": "pricing",
  "agent_id": "pre-sales-agent",
  "timestamp": "2026-03-15T10:30:00Z"
}

The trigger message is the key — it's what gets matched against future incoming messages.

2. Similarity Retrieval

When a new message arrives, the system searches past corrections for similar trigger messages. This uses vector similarity (embedding the messages and finding the closest matches) or simpler keyword/category matching.

The top 2-3 matching corrections are retrieved and injected into the agent's prompt as few-shot examples:

Here are examples of how similar questions were handled previously:

User asked: "Can I get a discount on the annual plan?"
Correct response: "Thanks for asking about annual pricing! Annual plans already include two months free compared to monthly billing..."

Now respond to the current question:
User: "Do you do any deals for yearly subscriptions?"

3. Agent Response

The agent sees the corrections as concrete examples of what a good response looks like. It adapts its answer to follow the same pattern — matching the tone, level of detail, and approach.

This is more effective than adding a rule like "be helpful about pricing" because the agent sees the exact transformation from bad response to good response.

Designing the Feedback Loop

The feedback loop has four stages:

Stage 1: Agent responds. The AI generates a response based on its system prompt and any retrieved corrections.

Stage 2: Human reviews. In HITL mode, a human sees the agent's draft alongside the conversation context. They approve, edit, or reject.

Stage 3: Correction stored. If the human edited the response, the original and corrected versions are stored as a correction record.

Stage 4: Future retrieval. The next time a similar question comes in, the correction is retrieved and injected as a few-shot example.

What Makes Good Correction Data

Not all corrections are equally useful. The most valuable corrections are:

Substantive, not cosmetic. "Changed 'Hi' to 'Hello'" is noise. "Completely rewrote the pricing explanation to include the annual discount" is signal.

Specific to a pattern. A correction that applies to a class of questions (pricing questions, feature comparisons) is more useful than one that applies to a single unique situation.

Recent. Product information changes. A correction from six months ago about pricing might be wrong today. Implement TTL (time-to-live) on corrections or weight recent corrections higher.

Measuring Correction Impact

Track these metrics over time:

• Correction rate — percentage of agent responses that humans edit. This should decrease as correction memory builds up.
• First-time accuracy — for new question types the agent hasn't seen before, what's the accuracy? This tells you how well the base prompt performs.
• Repeat accuracy — for question types with existing corrections, what's the accuracy? This tells you how well correction retrieval works.
• Correction memory size — total number of stored corrections. After 100-200 corrections, you should see a meaningful drop in correction rate.

If repeat accuracy isn't improving, check that similarity retrieval is working correctly — the right corrections might not be getting matched to the right questions.

When to Update the Prompt Instead

Correction memory handles edge cases well, but some patterns are better addressed in the system prompt:

• Systemic issues — if the agent consistently gets the same type of thing wrong across many different questions, update the prompt rather than relying on dozens of individual corrections
• Policy changes — if your company changes its pricing, return policy, or product capabilities, update the prompt and expire old corrections
• Tone and voice — brand voice guidelines belong in the system prompt, not in corrections

A good rule of thumb: if you've stored more than 5 corrections for the same underlying issue, it's time to fix it in the prompt.

Summary

Correction memory transforms HITL from a bottleneck into a learning system. Every human edit makes the agent better at handling similar situations. Keep your system prompt lean and focused on general behaviour, and let correction memory handle the long tail of edge cases. Track correction rates to know when to promote patterns from corrections into the prompt itself.