Genspark's Countermeasures for "Forgetting Instructions": Practical Techniques to Prevent Mistakes by Having Claude Repeat Instructions
Table of Contents
- Introduction
- What is the AI "Forgetting Instructions" Problem?
- Solution: Introducing Repetition Prompts
- Why Repetition is Effective for Claude
- Synergistic Effects with History Accumulation Strategy
- Secondary Effects of Repetition
- Summary
Introduction
When developing web applications with Genspark, have you ever experienced this?
- "I just explained the deployment procedure, but it's asking again."
- "I'm sure I told it important settings, but it forgets them in the next conversation."
- "It's a hassle to repeat the same precautions every time."
This problem is due to AI's context window limitations and lack of long-term memory.
This article introduces practical prompt engineering techniques that dramatically reduce instruction forgetting by making the Claude model obligated to "repeat".
What is the AI "Forgetting Instructions" Problem?
Context Window Limitations
LLMs (Large Language Models) have limitations in the amount of information they can process at once (context window).
- Gemini 2.0 Flash Thinking: Approximately 32,000 tokens
- Claude 3.5 Sonnet: Approximately 200,000 tokens
However, as conversations get longer, older information's priority decreases, and it is effectively "forgotten".
Specific Examples from the Development Field
Examples of "forgetting instructions" I've actually encountered:
- Forgetting deployment procedures: Even after being told "Access Cloudflare with your email address and API key," it attempts a different method during the next deployment.
- Ignoring naming conventions: Despite being instructed "Do not use full-width parentheses; use half-width parentheses," it reverts to full-width after a few attempts.
- Overlooking important matters: Even after being reminded "Always back up before making corrections," it immediately proceeds with the correction.
- Repeating but not executing: Even after repeating "Execute using procedure XX," it actually executes using a different method.
In other words, repetition alone is insufficient; it is necessary to explicitly state that it must understand and execute.
Solution: Introducing Repetition Prompts
The Power of "Repetition"
Even in human communication, important matters are confirmed by repeating them. This is also applied to AI.
Actual Prompt Example
Important: Access Cloudflare with your email address and API key.
Access information is in the API credentials folder.
Important: Please repeat the important matters. Do not just repeat, but understand and execute the content.
If you do not repeat, it will be considered forgotten.
Effects of the Repetition Prompt
As a result of introducing this prompt:
- ✅ Deployment procedure errors dramatically reduced
- ✅ No longer forgets important settings
- ✅ AI explicitly states "understood content" at the beginning of the conversation
- ✅ Now actually executes what it repeated
Why "Understand and Execute" is Important
Initially, it was just "Please repeat," but cases of repeating but not executing occurred frequently.
Example:
Me: "Please access Cloudflare with your email address and API key."
AI: "Understood. I will access with my email address and API key." (Repeats)
Actual: Tries to use an API TOKEN (Repeats but does not execute)
To prevent this problem, "understand and execute the content" was added to explicitly state AI's responsibility for execution.
Why Repetition is Effective for Claude
Characteristics of LLM's "Attention Mechanism"
LLMs pay high attention to recent utterances and emphasized information.
By the instruction "Please repeat," AI re-outputs the relevant information as its own utterance. This leads to:
- Information moves to the latest position in the conversation history
- Strengthened in AI's internal representation
- Becomes easier to refer to during the next response
Compatibility with Claude's Characteristics
Claude 3.5 Sonnet has characteristics of being faithful to instructions and excellent in context understanding.
When given a repetition instruction, it reliably repeats, making it ideal for context maintenance.
Furthermore, by adding "understand and execute," AI recognizes the repeated content as a guideline for action and refers to it during actual execution.
Synergistic Effects with History Accumulation Strategy
When combined with the "History Accumulation Strategy" (details here) introduced in a previous article, it becomes even more effective.
Two-Tier Memory System
| Method | Effect | Target |
|---|---|---|
| Repetition Prompt | Enhancement of short-term memory | Important matters in the current conversation |
| History Accumulation Strategy | Supplementation of long-term memory | Records of the entire past conversation |
Practical Example: Reliable Execution of Deployment Procedures
- First time: Explain "How to access Cloudflare" and have it repeated.
- Quality check: Record in the history accumulation system.
- Next deployment: Refer to past procedures from history and reconfirm with repetition.
→ Deployment failure rate dramatically reduced
Summary
4 Major Benefits of the Repetition Prompt
- Context Maintenance: Important information moves to the latest position in the conversation.
- Error Prevention: AI explicitly states "understood content."
- Execution Guarantee: Actually executes what it repeated.
- Efficiency: The "don't make me repeat myself" problem is resolved.
Ready-to-Use Template
Important: [Describe important matters here]
Important: Please repeat the important matters. Do not just repeat, but understand and execute the content.
If you do not repeat, it will be considered forgotten.
Secondary Effects of Repetition
The benefits of repetition are not limited to preventing AI from forgetting. It also has the following secondary effects:
1. Visualization of Hallucinations
You can visualize the effects of hallucinations that users would not have been able to know otherwise.
When using repetition, you can see that AI gradually summarizes or deletes the repeated content. Being able to roughly predict whether the AI has forgotten or is about to forget is also an effect of repetition.
2. Concise Feedback is Possible
When the repeated content has not been executed, a single phrase "Please check the repetition" is sufficient.
There is no need for lengthy re-explanations, and you can prompt corrections simply by having the AI refer to the repeated content.
3. Easy Recall of Memory
When repetition has not occurred, you can easily remind it by copying and pasting the previous repetition.
This saves the effort of re-typing important instructions, allowing for efficient work progress.
Next Steps
Use this technique to further streamline development with Genspark!
If AI performance evolves, such things might become unnecessary, but it seems we will need to skillfully deal with AI's forgetfulness for a few more years.