AgenticSTS: A Structural Fix for Token Bloat in Game-Playing Agents

If you've deployed an AI agent on any task that unfolds over many steps — a game, a multi-stage workflow, a long-running automation — you've almost certainly hit the same wall: the context window fills up. Every observation, every action, every model response gets appended to a growing log until you're burning through hundreds of thousands of tokens per session, inference costs spiral, and reasoning quality degrades. AgenticSTS offers a concrete architectural answer to this problem, and its results on the card-based roguelike Slay the Spire 2 make a compelling case for rethinking how agents manage memory.

This tutorial walks through the AgenticSTS memory layer design, explains why it keeps prompts stable at approximately 5,000 tokens instead of the 500,000 tokens that naive logging approaches can reach, and gives you a blueprint you can adapt for your own agent deployments.

What You'll Build — and What You Need First

By the end of this tutorial you'll understand how to:

• Decompose agent memory into five distinct structural layers
• Write update logic that keeps each layer bounded in size
• Assemble a final prompt from those layers at inference time
• Validate that token counts remain stable across arbitrarily long episodes

Prerequisites: You should be comfortable with LLM API calls, basic Python, and the concept of a ReAct-style agent loop (observe → think → act). No game-development experience is required — Slay the Spire 2 is just the test environment; the pattern applies anywhere.

Step 1 — Understand Why Chat Logs Explode

The default pattern for most agent frameworks is append-only: every turn, you push the latest observation and the model's last response onto a list, then send the whole list as the conversation history. This works fine for short tasks. For long ones, it's a disaster.

Consider a game agent playing through a 40-combat run of Slay the Spire 2. Each combat might involve 15–30 decision steps. Each step produces a game-state description (deck contents, enemy intents, current HP, relics, status effects) plus the model's reasoning trace. Multiply that out and you're looking at thousands of message turns before the run ends.

Naive logging approaches can push prompts past 500,000 tokens per session — roughly 375,000 words of context the model must attend to on every single inference call.

This creates three compounding problems:

1. Cost: Token pricing is roughly linear, so a 100× token expansion is a 100× cost expansion.
2. Latency: Larger contexts mean slower prefill, especially on models without efficient KV-cache sharing.
3. Reasoning degradation: Research on long-context models consistently shows attention dilution — early, important information gets drowned out by recent noise.

AgenticSTS's insight is that the full log is almost never what the agent actually needs. What it needs is structured, bounded summaries of different kinds of information.

Step 2 — Map the Five Memory Layers

AgenticSTS replaces the monolithic chat log with five separate memory stores, each with its own update cadence and size budget. Think of them as different timescales of agent knowledge.

Layer 1: Static Game Knowledge

This is read-only reference information — card descriptions, relic effects, enemy move patterns, game rules. It doesn't change during a run. You load it once at agent initialization and include a relevant subset in each prompt (retrieved by keyword or embedding similarity based on what's currently in play).

Size budget: 800–1,200 tokens of retrieved context per inference call.

python def retrieve_static_knowledge(game_state: dict, knowledge_base: list[dict]) -> str: relevant = [ entry for entry in knowledge_base if any(card in game_state["hand"] for card in entry["related_cards"]) or any(relic in game_state["relics"] for relic in entry["related_relics"]) ] return "\n".join(r["text"] for r in relevant[:10]) # cap at 10 entries

Layer 2: Run-Level Summary

A compact, overwritten-not-appended summary of the current run's strategic state: deck archetype, key relics acquired, current HP and max HP, floor number, boss identity. This gets regenerated (not extended) after each combat.

Size budget: 200–400 tokens. One structured paragraph or a short bullet list.

python def update_run_summary(run_state: dict) -> str: return ( f"Run summary — Floor {run_state['floor']}/55. " f"Deck: {run_state['deck_archetype']} ({len(run_state['deck'])} cards). " f"HP: {run_state['hp']}/{run_state['max_hp']}. " f"Key relics: {', '.join(run_state['key_relics'][:5])}. " f"Next boss: {run_state['upcoming_boss']}." )

Layer 3: Combat Working Memory

This is the only layer that resembles a traditional short-term log — but it's scoped to the current combat only and is wiped at combat end. It holds the last N turns of play: what cards were played, what the enemy did, what status effects are active.

Size budget: 600–900 tokens. Keep only the last 5–8 turns.

python from collections import deque

class CombatMemory: def init(self, max_turns: int = 6): self.turns = deque(maxlen=max_turns)

def record_turn(self, turn: dict):
    self.turns.append(turn)

def to_prompt_string(self) -> str:
    lines = []
    for i, t in enumerate(self.turns):
        lines.append(
            f"Turn {t['number']}: Played {t['cards_played']}. "
            f"Enemy used {t['enemy_action']}. "
            f"Result: {t['outcome']}."
        )
    return "\n".join(lines)

def reset(self):
    self.turns.clear()

Layer 4: Episodic Lessons

This is where AgenticSTS gets genuinely interesting. After each combat, the agent (or a lightweight summarizer call) extracts one or two lessons — generalizable strategic insights — and appends them to a capped episodic store. Old lessons are evicted when the store is full, or ranked by recency × relevance.

Size budget: 400–600 tokens. A rolling list of ~8–10 lessons.

Example lessons the agent might generate:

• "Ironclad Bash + Inflame combo is strong against high-HP elites; prioritize Inflame upgrade at campfires."
• "Cultist fight: don't waste AoE on first turn; single target removes the ritual buff threat."

python class EpisodicMemory: def init(self, max_lessons: int = 10): self.lessons: list[str] = [] self.max_lessons = max_lessons

def add_lesson(self, lesson: str):
    self.lessons.append(lesson)
    if len(self.lessons) > self.max_lessons:
        self.lessons.pop(0)  # evict oldest

def to_prompt_string(self) -> str:
    return "Lessons learned this run:\n" + "\n".join(
        f"- {l}" for l in self.lessons
    )

Layer 5: Immediate Observation

The raw current game state — what's in your hand right now, enemy HP and intent, your current HP, active buffs and debuffs. This is fully replaced every turn. It's the only truly ephemeral layer.

Size budget: 300–500 tokens.

Step 3 — Assemble the Bounded Prompt

At inference time, you concatenate all five layers in a deliberate order: stable context first, dynamic context last. This mirrors how humans brief themselves before a decision — background knowledge, then current situation.

python def build_prompt( static_kb: str, run_summary: str, episodic: EpisodicMemory, combat_memory: CombatMemory, current_observation: str, task_instruction: str, ) -> str: sections = [ f"## Game Knowledge\n{static_kb}", f"## Run State\n{run_summary}", f"## Lessons\n{episodic.to_prompt_string()}", f"## Recent Combat History\n{combat_memory.to_prompt_string()}", f"## Current Situation\n{current_observation}", f"## Task\n{task_instruction}", ] return "\n\n".join(sections)

With the size budgets above, the assembled prompt lands at roughly 2,300–3,600 tokens for the memory content alone. Add a system prompt and a chain-of-thought reasoning prefix and you're comfortably under 5,000 tokens — regardless of how many turns or combats have elapsed.

AgenticSTS keeps prompts stable at approximately 5,000 tokens throughout an entire run, compared to naive approaches that exceed 500,000 tokens — a 100× reduction in context size.

Step 4 — Handle Layer Transitions

The trickiest part of implementation is defining clean transition events — moments when one layer gets updated or reset. For a game agent, these map naturally to game structure:

For non-game agents, map these to your own task structure: a customer support agent might reset working memory per ticket, update a session summary per conversation, and maintain a lessons layer of common issue patterns.

Step 5 — Validate Stability

Before deploying, instrument your agent to log token counts per inference call. You're looking for a flat distribution — token count should not trend upward over time.

python import tiktoken

enc = tiktoken.encoding_for_model("gpt-4o")

def count_tokens(prompt: str) -> int: return len(enc.encode(prompt))

In your agent loop:

for step in episode: prompt = build_prompt(...) tokens = count_tokens(prompt) assert tokens < 6000, f"Token budget exceeded: {tokens}" response = llm_call(prompt)

If you see token counts creeping upward, the culprit is almost always one of two things: the episodic lesson store isn't evicting correctly, or the static knowledge retrieval is returning too many entries. Both are easy to fix with tighter maxlen caps.

The Results: Why This Architecture Works

The AgenticSTS project tested this approach on Slay the Spire 2 — a game that demands long-horizon planning, deck-building strategy, and adaptive decision-making under uncertainty. The results were stark:

The AgenticSTS agent wins 6 out of 10 games on Slay the Spire 2. Competing agents using conventional context management win none.

The win rate gap isn't just about token efficiency — it's about reasoning quality. When an agent's context is bloated with 500,000 tokens of undifferentiated history, the model struggles to locate the information that actually matters for the current decision. Structured memory layers solve this by ensuring the model always sees a curated, relevant view of its situation.

This is a critical ai agent deployment best practice: don't treat memory as a passive accumulation problem. Treat it as an active information architecture problem.

Adapting This Pattern Beyond Games

The five-layer structure maps cleanly to other long-running agent applications:

• Coding agents: Static layer = codebase documentation and API references; run summary = current task spec and progress; combat memory = last N file edits; episodic = lessons from past debugging sessions.
• Research agents: Static = domain knowledge base; run summary = research question and current hypothesis; working memory = last N search results; episodic = dead ends and promising directions.
• Customer support agents: Static = product documentation; run summary = customer profile and issue category; working memory = current conversation turns; episodic = resolution patterns that worked.

In each case, the key discipline is the same: every layer must have a defined update rule and a hard size cap. The moment you allow any layer to grow unboundedly, you've reintroduced the problem AgenticSTS was designed to solve.

Key Takeaways

• Token bloat is architectural, not incidental — it's the predictable result of append-only logging in long-running agents.
• Five bounded memory layers (static knowledge, run summary, working memory, episodic lessons, current observation) replace the monolithic chat log.
• Prompt size stays flat at ~5,000 tokens regardless of episode length, cutting inference costs and preserving reasoning quality.
• Transition events — not timers — are the right trigger for updating each layer.
• Instrument token counts in your agent loop before production deployment; a simple assertion catches regressions early.

The AgenticSTS results on Slay the Spire 2 demonstrate that structured memory isn't just an efficiency optimization — it's what separates agents that can actually complete complex, multi-step tasks from those that collapse under the weight of their own history.

Sources:

• AI agents win at Slay the Spire 2 after researchers replace growing chat logs with structured memory — The Decoder

Last reviewed: July 12, 2026