When the System Prompt Vanishes: A Silent Failure in Semantic Kernel

• 2025-01-31 · Microsoft corrects the same defect on the .NET side in PR #10344 — the documented reference for what correct behavior looks like.
• 2025-06-27 · A community user files issue #12612: the Python ChatHistoryTruncationReducer deletes the system prompt. The report sits open, unsurfaced at runtime, for months — the .NET fix never having been ported to Python.
• 2026-03-01 · Hermes Labs reproduces the Python deletion, traces the root cause to the summarization-only extract_range helper, ports the .NET approach to the truncation reducer, and opens PR #13610 with regression tests. The same diagnosis surfaces that the sibling summarization reducer carries the same defect; the PR documents it and scopes its repair to a follow-up to keep the change focused.
• 2026-03-19 · Microsoft maintainers review and merge PR #13610. The Python truncation reducer now preserves the system prompt across reduction; the flagged summarization defect remains open for the follow-up.

The shape worth naming: a defect that throws no exception and logs no warning can sit live in a widely-used SDK for months, reported but unfixed, even after the same behavior has been corrected in a sibling SDK — because nothing surfaces it at runtime. The engineering job here was to reproduce the Python failure, confirm it against the .NET reference, and produce a patch a maintainer could adopt without re-doing the work.

The same binary-gated discipline Hermes Labs runs as a diagnostic pipeline on private client stacks also governs how it produces a public upstream fix: pass a gate, or stop. No gate is skippable. No gate is judgment-graded. The intent is that an opinionated process produces a falsifiable artifact at each step, so the diff at the end is one a maintainer can adopt on its own merits.

Each gate emits a falsifiable artifact, so the work can be re-checked rather than trusted. The proprietary tooling that runs the pipeline stays in-house.

The defect was the truncation reducer routing through a summarization-only helper that strips system and developer messages. The fix stops that routing and preserves the system prompt explicitly. The target behavior did not have to be invented: Microsoft had already corrected the same defect on the .NET side in PR #10344, which served as the verified reference for what correct looks like. The truncation reducer now locates and retains the first system or developer message:

# Preserve the first system/developer message so it is not
# lost during truncation (matches the .NET SDK behavior).
system_message_index = next(
    (i for i, msg in enumerate(history)
     if msg.role in (AuthorRole.SYSTEM, AuthorRole.DEVELOPER)),
    -1,
)
system_message = history[system_message_index] if system_message_index >= 0 else None

The reduction index calculation takes a new has_system_message flag so the preserved prompt is accounted for in the target count, with a guard added so a small target_count cannot produce an IndexError:

if has_system_message:
    target_count -= 1
    if target_count <= 0:
        logger.warning(
            "target_count after accounting for system message is %d; "
            "reduction will keep only the system message.",
            target_count,
        )

The patch touched the truncation reducer and the shared reducer utilities. The merged PR updated the existing truncation tests and added new ones covering system messages, developer messages, the no-system case, and the target_count=1 edge. The .NET fix gave the reference behavior, and the Microsoft maintainers reviewed and merged the Python port. The diagnosis also reached the sibling summarization reducer, which carries the same defect through the same helper; the PR documents that and scopes the repair to a follow-up to keep the change focused, so its tests are unaffected here. The community report and the .NET fix had not flagged the Python sibling, which is the measure of how far the diagnosis ran past the reported bug.

For a team that runs a history reducer to keep long conversations inside the context window, the cost of this failure is every long conversation in which the model ignored its system prompt: the safety framing, the tool-use rules, the persona, the task constraints. There is no error to catch, no metric that moves, no log to grep. A short test conversation never grows long enough to trigger truncation, so the deletion is invisible at runtime and surfaces only in long-running production sessions that are hardest to inspect. The failure is reproducible and testable once you know to look, which is precisely why a targeted regression test pins it and why the diagnosis is the work: the bug hides not because it cannot be tested, but because nothing prompts you to write that test.

This case is public, so the credit is shared. The symptom was reported by a community user, and the correct behavior was already established on the .NET side. The Python port carried that behavior over, held up in a Microsoft maintainer's review, and merged. When Hermes Labs audits a private stack, the same discipline turns to discovery, and the deliverable is the receipts: the reproduction script, the falsification log, the failing-then-passing regression test, and the verified contract behind the fix. A buyer can read the report cold and decide whether the failure applies to their stack and whether the audit checked the right things.

This is the same pipeline Hermes Labs runs whether the target is a chat history reducer, a retrieval layer, a prompt template, an agent harness, or a tool definition. The systematic failure categories vary; the gate structure does not. Our upstream contributions are listed at /open-source/contributions.

If you ship an LLM-based product where the model is supposed to follow a system prompt, ask what happens to that prompt when the conversation gets long. A silent instruction relaxation does not throw, does not log, and does not move a dashboard. We run this diagnostic pipeline against production AI stacks under NDA.

References: SK #13610 · SK #12612 · .NET #10344 · PR filed under our founder's GitHub handle, roli-lpci · Upstream contributions: /open-source/contributions