AI Reliability Glossary

Glossary

Operational definitions for Hermes Labs work on AI reliability, epistemic failure modes, retrieval, agents, and language-runtime systems. These terms name failure surfaces and control surfaces in AI systems. This is not a general AI dictionary.

Seeing one of these failure modes in production? Discuss a reliability review →

Core concepts

Epistemic Engineering

Epistemic Engineering is the practice of engineering how AI systems handle evidence, uncertainty, sources, justification, and meaning across real workflows. For Hermes Labs, this work happens primarily at the language and runtime layer (prompts, retrieval, memory, policies, rubrics, traces, and tool schemas) rather than in model weights.

See also

Language as runtime execution layer

In modern AI systems, prompts, instructions, retrieved context, memory, summaries, rubrics, policies, and tool schemas can function as part of the system's execution path, not merely as descriptions around it. Hermes Labs refers to this as the language runtime execution layer: the operational layer where meaning, constraints, evidence, and behavior are shaped before and during model use.

See also

Silent AI failure mode

A silent AI failure mode is a failure where an AI system returns a plausible-looking output while a consequential error remains hidden. Typical forms include omitted evidence, uncalled tools, softened instructions, lost constraints, or unjustified certainty. These failures can pass demos and narrow tests while surfacing only later in real use.

Related terminology

silent failure
false success
hidden failure
undetected failure

Not the same as

hallucination
explicit runtime error

See also

Epistemic failure mode

In AI systems, an epistemic failure mode is a failure in how the system handles evidence, uncertainty, sources, contradiction, absence, or justification. It differs from ordinary factual error because the content may be partly correct while the system's confidence, scrutiny, source handling, or evidential framing is wrong.

See also

Generative horizon

The generative horizon is the present operative configuration through which a language-model agent interprets and generates: instructions, prior turns, retrieved records, tool results, summaries, corrections, and the statuses they carry. The represented situation is not only what the model computes about; it is part of what the model computes through. The horizon exists within a single generation and does not require persistent memory. The distinct downstream process in which a retained output changes a later horizon is recursive interpretive conditioning. The term names a situated condition, not a hidden component inside the model.

Recursive interpretive conditioning

Recursive interpretive conditioning is the feedback process in which a generated interpretation, once retained as context, memory, policy, evidence, or authorization, alters the conditions of later interpretation and action. Ordinary error propagation copies a mistake through a chain; recursive conditioning changes the conditions under which later claims are produced and judged, even when the original wording disappears. The proposition can remain visible while its contingency disappears.

Applied hermeneutics

Applied hermeneutics translates the study of interpretation into engineering requirements for systems in which language is part of the operational loop. It asks how a system should represent the difference between an observation and an inference, a retrieval and an endorsement, a summary and a source, a correction and a supersession, and it binds a representation's authority to evidence proportionate to what accepting it permits.

Canonical epistemic failure modes

These terms come from the Hermes Labs taxonomy of epistemic failure modes in large language models.

Null-Result Asymmetry

Null-Result Asymmetry is a measured tendency to assign a null or negative finding less conclusion-consistent probability than a matched positive finding under otherwise identical conditions. The same system that states a positive result plainly will hedge the corresponding negative one, even when the evidence of absence is clear. This blocks automating clean-bill-of-health work in compliance and review.

Source-Status Credibility Bias

Source-Status Credibility Bias is the tendency to scrutinize a claim less when it is attributed to a high-prestige source and more when the identical claim comes from a low-prestige one. Swapping the cited source, with the claim unchanged, shifts whether the model challenges or accepts it. Prestige, a surface signal, ends up standing in for evidence.

Agency Dissolution

Agency Dissolution is the softening of who did what under social or politeness pressure, where a model turns settled, authoritative findings into hedged, agentless allegations. “The investigation concluded fraud” becomes “the report suggests potential concerns,” and both the actor and the certainty quietly disappear. Automated summaries then understate risk to the people who act on them.

Performative Hedging

Performative Hedging is the use of hedging language as a social signal rather than a calibrated statement of confidence. Qualifiers like “it is worth noting” or “arguably” perform caution without tracking the model's actual uncertainty. Because readers treat hedges as confidence information, decorative hedging quietly misinforms the decision that follows.

Constraint Evasion

Constraint Evasion is surface-level compliance with a stated constraint while its intent is violated. The letter of the instruction is met (a banned word is absent, a format is followed) while the purpose behind it is not. Constraints that can be satisfied in letter but not in spirit give false assurance that a control is working.

Silent Instruction Relaxation

Silent Instruction Relaxation is the weakening of a constraint across turns without acknowledgment. The instruction still sits in context but no longer binds behavior, and nothing flags that it has lapsed. Multi-turn agents drift away from their guardrails precisely when no one is re-checking the early instructions.

Related terminology

instruction drift
context drift
agent drift
instruction-following degradation

Not the same as

Constraint Evasion
prompt injection

See also

Controversy-Truth Conflation

Controversy-Truth Conflation is the use of controversy markers such as “debated” or “contentious” as a proxy for low factual confidence, regardless of whether the underlying claim is actually contested. Disagreement about a topic gets mistaken for uncertainty about a fact, so the model softens well-established findings that happen to sit in charged areas.

Null-result omission

Null-result omission is the downstream operational failure where a system drops the fact that a relevant search, test, or retrieval returned nothing, and proceeds as if the absence were irrelevant. Null-Result Asymmetry names the measured pattern; null-result omission names the operational failure it produces, where absence-based evidence is dropped from the output.

Context and meaning preservation

Hermeneutic Drift

Hermeneutic Drift is a shift in what the system takes the task, document, or referent to be about as context is retrieved, summarized, or carried across turns. A model answers about the wrong document or entity because recency or adjacency pulls the latest-retrieved context to the foreground; the words of the question stay the same while the referent moves.

Related terminology

agent drift
semantic drift
context drift
behavioral drift

Not the same as

model drift
retrieval mutation

See also

Context integrity

Context integrity is the degree to which relevant meaning, qualifiers, and constraints stay intact as context is retrieved, summarized, stored, transformed, and reused. A qualifier that changes the answer either survives or is lost along the way. Because later steps act on the context they inherit, degraded context produces plausible answers built on a damaged premise.

See also

Retrieval mutation

Retrieval mutation is any meaningful distortion introduced between an original source and the retrieved context a system actually uses, including truncation, smoothing, reframing, selective quoting, or a dropped decisive qualifier. The retrieved text can look faithful while no longer meaning what the source meant, and the system then reasons over the mutated version as if it were the source.

See also

Reduction drift

Reduction drift is the loss or reweighting of meaning when richer material is compressed into a smaller representation such as a summary, score, memory item, or rubric output. Each reduction step can quietly change emphasis, so summarization and scoring are treated as part of the language runtime layer rather than as neutral plumbing.

Related terminology

context compression
summarization loss
context compaction
lossy memory compression

Not the same as

semantic drift
retrieval mutation

See also

Telemetry-to-Claim Gate

The Telemetry-to-Claim Gate is a proposed operational framework, from the Precise Records study, for structuring and recording the evidentiary basis of a telemetry-based claim. Before a metric is promoted into evaluation, routing, governance, or redesign, the operator states the claim as a tuple of metric, analytical unit, construct, population, window, denominator, and interpretation, then tests provenance, coverage, linkage, validation, and sensitivity, and records a disposition.

See also

Context engineering

Context engineering is the design and management of the instructions, retrieved evidence, memory, tools, permissions, summaries, and other runtime state a model receives as work progresses.

Related terminology

prompt engineering
context management
agent harness
memory management

Not the same as

prompt engineering
context integrity

See also

Agent reliability and evaluation

Behavioral Canarying

Behavioral Canarying is a pre-execution prompt-injection sensing pattern that exposes untrusted input to a sacrificial model with no application authority, then analyzes the model's response for evidence of compromise before the primary agent acts. Routing disposition and inspection coverage are reported separately; the method is a sensor, not a security guarantee.

Related terminology

prompt-injection detection
indirect prompt injection
behavioral probing
canary analysis

Not the same as

canary tokens
input classification
sandboxing

See also

Tool Differentia

Tool Differentia is the distinguishing information that neighboring AI-agent tool descriptions provide, or fail to provide, about when one tool should be selected instead of another. LintLang's H1.6 performs a bounded deterministic pairwise check under a finite term model; a clean result does not prove semantic distinguishability or correct runtime selection.

See also

Inspection coverage

Inspection coverage records whether the evidence-producing check that a verdict depends on actually examined the relevant material. A workflow may continue while coverage is degraded or partial; that continuation must not be represented as a clean inspection result.

See also

Epistemic integrity

In Hermes's proposed operational usage, epistemic integrity is the degree to which an AI system preserves the evidence, provenance, uncertainty, and status needed to justify its claims and actions as information moves through a workflow. It is not presented as a standard or a universal metric.

Related terminology

epistemic reliability
evidence integrity
provenance
agent auditability

Not the same as

factual accuracy
context integrity

See also

Agent drift

Agent drift is a progressive change in an agent's behavior or decision process away from its intended or previously stable operation across extended interactions. It is a broad emerging term that can include semantic, coordination, and behavioral manifestations; it is not equivalent to Hermeneutic Drift.

Related terminology

semantic drift
behavioral drift
context drift
coordination drift

Not the same as

model drift
Hermeneutic Drift

See also

Agent instruction linting

Agent instruction linting is static analysis of language-bearing files that configure an AI agent, such as AGENTS.md, CLAUDE.md, skills, tool descriptions, and policies, before the agent runs. This is an emerging descriptive category; a clean lint result does not establish runtime correctness or safety.

Related terminology

prompt linting
static analysis
agent harness
configuration linting

Not the same as

prompt testing
runtime guardrails
agent observability

See also

Agent auditability

Agent auditability is the degree to which an AI agent's consequential behavior can be reconstructed and examined from scoped records of inputs, state, tool calls, decisions, and outcomes. It concerns reconstructable evidence for a particular review, not a general compliance status.

Related terminology

traceability
provenance
agent observability
evidence integrity

Not the same as

observability
logging

See also

LLM-as-a-judge reliability

LLM-as-a-judge reliability is the task- and protocol-specific stability and validity of model-based evaluation: whether a judge's scores remain consistent under irrelevant variation and track the quality or criterion they are intended to measure. A score is evaluation evidence under its stated conditions, not truth by itself.

Related terminology

model-based evaluation
rubric-based evaluation
evaluator calibration
evidence-grounded evaluation

Not the same as

inter-rater agreement
factual correctness

See also

Indirect prompt injection

Indirect prompt injection occurs when an AI system encounters attacker-controlled instructions embedded in external content, such as webpages, documents, email, retrieved data, or tool output, and treats them as instructions during task execution. Detection can surface possible compromise, but it does not by itself solve the attack.

Related terminology

prompt injection
prompt-injection detection
context poisoning
agent security

Not the same as

jailbreak
direct prompt injection

See also

Terminology note

Earlier or adjacent Hermes Labs materials may refer to null-result bias. In this glossary, Null-Result Asymmetry refers to the measured pattern, while null-result omission refers to the operational failure where absence-based evidence is dropped.