Prompt Linters Do Different Jobs

Every AI system runs on written instructions. The system prompt, the descriptions that tell a model what each tool does, configuration files like AGENTS.md — the model reads those words and acts on them. They are part of the system’s runtime, not decoration around it.

That means mistakes in the words become mistakes in behavior. A vague tool description makes the model pick the wrong tool. A missing stop condition lets an agent retry forever. Two rules that quietly contradict each other leave the model to guess which one wins.

A prompt linter checks those instructions as text, before the model ever runs — the same way a code linter flags likely bugs before the program executes.

The reason to care is cost and timing. An instruction defect is cheap to catch as text and expensive to discover in production, where it surfaces as confusing behavior you have to debug backwards. Catching it early is the whole job.

In practice the name “prompt linter” gets applied to at least four different kinds of tools. They map onto stages every software team already knows: lint, test, validate, monitor.

Real tools exist for each layer, and they complement each other more than they compete. Promptfoo is the widely used option for prompt testing. Guardrails AI and NVIDIA NeMo Guardrails handle runtime validation. LintLang — our own tool, one option in the static category — lints instruction text. Some teams also have a model critique their prompts; that is a useful review practice, but its findings vary run to run, so treat it as advice rather than a check.

Persistent instruction files deserve the same care as prompts, because one defect in a shared file affects every session that loads it. Size is a real defect too — a single oversized AGENTS.md can crowd out the task itself, per OpenAI’s harness-engineering write-up.

One vague tool description shows why you may want more than one layer:

process_request: Handles the request appropriately.

A linter flags the text: the description never says which request, what the tool changes, or when to choose it. A test goes further and shows the consequence: on realistic cases, the model routes the wrong requests to it. A guardrail catches what slips through anyway: the tool cannot execute without valid parameters and permission.

Each layer answers a different question — is the text sound, does the model behave, should this action be allowed — and none of them answers the others. A team that only lints misses behavioral errors. A team that only tests keeps paying model calls to rediscover defects a text check would have caught for free. A team that only guards blocks bad actions without ever fixing the instructions causing them.

The strongest stack uses the cheapest reliable check at the earliest boundary, then adds the later layers where the risk justifies them.

“The instructions are vague or contradictory.” Static linting (e.g., LintLang).

“The model got worse after we changed the prompt.” Prompt testing with versioned test cases (e.g., Promptfoo).

“The model must return valid structured data.” Structured output where your provider supports it, plus runtime validation (e.g., Guardrails AI or NeMo Guardrails).

“The agent must never take a destructive action without approval.” Enforce that in the permission layer. A sentence in the prompt is a request, not a guarantee.

Prompt quality is not one score. A concise prompt can be unsafe; a detailed one can contradict itself; a clean-scanning one can request the wrong behavior. Start from the failure in front of you, and pick the layer that catches it earliest.

LintLang (current release 0.3.8) is Hermes Labs’s static linter for AI instructions. It flags patterns like vague or overlapping tool descriptions, missing stop conditions, and contradictory directives — deterministically and locally, with no model or network calls — and runs in CI against files like AGENTS.md.

That is the honest limit of the whole category: a linter sees text, and behavior also depends on the model, the assembled context, the tools, and the environment. So the right question is never “which prompt linter is best?” It is: which failure are we trying to catch, at which layer, and what will a passing check actually tell us?