LLM-as-a-judge reliability

LLM-as-a-judge reliability is the task- and protocol-specific stability and validity of model-based evaluation: whether a judge's scores remain consistent under irrelevant variation and track the quality or criterion they are intended to measure. A score is evaluation evidence under its stated conditions, not truth by itself.