Current models are more advanced than what we would see 3 years ago but how do we differentiate from a more articulate search engine, complete with the possibility of confidently repeating someone else's nonsense. When reviewing the paper, When Trust Meets Truth: Trust–Truth Separability in LLM-as-Judge. The researchers found that changing the attributed author influenced factual correctness judgments even when the answer itself remained unchanged.
That suggests source identity was affecting the model's assessment of truth. Can we know whether the model is actually evaluating the proposition, or does it use contextual signals as shortcuts for that evaluation.
The problem with scoring credibility
Suppose three sources make the same claim.
Source A publishes the original claim without supporting evidence.
Source B repeats A, adding an expert-sounding explanation.
Source C cites B and describes the claim as widely accepted.
An LLM retrieves all three. A simplistic credibility mechanism might interpret this as corroboration. Three sources agree, after all.
But there's only one independent claim, and it may be completely unsupported.
What should we expect from an LLM?
A capable research assistant should perform three distinct operations.
1. Gather: Retrieve relevant information, identify original sources, locate competing explanations and establish what's actually being claimed.
2. Evaluate: Examine the evidence, test the reasoning, identify dependencies between sources, distinguish observations from interpretations and investigate contradictions.
3. Synthesise: Present the best-supported conclusion, explain what justifies it, identify meaningful uncertainty and distinguish established findings from plausible hypotheses.
Current LLMs can perform all three to varying degrees. What they cannot guarantee is that their evaluation is independent, logically sound or grounded in evidence rather than plausible reconstruction.
Tests run based on the research paper
Could we design an evaluation where a model receives increasingly persuasive but unsupported claims, alongside increasingly obscure but verifiable evidence, and measure the point at which apparent authority overwhelms actual evidence?
Questions I will be testing:
Which system finds better evidence?
Which system reasons better over equivalent evidence?
Which system produces the better answer in its actual operating environment?
That would tell us considerably more about its reasoning reliability than asking whether it trusts humans more than AI.
Adjusting a few conditions, I will be running the below tests:
| Test | Conditions | What it measures |
|---|---|---|
| Equal evidence | Both models receive the same source documents, without browsing | Reasoning, contradiction detection, synthesis |
| Independent search | Each uses its own native search and retrieval tools | Real-world research effectiveness |
| Personalised assistance | Each uses its normal, explicitly permitted context and integrations | Practical usefulness, context handling and permission boundaries |
The hypothesis tested:
If the model's judgment tracks evidential strength or apparent authority.
Model consistently favouring prestigious attribution over stronger contradictory evidence.
Original Paper Analysis
Title: When Trust Meets Truth: Trust–Truth Separability in LLM-as-Judge
Authors: Xin Sun, Di Wu, Yuchen Guo, Jiahuan Pei, Isao Echizen, Abdallah El Ali and Saku Sugawara
Published: October 2026, EMNLP Findings; original preprint August 2026
Status: Peer-reviewed conference paper
Original: Read the paper
-
Can an LLM independently evaluate whether information is trustworthy and whether it's factually correct?
Or does its perception of the source contaminate its judgment of the evidence?
-
Researchers gave five LLM judges identical question-answer pairs, changing only whether the answer was attributed to a human or an AI.
The models included GPT-5.4, Claude Sonnet 4.6, GPT-OSS-120B, Llama 3.3 70B and Qwen3.6-35B.
They tested three domains: general knowledge, health questions and fact-checking.
Crucially, trust ratings and factual correctness judgments were obtained through separate requests, reducing the possibility that the models simply copied one judgment into another.
-
Identical answers were more likely to be judged correct when attributed to humans.
Across models and domains, human attribution increased:
Trust ratings by 0.57 points on a seven-point scale.
The probability of receiving a correct verdict by 3.92 percentage points.
False acceptance of incorrect answers by 3.67 percentage points.
The last result is particularly important.
The models weren't simply expressing a preference for human-written material. That preference affected whether they accepted objectively incorrect information.
The researchers also examined internal probability signals for the open-weight models and found corresponding shifts.
-
The experimental design is relatively strong because the factual content remains constant.
However, the human comparison is limited: 54 participants evaluated a small number of examples, and they weren't subjected to the same source-switching experiment.
The researchers therefore cannot establish that LLMs are more susceptible than humans.
Nor have they established the internal mechanism responsible.
My interpretation: This is evidence of provenance bias affecting factual judgments, not evidence that models cannot distinguish truth from trust under any circumstances.
