Why This Question Matters Before You Trust an Accuracy Number
By Connected Diagnostics Evidence Editorial Team | Updated September 15, 2026
You’re reading about a connected diagnostic device — maybe a digital otoscope, a phone-linked skin scanner, or a home test that pairs with an app — and the maker cites a study claiming the device is highly accurate. Before you weigh that number, there’s one question worth asking: did every person in the study get checked the same way to confirm whether they actually had the condition?
If the answer is no, the accuracy number may be less trustworthy than it looks. This is called verification bias, and it’s one of the most common — and least visible — problems in diagnostic accuracy research.
What “Verification” Means in a Diagnostic Study
To test how accurate a new device is, researchers compare its results against a reference standard. The FDA’s statistical guidance on reporting diagnostic test studies describes a reference standard as the best available method for establishing whether a target condition is truly present or absent — a single test, or a combination of methods, that does not depend on the result of the new device being evaluated.
In an ideal study, every participant gets both the new device’s test and the reference standard, so researchers can see exactly how often the device was right. In practice, that’s not always what happens.
Partial Verification: When Only Some People Get Checked
Sometimes the reference standard is invasive, expensive, or impractical to use on everyone — a biopsy, for instance, isn’t something researchers can ethically perform on every study participant “just to check.” When that happens, researchers may apply the reference standard to only a subset of people, often those whose device result was unclear or positive.
The FDA’s guidance addresses this directly. When a reference standard exists but using it on every participant isn’t feasible, the guidance describes this as a partial-verification or two-stage study design, and it specifically warns that calculating sensitivity and specificity with the ordinary formulas in this situation produces biased results — a problem it names verification, or work-up, bias. The guidance notes that correcting for this requires specific adjusted statistical methods, and recommends researchers consult a statistician before using this kind of design.
In plain terms: if only the people who tested positive on the new device get sent for confirmation, the reported accuracy can look better than it really is, because the study never checked whether people who tested negative were actually negative.
A Related Question Worth Asking: Was Everyone Confirmed the Same Way?
Beyond whether everyone was verified, it’s worth asking whether everyone who was verified was checked using the same confirmation method. If people who tested positive on the device were confirmed one way and people who tested negative were confirmed a different way, the two groups aren’t being held to the same standard — and any comparison between them may not mean what it appears to mean. This isn’t a claim about how any specific study you might encounter is designed; it’s a question to ask when reading one.
Why a Reporting Checklist Matters Here
Because these issues are easy to miss, diagnostic accuracy research has its own reporting standard: STARD (Standards for Reporting of Diagnostic Accuracy Studies), currently in its 2015 update. According to the EQUATOR Network, which maintains the official listing of health-research reporting guidelines, STARD’s stated objective is to improve the completeness and transparency of how diagnostic accuracy studies are reported, so that readers can judge the study’s potential for bias and how well it generalizes beyond the study group.
A study that follows STARD-style reporting gives an outside reader — including you — more of what’s needed to check for partial verification. A study that stays vague about its reference standard and how it was applied makes that much harder to assess.
A Worksheet for Reading an Accuracy Claim
When you come across an accuracy claim for a connected diagnostic device — in a study, a product page, or a press release — these questions can help you spot verification issues:
- What was the reference standard? Look for a named confirmation method (a specific test, exam, or expert panel), not just a vague claim of “verified results.”
- Did everyone get it? Check whether the reference standard was applied to the full study group or only a subset — sometimes described as “confirmed cases,” a number smaller than the total enrolled.
- Was it the same reference standard for everyone who got one? If people were confirmed using different methods depending on their device result, that’s worth flagging as a limitation.
- Does the source mention verification bias or work-up bias as a limitation? A study that names and discusses this issue is being more transparent than one that stays silent on it.
- How many people were enrolled versus how many were verified? A meaningful gap between those two numbers is worth noticing, even if the study doesn’t call attention to it.
What This Does — and Doesn’t — Tell You
Recognizing verification bias doesn’t tell you whether a specific device works or how accurate it actually is. It tells you whether a given accuracy claim was measured in a way that supports the claim. A device could still be genuinely useful even if one study behind it has verification gaps — and a device could be less reliable than a headline number suggests, for the same reason.
This article is general information about how diagnostic accuracy research is designed and reported. It is not medical advice, and it does not evaluate, endorse, or recommend any specific device, test, or manufacturer. If you’re weighing whether a diagnostic tool or result is right for your situation, that’s a conversation for a qualified healthcare provider who can look at your specific circumstances.
Where to Go Next
For background on how we evaluate and describe evidence on this site, see our Start Here guide and How We Research page. Connected Diagnostics Evidence is an independent educational publication and is not affiliated with, and does not speak on behalf of, any current or former operator of this domain.
Leave a Reply