Blinding in Diagnostic Device Studies: Why Independent Review Can Change What Results Mean

By the Connected Diagnostics Evidence Team

What “Blinding” Means in a Study Like This

Blinding (also called masking) means keeping certain people in a study from knowing which result a device gave, or which test a patient actually received. The idea is simple: if you know the answer in advance, that knowledge can quietly change what you notice, how you interpret an unclear case, or how you report a result — even without meaning to. Reading a study means asking who knew what, and when they knew it.

Who Can Be Blinded — and Who Usually Can’t

In a study of a treatment, it’s often possible to keep the patient and the person giving the treatment from knowing which group is which. Diagnostic device studies are different. FDA guidance on designing pivotal device studies notes that a doctor running the diagnostic test almost always has to see the result to act on it, so that person typically cannot be blinded.

What often can be blinded is a separate person: a third-party evaluator who checks the case afterward without being told what the device originally reported. This is one of the main ways a diagnostic study builds in independent review even when full blinding isn’t possible for everyone involved.

The Comparison Point: Why It Has to Be Independent

Every diagnostic accuracy study needs something to compare the device’s result against — FDA calls this the reference standard: the best available method for deciding whether a patient actually has the condition being tested for. A meaningful reference standard has to be applied and interpreted without knowledge of what the device under study said. If the person applying the reference standard already knows the device’s answer, that knowledge can bias their judgment toward agreeing with it, which makes the device look more accurate than it really is.

When a full reference standard isn’t practical for every patient, FDA statistical guidance describes a common workaround: applying it only to a subset, often the cases where results disagreed. This introduces its own bias, called verification bias, unless the analysis is specifically adjusted to account for it.

Why Independent Review Can Change the Reported Numbers

FDA’s statistical guidance on reporting diagnostic test results describes a practice called discrepant resolution: when a new test and a comparison test disagree, a third “resolver” test is used to decide which one was right. It can seem reasonable, but the guidance is direct that using those resolver results to revise the original results table produces a misleading picture, because agreeing cases are assumed correct without being checked, while only disagreeing cases get a second look. That one-sided review can make a device look more accurate than an unbiased comparison would show.

This is a concrete example of how independent, unbiased review changes what a result means. A number presented as “accuracy” or “agreement” can shift substantially depending on whether the comparison was truly independent of the device being tested — not because the device changed, but because how it was checked changed.

Common Assumptions vs. What the Evidence Shows

• Assumption: If a study reports high accuracy, the device is accurate. What the evidence shows: Accuracy numbers only mean what they claim if the comparison method was applied independently of the device’s result — otherwise the number can be inflated.
• Assumption: Blinding either happens or it doesn’t. What the evidence shows: Diagnostic studies often blind some roles (a reviewing evaluator) while others (the clinician running the test) cannot realistically be blinded at all.
• Assumption: When two tests agree, they’re both correct. What the evidence shows: FDA’s guidance shows agreement and correctness are different things — two tests can agree with each other and still both be wrong.
• Assumption: A larger study fixes bias. What the evidence shows: A bigger sample size reduces random error, but it does nothing to remove bias from how a comparison was set up.

The Limits of This Evidence

Blinding and independent review reduce certain kinds of bias, but they don’t commitment a study is free of others. A study can use a well-blinded, independent reference standard and still be affected by which patients were selected, how representative those patients are of real-world use, or how the device’s instructions for use were followed during testing. Blinding is one piece of a larger picture, not a stand-alone commitment of accuracy.

This article explains how to read the methods section of a diagnostic device study. It is not medical advice, and it does not evaluate or endorse any specific device, test, or company.

A Conversation Worksheet: Questions to Ask When Reading a Diagnostic Study

These questions can help you or a clinician evaluate what a diagnostic study’s results actually support:

1. What was compared against the device’s result — a recognized reference standard, or another test that hasn’t been independently verified?
2. Did the person applying that comparison know the device’s result before making their own judgment?
3. Were all patients checked against the comparison method, or only some — such as only the disagreements?
4. Is the reported number labeled “sensitivity and specificity” (compared to a true reference standard) or “agreement” (compared to a non-reference method)? These mean different things.
5. Were patients across the full range of disease severity included, or mostly clear-cut cases?

Where This Fits

This guide is part of how Connected Diagnostics Evidence approaches source evaluation. For a broader introduction to how this publication reads and verifies evidence, see our Start Here page.

Sources

This article is based on two FDA resources: the CDRH Learn transcript on design considerations for pivotal medical device studies, and the FDA’s statistical guidance on reporting results from diagnostic test studies.

This article is for general education about how diagnostic device studies are designed and reported. It is not medical advice and does not diagnose, evaluate, or recommend any product. Connected Diagnostics Evidence is an independent editorial publication at CellScope.com and is not affiliated with, and does not continue the operations, products, research, or staff of, the former CellScope company. Last updated: September 9, 2026.