Reading Diagnostic Studies: A Guide to Reading the Evidence

Maybe a school sent home a reading screening score and the letter did not say what it meant. Maybe a sales page cited a study for a classroom screener or an ear device, and the numbers sounded strong, but you could not tell what they covered. That uncertainty is normal, and it can be sorted out.

You do not need a statistics background to read these studies. You need a few checks, in the right order. A reading diagnostic study tells you how well a test sorts people into the right group, and you can judge that by checking what it measured, who it tested, where the cutoff sits and whether unclear results were counted.

Step 1: Start with the score you were given

You are starting with a reading score, a screener name or a test report in hand.

Reading assessments come in many forms. One example in the research is curriculum-based measurement of oral reading, called R-CBM, which tracks how a student reads aloud. A 2014 meta-analysis in the Journal of School Psychology pooled 34 studies on R-CBM used to screen whole groups of students, including 20 peer-reviewed articles, 7 dissertations and 7 technical reports.

The authors reported that the accuracy results mostly met the standards for an acceptable universal screener.

For the Foundations of Reading exam, the 3-2-1 rule, phonics policy and lists of reading skills, check your state education department or your child’s school. Those rules come from them, not from device research.

Once you know what the score is meant to predict, the next step is reading the numbers that measure accuracy.

Step 2: Read the two accuracy numbers

You want to know how often the test is right, and where it goes wrong.

In the R-CBM meta-analysis, predicting outcomes within a school year gave sensitivity of .80 to .83 and specificity of .71 to .73.

A meta-analysis of diagnostic accuracy is the kind of study that reports these numbers side by side.

A 2003 review in Neurologia on reading diagnostic test articles says the reader must check the accuracy numbers against a gold standard, meaning the best available reference for who really has the condition.

Next, check whether the people in the study look like the people the test will be used on.

Step 3: Check who was tested and how they were compared

You are looking for whether the study group matches your situation.

The 2003 Neurologia review names the key questions. Was the test result compared blindly and independently with the gold standard? Is the gold standard adequate? Was the test applied across an appropriate range of the condition? Was selection bias checked? The review also asks for enough method detail that someone could repeat the study.

Missing data matter too. A study that drops people without explaining why may report accuracy that does not reflect everyone who started it. The guide on missing data and withdrawn participants shows how to spot this.

Two more guides go deeper. The bias and representation guide covers studies whose people or settings do not match your use.

With the study group clear, the next question is where the line between positive and negative sits.

Step 4: Check the cutoff and the unclear results

Cut scores are the lines used to decide whether a result is positive or negative.

The R-CBM meta-analysis found that cut scores varied widely within grade and time of year. Its authors suggest that one cut score used everywhere may be too rigid, and that local cut scores may be necessary.

Unclear results need counting too. The guide on indeterminate results explains why unreadable images belong in the study counts. The guide on blinding explains why a reader who does not know the other result can change what the numbers mean.

Who reads the result matters. A 2018 systematic review, which is a structured search and summary of published studies, found that double reading (two readers on the same images) in radiology found discrepancies in 0.4% to 22% of cases, depending on setting. A 2017 study in the Journal of Nuclear Medicine found that readers with limited experience were more accurate when they had extra quantitative data, moving from 79.2% to 85.8% accuracy.

Now look at what the maker is claiming and what was actually tested.

Step 5: Check what the maker claims and what was tested

You are looking at an ad, a product page or a label, and want to know whether the claim matches the testing.

The bench, simulation and clinical evidence guide explains what each kind of device testing proves.

A maker’s claim is what the maker says. The intended use and FDA classification guide explains what a label is meant to cover.

The guide to reading a safety alert or recall shows how to read one.

If an ear is part of the question, a clinician is the next step.

Step 6: If an ear is the worry, bring in a clinician

If your question is about your ear and a device, a clinician is the person who looks at your ear and decides what is going on.

The guide on what can be mistaken for an ear infection and the guide on how to tell if you have an ear infection give you better questions to bring to a visit.

A guide on smartphone-enabled otoscopy lists what the evidence can and cannot yet show about phone-based ear cameras.

If you are ready to look at a specific product, the AudiQuiet guide on ingredients, amounts and evidence lays out the ingredients, their amounts and the evidence behind them. If the product is a supplement, note that a 2023 narrative review in Innovations in Pharmacy, a review article rather than a trial, reported 79,071 adverse events linked to dietary supplements sent to the Center for Food Safety and Applied Nutrition, part of the FDA, between 2004 and 2021.

Last, check the software, the stored images and the privacy terms before you rely on any result.

Step 7: Check software, AI and privacy before you trust a result

A connected device’s result can also depend on its software.

A 2023 review in the European Journal of Radiology on AI for breast tomosynthesis reported reductions in recall rates of 2% to 27% and reading times of up to 53% in the studies it covered. The same review says prospective validation is still needed to judge cost-effectiveness and real-world impact. The AI transparency questions guide lists what to ask about tools like this.

Software changes can change results. The software changes guide explains why version and workflow matter.

Your next step

1. Write down the score, product name or study title you are reading.
2. Ask what it was meant to predict, and in which group of people.
3. Find sensitivity, specificity and the cutoff, and ask who set the cutoff.
4. Check who was tested, how the comparison was made, and who was dropped.
5. Compare the maker’s claim with the bench, simulated and clinical testing it cites.
6. If an ear is involved, book a clinician visit and bring your notes.
7. For reading questions, ask your school which screener it uses and check your state education department for exam rules.

Every guide on this topic

Start with the study

• How to Read a Diagnostic-Accuracy Study: the four things to check first.
• Positive Predictive Value and Prevalence: why common and rare conditions change results.
• Bias and Representation in Connected Diagnostic Evidence: whether the study group matches your use.
• Reference Standards in Digital Otoscopy Research: what the comparison answer key is.

Comparison, missing data and blinding

• Verification Bias in Diagnostic Research: when not everyone gets the same comparison.
• Missing Data and Withdrawn Participants: who a study left out.
• Blinding in Diagnostic Device Studies: why independent review changes results.
• Bench, Simulation and Clinical Evidence: what each kind of testing proves.
• Analytical and Clinical Performance: lab accuracy versus real-world accuracy.
• Inter-Rater Agreement in Digital Otoscopy: whether readers agree with each other.
• Training and Instructions in Home Device Evidence: who operates the device, and how well.

Cutoffs and unclear results

• Diagnostic Cutoffs in Connected Devices: why a threshold changes the result.
• Indeterminate Diagnostic Results: why unreadable images count.
• Device Error Labeling and Workflow: what an error message should explain.

Claims, labels and recalls

• Intended Use and FDA Classification: what a label covers.
• Using FDA Device Databases: how to check a claim yourself.
• Reading a Device Safety Alert or Recall: what a recall notice says.
• Human Factors and Device Safety: why usability is a safety question.

Ear questions and products

• What Can Be Mistaken for an Ear Infection: look-alikes to raise with a clinician.
• How to Tell If You Have an Ear Infection: symptoms and when to see someone.
• Smartphone-Enabled Otoscopy: Clinical Evidence Limits: what phone-based ear cameras can show.
• Digital Otoscopy Evidence Limits: the limits of connected ear exams.
• Remote Ear Assessment and Follow-Up: how image review fits into care.
• AudiQuiet Ingredients, Amounts and Evidence: ingredients, amounts and the evidence behind them.
• Mobile Clinical Microscopy Point-of-Care Evidence: phone-linked microscopy in point-of-care settings.
• Mobile Clinical Microscopy Validation: how such tools are validated.
• Mobile Clinical Microscopy Laboratory Performance: lab results versus real-world use.

Software, AI, privacy and safety

• AI-Enabled Devices Transparency Questions: what to ask about AI readers.
• Software Changes: Version and Workflow: why updates matter.
• Clinical Decision Support Software: how decision software differs from a measurement tool.
• Privacy, Consent and Security for Exam Images: who can see your images.
• Cybersecurity Questions for Image Systems: what to ask about device security.

Connected Diagnostics Evidence is a publication. These guides are for education and do not replace care from a qualified professional.

By the Connected Diagnostics Evidence Team