Developing a diagnostic test is only part of the challenge. Once you've built a sensor with excellent performance characteristics, a more important question emerges:
What does a positive test result actually mean?
Surprisingly, even a test with 95% sensitivity and 90% specificity can produce positive results that are wrong more often than they are right.
Understanding why requires familiarity with a few key concepts: the confusion matrix, sensitivity, specificity, prevalence, and positive predictive value (PPV). These statistical tools are essential for anyone developing medical diagnostics, biosensors, or electrochemical assays.
The Question That Catches People Out
Consider the following:
- Sensitivity: 95%
- Specificity: 90%
- Disease prevalence: 1% (1 in 100 people)
- A patient tests positive
How likely is it that the patient actually has the disease?
Many people intuitively answer:
- 95%
- 90%
- Something close to the test performance
However, the correct answer is only 8.8%.
That result surprises many clinicians, scientists and engineers alike.
To understand why, we need to start with the confusion matrix.
The Confusion Matrix: A Diagnostic Scoreboard
When developing a diagnostic test, we usually compare it against a known reference method or a population whose disease status is already known.
The results can be organised into a confusion matrix.
| Disease Present | Disease Absent | |
|---|---|---|
| Test Positive | True Positive (TP) | False Positive (FP) |
| Test Negative | False Negative (FN) | True Negative (TN) |
Each sample tested falls into one of these four categories.
True Positive
The patient has the disease and the test correctly identifies it.
False Positive
The patient does not have the disease, but the test incorrectly indicates they do.
False Negative
The patient has the disease, but the test misses it.
True Negative
The patient does not have the disease and the test correctly reports a negative result.
The confusion matrix is the foundation for calculating all major diagnostic performance metrics.
What Is Sensitivity?
Sensitivity measures how effectively a test identifies people who truly have the disease.
Sensitivity = TP / (TP + FN)
A sensitivity of 95% means:
If 100 people truly have the disease, the test correctly identifies about 95 of them.
Only around 5 cases would be missed.
What Is Specificity?
Specificity measures how effectively a test recognises people who do not have the disease.
Specificity = TN / (TN + FP)
A specificity of 90% means:
If 100 healthy individuals are tested, approximately 90 will correctly receive a negative result.
However, about 10 will incorrectly receive a positive result.
What Is Accuracy?
A term frequently used in diagnostic development is "accuracy."
Within the confusion matrix framework:
Accuracy = (TP + TN) / Total Population
For example:
| Metric | Count |
|---|---|
| True Positives | 95 |
| False Positives | 10 |
| False Negatives | 5 |
| True Negatives | 90 |
Accuracy = (95 + 90) / 200 = 92.5%
While that sounds impressive, accuracy alone does not tell the whole story.
Prevalence: The Hidden Variable
Prevalence is not a property of the test.
Instead, prevalence describes the population being tested.
It answers the question:
What proportion of people actually have the disease?
Examples:
- 1% prevalence = 1 person in 100 has the disease
- 30% prevalence = 30 people in 100 have the disease
This distinction is crucial because prevalence dramatically influences the practical usefulness of a diagnostic test.
The same test can perform very differently depending on the population in which it is deployed.
Positive Predictive Value (PPV)
Positive Predictive Value answers the question clinicians really care about:
If a test result is positive, what is the probability that the patient genuinely has the disease?
PPV = TP / (TP + FP)
Unlike sensitivity and specificity, PPV is strongly influenced by prevalence.
Example 1: A Disease with 1% Prevalence
Imagine testing 10,000 people.
How many actually have the disease?
- 100 have the disease
- 9,900 do not
Apply the sensitivity
- True positives = 95
- False negatives = 5
Apply the specificity
- True negatives = 8,910
- False positives = 990
Calculate PPV
PPV = 95 / (95 + 990)
PPV = 8.8%
The Surprising Result
Despite the test being:
- 95% sensitive
- 90% specific
A positive result means there is only an 8.8% chance the patient genuinely has the disease.
Most positive results are false positives.
Why This Happens
The issue is not the quality of the test.
The issue is prevalence.
When screening a very large population where few people actually have the disease, even a small false positive rate generates large numbers of incorrect positive results.
Those false positives can outnumber the true positives.
Example 2: A Specialist Clinic with 30% Prevalence
Now imagine the exact same test is used in a specialist clinic where 30% of people have the disease.
Testing 10,000 people gives:
- 3,000 disease positive
- 7,000 disease negative
Applying sensitivity:
- True positives = 2,850
Applying specificity:
- False positives = 700
PPV = 2,850 / (2,850 + 700)
PPV = 80%
Suddenly a positive test becomes highly meaningful.
Nothing changed about the test.
Only the prevalence changed.
Bayes' Theorem
The relationship between sensitivity, specificity and prevalence can be expressed using Bayes' Theorem:
[ PPV = \frac{Sensitivity \times Prevalence} {(Sensitivity \times Prevalence) + ((1-Specificity)\times(1-Prevalence))} ]
This allows scientists and clinicians to estimate the real-world value of a diagnostic before it is deployed.
Why This Matters for Biosensor Developers
At Zimmer & Peacock, we frequently work with developers creating electrochemical biosensors and diagnostic assays.
A sensor with excellent laboratory performance may still struggle in real-world screening applications if prevalence is low.
When developing a diagnostic, it is important to consider:
- Expected prevalence
- Intended use environment
- Screening versus specialist use
- False positive burden
- Clinical workflow after a positive result
- Overall health-economic impact
The best diagnostic is not simply the most sensitive sensor. It is the sensor that delivers meaningful information within the population where it will actually be used.
Related Resources
🔬 Biosensors
https://shop.zimmerpeacock.com/en-gb/pages/sensors
🧪 Recommended Chemicals & Solutions
https://shop.zimmerpeacock.com/en-gb/collections/chemical-and-solutions
📈 SenseItAll Platform
https://shop.zimmerpeacock.com/en-gb/products/senseitall-sia-generation-4-device-only
📧 Contact Zimmer & Peacock
https://www.zimmerpeacock.com/contact
Key Takeaways
✅ Sensitivity measures how well a test detects disease.
✅ Specificity measures how well a test excludes disease.
✅ Accuracy measures the proportion of correct results.
✅ Prevalence describes how common the disease is within a population.
✅ Positive Predictive Value (PPV) tells us how likely a positive result is to be correct.
✅ PPV depends heavily on prevalence.
✅ Even excellent tests can produce surprisingly low PPV when prevalence is low.
✅ Understanding prevalence is essential for translating biosensor performance into clinical utility.
Watch the video above for a full walkthrough of sensitivity, specificity, confusion matrices, Bayes' theorem, and positive predictive value in diagnostic testing.
