Those of us who are called as expert witness or report about our work in forensic situations often hear the term “likelihood ratio” (LR). It sounds scientific and does convey a quantitative measure of probability. However, a report from NIST cautions that the LR concept is overused in forensic settings and may not be justified as commonly practiced. (Note that LR has a different meaning in clinical diagnostic settings.)
In current use, the LR is used by forensic experts to combine results from various forensic evidence examinations (fingerprints, DNA, hair, etc.) into an overall estimate of the probability that these can be mapped to a particular individual. The problem is that exact matches to a unique person are not possible. Some go so far as to claim the LR is the only method for a technical expert to explain evidence to jurors or attorneys.
Recall that NIST is responsible for managing the Organization of Scientific Area Committees (OSAC) to put science back into forensic science. The press release cites a paper (Lund, S.P. and Iyer, H. Likelihood ratio as the weight of forensic evidence: a closer look. J. of Research of the National Institute of Standards and Technology, Oct. 12, 2017) that discusses problems with LR as commonly applied and other science-based approaches that are more accurate and reliable. The authors argue that the Bayesian decision theory breaks down when used to convey information interpersonally, such as courtroom testimony.
The problem is that an LR from one expert can differ widely with that from another seeing the same evidence. If you are a juror, which evaluation do you believe? Do you average the two? Also, remember that coincidence does not prove cause.
The authors maintain that for a technique or measure to be acceptable, it should be based upon measurements that can be replicated. LR fails this expectation. “It is not a standardized measurement.” The difference in LR values “estimated by two individuals may be substantial.”
Lund and Iyer caution that better methods of communicating confidence are being developed. However, until they are developed and tested, people should be aware that LRs should be used cautiously, if at all.
Robert L. Stevenson, Ph.D., is Editor Emeritus, American Laboratory/Labcompare; e-mail: [email protected].