Shepherd Technologists on Predictive Coding: Be Prepared
POSTED ON April 14

Welcome to the second post in a three-part series on predictive coding. Be prepared to be amazed, but first a cautionary tale.

With predictive coding, an attorney reviews documents for relevance to a case and codes a sample of them. That sample is then used to teach an automated predictive coding system the terms and conceptual patterns for the relevant case documents. The system segregates the wheat from the chaff by matching the on-target terms and patterns from the sample to the universe of documents. And, it zips through an enormous amount of additional data, delivering a more relevant set of files for the attorneys’ final review—all in record time at a lower cost. We often hear attorneys say they feel as if they gained additional brainpower.

But, here’s where the cautionary tale comes in: one cannot take shortcuts with the initial attorney review process. If an attorney does, they risk that additional brainpower veering off on a tangent and pulling in the wrong documents or excluding the right ones. We have heard of review attorneys who attempted broad bulk document selection rather than following repeatable, proven sampling procedures. They had terrible results and realized that they had to repeat their entire review process–a waste of time and brainpower.

Here are some thoughts from Shepherd’s Ben Legatt, Director of Client Consulting, and Brandon Ward, IS Director, about how to prepare for predictive coding success.
“Predictive coding is more than just pushing a button,” Legatt states. “There’s a common misperception that the process involves looking at just a few documents,” he continues. Instead, to give the system enough information, a “seed” or sample set needs to be created and then coded manually. The “seed” informs the predictive system. “The best practice for sampling is use of a random set or to create a set based on prior coding,” Legatt says.

The reviewer for the seed set should be a senior attorney who is very familiar with the case. As Legatt puts it, “The reviewer should understand everything about the case.” That person’s experience and knowledge will give the system the best start to identify relevant patterns in the data. Ideally, this initial review would be undertaken by one person to ensure consistency. “Every change affects the output and you want to arrive at a consistent result to achieve confidence in the process,” says Legatt. “The merits of the case can be so subjective,” Ward adds, so it makes sense to use a person who understands all sides of the matter.

The predictive model differs from the traditional discovery process, which often involved ranks of junior attorneys and which often involved ranks of junior attorneys and paralegals from day one of document review. But anyone’s eyes can glaze over while reviewing a stack of documents, so how does predictive coding deal with the human factor—especially when there’s only one human conducting the review? Relativity, a commonly used review platform—for among other things predictive coding–controls for that issue. The reviewer may be presented the same document multiple times to check if coding remains the same. These repetitions help the system confirm or update results to aid the development of relevant concepts.

Once through the initial review stage with the seed set, predictive coding really proves its worth. Next time, Ben and Brandon tell us what to expect as a predictive coding project goes into high production mode.

About the Author Chris

Author Avatar Christine Chalstrom is the Founder, CEO, and President of Shepherd Data Services, Trustee, Mitchell Hamline Law School and Adviser, Center for Law and Business. She has spoken widely on the Amendments to the Federal Rules of Civil Procedures, Digital Forensics, and eDiscovery best practices. Her credits include presentations to the American Bar Association, Association of Certified e-Discovery Specialists (ACEDS), Corporate Counsel Institute, MN Association of Corporate Counsel, MN Association of Litigation Support Professionals, MN CLE, Mitchell Hamline School of Law, Upper Midwest Employment Law Institute. She is an attorney, programmer, and forensic examiner.