I've always been suspicious of the claims of the "predictive coding" zealots. Every year it seems that there is a new buzzword in the field of Ediscovery. Technology Assisted Review - Check! Information Governance - Check! Cloud Computing - Check. Big Data - Check.
My friend John Martin did a magnificent job of distilling some of my concerns in his blog entry that can be found right here.
The Emperor has No Clothes - and PC Can't See Image-Only Documents
There are several parallels between predictive coding (AKA technology assisted review) and Hans Christian Andersons' tale, "The Emperor's New Clothes." In the story, two weavers tell the emperor they will make him a suit of clothes that will be invisible to those people who are unfit for their position, stupid, or incompetent. None of the emperor's subjects want to admit to those deficiencies so the emperor parades around with no clothes on until a child states the obvious - the emperor has no clothes.
In the case of predictive coding, its advocates have touted the efficacy of their approaches in white papers, blogs, and social media postings, and have practically created a separate industry to host conferences promoting the wonders of predictive coding. Few people want to ruin the moment or buck the trend by pointing out what is obvious when one considers the technology underlying predictive coding - it is completely dependent on having text to analyze. It will absolutely fail to analyze documents for which there is no text, and will do a miserable job where the text is of poor quality.
This might be just an esoteric debating point if virtually all documents had associated text. However, in some industries like oil & gas, half or more of some collections will be engineering drawings and schematics that were output to image-only PDF for distribution and use by those who don't have the software licenses needed to view the documents in their original file formats.
In practically all industries it is common practice to develop documents in one application like Word and then, once finalized, distribute them as image-only PDF so they can be viewed on a variety of devices and so recipients can't easily change the content. In one collection we analyzed, only 20% of the PDFs had associated text. Even if predictive coding were 100% effective, the most it could classify would be 20% because it literally cannot "see" the 80% without text. If in fact predictive coding has a recall rate of 70-80% of what it can see, that would mean that predictive coding would have identified 14 to 16% of the total PDFs (70% x 20% = 14% or 80% of 20% = 16%). By contrast, BR's visual classification technology classified 100% of them.
PDFs will potentially be among the most relevant file types in a collection because that is the format used to distribute information within and among groups of people within an organization, and among organizations. Note that even if in some unique e-discovery settings predictive coding is acceptable, the text-restriction failing of predictive coding will be fatal for broader information governance purposes.
So... if you're going to use predictive coding, at the very least measure what PC doesn't "see." If you're planning on using PC for information governance purposes, make sure that the organization doesn't mind not classifying a potentially significant percentage of its documents.
Most of us have heard about the parable of the six blind men and the elephant - it may actually be the first recorded instance of faceted classification. Six blind men touched different parts of an elephant and each described a completely different thing based on their own perspective or “view” of the elephant: The one who felt a tusk reported it as a pipe, the one who felt an ear reported a fan, the belly was reported as a wall, the trunk as a branch, a leg was reported as a pillar and the tail was described as being a rope.
Different Stakeholders Have Different Views & Needs. People’s views of and information needs from any given corpus of documents will vary according to where they are in an organization and the functions they perform. People with different roles in a company will naturally be interested in different attributes of the documents in the corpus and may well use different descriptors when describing or trying to find some of them. While some document attributes are common to all stakeholders, others, namely those which enable an individual or group to perform their job function within an organization, are probably not.
Just preserving documents without having a systematic, dynamically updatable and holistic view created by assimilating other interrelated data points will result in an incomplete picture of a project or process. Without a holistic way of assembling and viewing all the extracted document attributes of interest to the various stakeholders, the overarching information governance needs of the organization will never be met. There will be incomplete, ambiguous, erroneous and superfluous data points.
Duplicated data sets. Without a holistic enterprise content plan, each stakeholder starts keeping their own copies of documents so they can extract the attributes they are interested in. The result is multiple copies of the same documents, multiple expenditures to extract the same attributes, and inconsistencies in ways that the same data is extracted and stored.
The highly experienced CA team chose to align with BR as it is the only technology in the world that automatically classifies electronic files or scanned paper documents based on their visual characteristics – and without having to waste time writing rules to identify each type of document or designating exemplars for each document type. This is tremendously important because accurate, consistent classification is the bedrock upon which all IG initiatives are built. BR solves this long-standing, previously intractable problem.
