This is part two of a series of blog articles about ethics and AI.Through a rigorous, human-driven research and development process, Blue J Legal is able to use the power of machine learning in order to synthesize the multitude of factors present in every case to make accurate predictions. Data scientists and experienced lawyers are involved at each step of the multi-faceted and laborious process, wherein we translate the universe of unstructured data, such as relevant cases and tribunal decisions, into structured data that is capable of being used in machine learning.This way of making accurate predictions about the law is part of a promise we make to bring absolute clarity to the law, everywhere and on demand. We are able to do this not solely because of our algorithms, but because of all of our people who have committed themselves to this same promise.The lifecycle of our system can be broken down into the following components:
The beginning of this process focuses on selecting legal questions that deal with areas of considerable legal uncertainty, as the questions that stem from these areas are especially difficult for lawyers when predicting the outcome of a case due to the interaction of multiple factors and the absence of bright line rules. Determining how these factors will interact in areas of legal uncertainty, as well as which factors will have more or less influence in any given case, is the problem that our legal research team sets out to solve using machine learning analysis.Experienced legal and accounting professionals often instinctively know the likelihood of any particular outcome for a case - however, in order to properly counsel a client, they must anchor their advice in legal precedent. Independently researching this precedent is indubitably time-consuming, and the interplay of multiple factors present in every case makes it all the more difficult to predict outcomes. Our system can help expedite the process of researching the existing case law by surfacing cases with similar factors, as well as supplying experienced professionals with the certainty that no factor goes unaccounted for by alerting them to factors that they may not have initially considered. Junior counsel can also benefit from our predictive analysis, as it can aid in the development of crucial legal instincts through illustrating which factors are the most significant in the pertinent area of law.
Once we have selected a legal question, our legal research team begins an extensive research process in order to identify everything potentially relevant to understanding that area of law. This involves reading all of the leading cases in the area of law and taking note of all the relevant factors in the court’s analysis. Following this preliminary research, the research lawyer creates an extensive questionnaire designed to extract all of the information relevant to the identified factors from the cases.
Every case discussing the relevant legal question is then reviewed by a member of our in-house research team in order to translate the relevant cases into structured data with the aim of including all relevant cases on the legal topic. These in-house data analysts consequently determine whether the case applies to the particular legal question being studied. This involves an understanding of the most common factors endemic to the particular area of law, as well as an interpretation of the factual information present in each case in order to accurately apply it to the questionnaire. The questionnaire may be refined in response to information gleaned from the translation of cases.Throughout the translation process, we continually update our questionnaire to ensure that we are capturing the highest quality of data. In doing so, we employ a rigorous quality assurance process to ensure consistency and alignment on the interpretation of particular elements of the questionnaire, as well as to eliminate unintended outcomes.
Following the completion of the translation process, our data science team uses advanced statistical analysis to identify patterns within the data, enabling us to determine if our legal instincts align with the patterns illustrated in the data. Moreover, we employ out of sample testing in order to verify the accuracy of our predictions. By holding back a portion of the cases and training our algorithm on the remainder, we can test the accuracy of our predictions by analysing the frequency with which we are able to correctly predict the outcomes of those cases that were not used to train the model.
To test and refine the data is an iterative process that involves both our data scientists and our research lawyers. We combine statistical analysis with our understanding of the law in order to reflect patterns in the data that inform legal decisions. To do this, we test multiple approaches in order to arrive at an algorithm that predicts accurately and reflects the state of the law. Once we can predict the outcomes of cases with 90% accuracy, we prepare our product for live release.
Once released, our customers -- consisting of lawyers, accountants, and human resources professionals -- use the modules to make predictions about the legal issues they are facing, as well as to find previously decided cases that closely match their specified fact scenarios or outcomes. Generally, users disagree with Blue J’s predicted outcome less than 2% of the time, but whenever they do, we quickly follow up with them to understand the reason for the feedback. This input often leads to further enhancements of the product modules.
The law is a living thing that constantly changes in response to new developments, such as legislative amendments or common law updates, thereby necessitating the testing of how our questionnaires perform under real-world, dynamic conditions. We input newly released decisions into the system each week, consequently allowing us to remain up to date with the latest developments in the law. Moreover, these updates permit us to track our accuracy in predicting the outcomes of actual cases. Any case that we failed to predict the outcome of is analyzed in order to determine if it represents an outlier or if it illustrates how the law is moving in a new direction.For cases that may contain special circumstances not present in other cases, our outlier flag feature examines the facts collected in our database and identifies when our model predicts a result that is different from the court’s actual decision. This feature alerts the user to distinctions in the particular case that are unique to specific decisions, ensuring that the user has the largest amount of information at their disposal when approaching the specific legal question.
Sign up for the Blue J newsletter today.
Whether you have questions or are interested in booking a demo, we would love to hear from you.
