In early March we announced CoCounsel, our groundbreaking AI legal assistant built on OpenAI’s latest, most advanced large language model. Now we’re excited to share the name of that model: GPT-4, released today by OpenAI. And a forthcoming study will confirm GPT-4 has passed both the multiple-choice and written portions of the Uniform Bar Exam (UBE), making it the first large language model (LLM) to do so.
Pablo Arredondo, our co-founder, Chief Innovation Officer, and a fellow at Stanford’s Center for Legal Informatics (Stanford CodeX) collaborated with OpenAI and Stanford CodeX fellows Daniel Katz and Michael Bommarito to study GPT-4’s performance on the UBE.
In earlier work, Katz and Bommarito found that a large language model released in late 2022 was unable to pass the multiple-choice portion of the UBE. GPT-4, however, passed the multiple-choice portion and both components of the written portion, exceeding not only all prior large language models’ scores, but also the average score of real-life bar exam test takers.
Passing both the multiple-choice and written portions of the UBE with a score higher than the average real test-taker’s is powerful validation. It proves GPT-4 can do more than just generate text, as ChatGPT is known for—it can use advanced reasoning to interpret text. Bottom line: while GPT-4 alone isn’t sufficient for professional use by lawyers, it is the first large language model “smart enough” to power professional-grade solutions that are—like CoCounsel.
While prior iterations of GPT models, most notably ChatGPT, have garnered deserved attention and interest—and generated output both astonishing and bizarre—it’s important to remember what OpenAI CEO Sam Altman said on Twitter: “ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. It’s a mistake to be relying on it for anything important right now.”
GPT-4, however, has provided the foundation necessary for creating CoCounsel. Our team has spent the last several months applying our deep legal practice and data security expertise to GPT-4, making possible a first-of-its-kind solution reliable and secure enough for legal professionals to use in their practice. We’ve spent thousands of hours tailoring, training, and testing CoCounsel’s output to ensure lawyers—and their clients—can trust it.
“GPT-4 leaps past the power of earlier language models,” said our Chief Innovation Officer Pablo Arredondo. “The model’s ability not just to generate text, but to interpret it, heralds nothing short of a new age in the practice of law.”
Casetext Chief Technology Officer Dr. Ryan Walker added, “CoCounsel combines the power of next-generation AI with the security and data privacy law firms require. Client data is never used to train the models, and law firms retain complete control over their data. CoCounsel is the most secure AI in legal technology.”
There’s been an outpouring of enthusiasm and positive feedback for CoCounsel from renowned providers of legal services across sectors—from top-tier global law firms to in-house counsel at Fortune 50 companies to high-impact nonprofits. Many are beta customers who have been using CoCounsel for months.
The difference between their reactions and a lot of the AI-for-the-law hype in recent years? These customers and professionals are talking about actual benefits realized by using CoCounsel, not just what they hope it will have the potential to do.
Global law firm and Casetext client DLA Piper received early access to CoCounsel and has been using the AI assistant since September 2022. “Casetext’s CoCounsel is changing how the law is practiced by automating critical, time-intensive tasks and freeing our lawyers to focus on the most impactful aspects of practice,” said Frank Ryan, DLA Piper’s Americas Chair. “AI development and adoption is crucial as companies look to grow their businesses and compete successfully. No firm wants to be the last one to implement this game-changing technology.”
Darth Vaughn, Litigation Counsel and Legal Innovation & Technology Operations Lead at Ford Motor Company, said that CoCounsel “has the potential to revolutionize the way lawyers practice. The value of CoCounsel is quickly becoming evident across a range of our practice groups. If implemented correctly, I believe this technology can also increase access to justice.”
“In our work what we see is that lack of resources is the problem; folks with resources are less likely to be wrongfully convicted,” said Michael Semanchik, Managing Attorney of the California Innocence Project. “Providing access to CoCounsel certainly helps us to level that playing field on the back end. Access to CoCounsel on the front end could help defense attorneys and public defenders level the playing field as well.”
