In 2023, it was a near certainty that if artificial intelligence (AI) came up in a discussion between lawyers, someone was going to say something like, “Did you hear about those lawyers in New York that filed that document full of fake cases they got from ChatGPT? It’s wild. I’d never do anything like that. How could anyone be so irresponsible?”
We’ve all heard about what happened: in the summer of 2023, two New York lawyers filed a legal brief in federal court that had citations to non-existent court opinions and included fake quotes. These were all generated by ChatGPT, a Generative AI (GenAI) platform, the lawyers used to write the document. Even though the lawyers did not know the citations and quotes were fake when they filed the document, the federal court imposed monetary sanctions on the attorneys and their firm. The court even ordered them to notify each judge falsely identified as the author of the bogus case law.
There’s also the similar story of the Colorado lawyer. The Colorado Supreme Court suspended him for 366 days, with 90 days to be served and the remainder to be stayed upon the successful completion of a two-year probation period. The lawyer filed a motion in May 2023 that cited case law from ChatGPT. He did not read the cases or otherwise try to verify that the citations were accurate. Before the motion was heard, the lawyer discovered the cases were incorrect or fake, but he did not alert the court or withdraw his motion. Instead, when the judge questioned the citations, he falsely attributed the mistakes to a legal intern and then, almost a week later, admitted that he used ChatGPT to draft the motion.
But that was 2023, when GenAI was new. This is 2024. We’re all smarter now. Everyone knows to be careful about using GenAI for legal work, right? Well, kind of.
The Latest GenAI Mishap
A few weeks ago, GenAI created problems for another New York lawyer, who was ordered to show cause why he should not be sanctioned for filing a letter brief that cited three non-existent cases. As many expected, GenAI was the reason this happened. The unfortunate attorney was the pro bono counsel for Michael Cohen, the onetime “fixer” for former President Trump.
Based on the declarations and public filings that have been made since, it appears that Pro Bono Counsel drafted the brief and sent it to his client, Cohen. Cohen and Pro Bono Counsel then exchanged multiple edited drafts of the brief, to which Cohen added the three fake citations. Pro Bono Counsel thought that these additions had been made by another one of Cohen’s attorneys. He, therefore, did not review them independently. Pro Bono Counsel then filed the letter brief that, unbeknownst to him or Cohen at the time, included the three citations to non-existent cases.
So where did those fake citations come from? From Cohen’s online research on Google Bard. Cohen thought that Bard was a supercharged search engine, not a GenAI service. As a result, he sent the bogus cases to Pro Bono Counsel, unaware they were fake; Pro Bono Counsel added the bogus cases to the brief, thinking they came from Cohen’s other attorney; and neither of the lawyers verified the bogus cases, resulting in them being included in the filed brief.
Lessons Learned
In many ways, the takeaways here are more nuanced and not specific to GenAI. The lesson for many lawyers in 2023 was to be careful about ChatGPT because GenAI can be “bad” or “untrustworthy.” By now, lawyers should know that ChatGPT isn’t the only GenAI tool available. They should be wary of other online resources that may be a GenAI platform, rather than a super-powered search engine.
It also isn’t enough for a lawyer to be careful about using GenAI or avoid using it entirely. You may end up in an embarrassing situation if you’re relying on clerks, associates, co-counsel, or clients—even ones who were lawyers before—that use GenAI, whether intentionally or inadvertently, without telling you. The need for lawyers to comply with their ethical duties of competence and diligence, particularly when supervising and working with other lawyers and nonlawyers, has never been greater.
For more information on artificial intelligence (AI), please contact Jason Kelly, Esq., CIPP/US at jason@annagueymccann.com.
