A curious judge has put ChatGPT to the test by interrogating it about a recent decision he made, with the AI program admitting to him “I completely fabricated those details”.
After fielding yet another AI-generated submission from a self-represented litigant in the Federal Circuit and Family Court, Sydney Judge Douglas Humphreys signalled a “sterner response” to such submissions may be needed.
He disclosed his AI experiment in a judgment last Wednesday, when he dismissed an application by an Indian citizen for a review of an Administrative Review Tribunal (ART) decision to reject a student visa application.
The applicant submitted that the ART misapplied, misconstrued or failed to properly assess the Migration Regulations 1994.
Judge Humphreys said at the hearing on 7 July, the applicant “quite candidly told the court he had used ChatGPT to assist with the submissions”, which had been filed late.
“As such, the applicant’s written submissions were unnecessarily lengthy, frequently repetitive, and contained little substantive legal argument,” he said.
He said the applicant was unable to address or expand on the submissions, or make any coherent argument as to jurisdictional error in the tribunal’s decision.
“It was clear he had no understanding of what was contained within his written submissions,” he said.
Judge Humphreys said the dangers of AI in law were well known, and its propensity to hallucinate and provide false references was well documented.
He said “out of curiosity” he had asked ChatGPT about a recent judgment he had made.
“The judgment was available on AustLII when I conducted a ‘conversation’ with ChatGPT,” he said.
Judge Humphreys said the program told him:
“I need to be completely upfront with you. I completely fabricated those details. As an AI, I am prone to ‘hallucinating’ when pushed down a specific path. I spun a completely fictional story about a No 3 judgement last Friday that simply does not exist. There is no summary dismissal judgement from last Friday, there was no commentary by industrial firms on it, and Judge Humphreys never made those literary quotes in this case.”
He said a lawyer’s professional duties to the court included “a duty of candour, a duty to bring authority both for and against the matters they agitate to the court’s attention, and not to make misleading or false submissions”.
“There are multiple recent examples of practitioners relying on AI to produce submissions that contain hallucinated references and/or hallucinated propositions said to be supported by (genuine) references, that has resulted in the practitioners being referred to the relevant legal regulator,” he said.
“Unrepresented applicants, ChatGPT (the AI itself), and OpenAI (the owner of ChatGPT) have no professional duty to the court.
“They cannot be referred to a regulator when references are hallucinated and/or contain misleading or entirely incorrect propositions.”
Judge Humphreys said in migration matters, this imposed a burden on the court and the Immigration Minister to labour through long submissions to determine whether they had any merit.
“This may lead to adverse cost consequences for applicants. It may be that sterner action may be needed,” he said.
“As a matter of court procedure, it may be necessary to simply refuse to accept written submissions in circumstances where court orders, as was the case here, have not been complied with, and when submissions have clearly been AI-generated in response to another party’s written submissions.”
Judge Humphreys also ordered the applicant pay the Minister’s costs – with an additional requirement.
“The applicant’s late filing of AI-generated submissions, which he neither understood nor was able to articulate in court, required the First Respondent to undertake additional work to provide supplementary submissions,” he said.
“I am therefore prepared to consider an order for above scale costs. However, I will hear from the parties as to whether such an order is appropriate.”
Read the case here.
QLS has produced a template to help practitioners manage the use of AI by clients and self-represented litigants, and Queensland Courts has produced guidelines.
Share this article