Artificial intelligence has entered legal practice quietly. Lawyers use it to locate authorities, summarise pleadings, compare statutes and prepare first drafts. Courts, too, are examining how AI can assist research and case management. The difficult question is no longer whether AI will enter the justice system. It already has. The question is what happens when the machine gets the law wrong.
The issue became concrete in India in 2026 when the Supreme Court confronted judgments built, in part, on cases that never existed.
1. When Artificial Intelligence Invents the Law?
In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., the Supreme Court set aside decisions of the NCLT and NCLAT after finding that they relied upon six AI-generated citations that were either non-existent or contained fabricated passages. The Court treated the problem as one going to the integrity of adjudication itself and emphasised that AI may assist adjudication, but human control over judicial decision-making must remain absolute.
India had already seen an earlier warning. In Sri Subhash Chakraborty v. Smt. Sandhya Deb, the Tripura High Court recorded that counsel had used ChatGPT to obtain an answer concerning damages in an injunction suit. The case did not concern an AI hallucination, but it demonstrated something important: AI had already begun entering the courtroom as a research tool.
The problem becomes sharper when AI-generated material is treated as authority rather than assistance.
The United States encountered precisely this difficulty in Mata v. Avianca, Inc.. Lawyers filed a brief containing fictitious authorities generated by ChatGPT. The court imposed a $5,000 sanction and made clear that using technology for research was not itself improper, but lawyers remained responsible for what they placed before the court.
Similarly, Park v. Kim, became another warning about lawyers submitting erroneous authorities generated through AI tools. By 2026, American courts were dealing with a growing body of cases involving fabricated authorities, sanctions and professional responsibility.
The context is very simple but important: AI can generate legal text without possessing legal authority.
2. India Is Moving From Experiment to Regulation
The Indian debate is no longer confined to individual lawyers making mistakes. The institutional question is now before courts and regulators.
The Supreme Court’s 2026 discussion follows a broader judicial move towards controlled technological adoption. At the National Conference on Judicial Process Re-engineering and Digital Transformation, the Supreme Court’s eCommittee emphasised that AI and digital tools should operate as aids rather than substitutes for judicial reasoning.
Indian litigation is simultaneously testing AI in other areas.
In ANI Media Pvt. Ltd. v. OpenAI OpCo LLC, the Delhi High Court considered copyright questions arising from the use of copyrighted news material in training and operating large language models. The judgment examines whether AI training and AI-generated outputs can fall within the statutory framework of copyright law, including Section 52 of the Copyright Act, 1957.
In IndiaMART InterMESH Ltd. v. OpenAI Inc., the Delhi High Court confronted questions concerning ChatGPT’s legal character, its generated responses, intermediary liability and the inadequacy of older statutory categories for dealing with generative AI. The Court observed that the present statutory framework was drafted before modern generative AI existed.
The same pattern is visible in personality-rights litigation. In Yuvraj Singh v. Ashok Kumar & Ors., the Delhi High Court dealt with protection against unauthorised use of name, image, voice and likeness through AI, deepfakes and related technologies. Comparable questions have arisen in Varun Dhawan v. Artist Booking Company & Ors., Preity G. Zinta v. Google LLC & Ors., and Shashi Tharoor v. Ashok Kumar & Ors.. These cases show that AI is no longer an abstract policy problem. It is already producing concrete disputes over identity, reputation, copyright and control over digital likeness.
3. The International Courts Have Already Asked the Hard Questions
The international experience is useful because other jurisdictions encountered algorithmic decision-making before generative AI became widespread.
In State v. Loomis, 2016 WI 68, the Wisconsin Supreme Court considered the use of the COMPAS risk-assessment system during sentencing. The court permitted its use subject to limitations and cautions, including that the algorithmic assessment could not be the determinative basis for sentencing.
Canada went further in Ewert v. Canada, where the Supreme Court held that correctional authorities could not simply rely on assessment tools without taking reasonable steps to establish their accuracy for Indigenous offenders.
The principle has wider significance for AI: a technologically sophisticated decision is not necessarily a reliable decision.
The question of responsibility for AI-generated statements has also reached courts. In Moffatt v. Air Canada, a British Columbia tribunal held the airline responsible when its chatbot provided incorrect information to a passenger. The tribunal rejected the argument that the chatbot should be treated as a separate entity whose statements were somehow outside the airline’s responsibility.
Intellectual property has produced another line of cases. In Thaler v. Perlmutter, the U.S. Court of Appeals for the D.C. Circuit considered whether an autonomously generated AI work could receive copyright protection and affirmed the requirement of human authorship.
Australia initially took a different approach in Thaler v. Commissioner of Patents, where the Federal Court held that an AI system could be named as an inventor under the Australian Patents Act. That position was subsequently overturned on appeal, illustrating how quickly legal doctrine can change when technology exposes assumptions embedded in older legislation.
More recently, Thomson Reuters Enterprise Centre GmbH v. ROSS Intelligence Inc. where it was examined the use of copyrighted legal materials to develop an AI legal research product. The court rejected Ross’s fair-use defence at the summary-judgment stage.
The UK’s Getty Images v. Stability AI Ltd. further demonstrates the difficulty of applying traditional copyright concepts to AI model development. The High Court ultimately rejected Getty’s secondary copyright claim on the evidence before it, while the litigation also raised important questions concerning AI training, model weights and intellectual-property rights.
4. The Real Legal Problem Is Not AI. It Is Accountability
These cases point towards a common principle.
A judge remains responsible for a judgment even when technology assists the research. A lawyer remains responsible for a pleading even when software prepares the first draft. A corporation cannot necessarily escape responsibility for an inaccurate chatbot merely because the response was machine-generated.
That principle becomes particularly important in adjudication.
Judicial reasoning has a distinctive constitutional character. A judgment affects liberty, property, reputation and rights. The legitimacy of that decision depends upon reasons that can be examined, challenged and understood.
An AI system can identify patterns at remarkable speed. It can compare thousands of documents. It can summarise a long record in seconds. But it can also produce a perfectly formatted citation to a judgment that never existed.
That is why verification cannot be delegated.
The human-in-the-loop principle should therefore mean more than having a human press the final button. The person using AI must understand what the system has produced, verify the underlying authority and remain capable of explaining why the conclusion is legally sound.
5. From “Can AI Judge?” to “Who Answers When AI Is Wrong?”
The more useful legal question is therefore not whether AI can replace judges. That framing is too simple.
The harder question is who bears responsibility when an AI-assisted legal decision goes wrong?
The answer should begin with the allocation of responsibility. AI may assist with research, classification, transcription, document comparison and other administrative functions. But the legal responsibility for findings, reasons and conclusions must remain with the human decision-maker.
The Indian Supreme Court’s decision in Pooja Ramesh Singh gives this principle an unusually concrete foundation. A fictional judgment may look genuine on a screen, but it cannot become precedent merely because an algorithm has cited it.
The emerging jurisprudence across India, the United States, Canada, Australia and the United Kingdom suggests that courts are not rejecting technology. They are attempting to place responsibility around it.
That distinction will define the next stage of AI law.
The future courtroom may contain sophisticated AI systems. What it cannot contain is a legal authority that nobody has verified, a decision that nobody can explain, or responsibility that nobody accepts.
And perhaps that is the most important rule for the age of artificial intelligence: the machine may assist the law, but the responsibility for justice must remain human.
