Abstract
This paper presents the first comprehensive empirical evaluation of Large Language Models’ (‘LLMs’) performance in Indian legal education. We compare six Artificial Intelligence (‘AI’) chatbots with law students at the National Law School of India University, Bengaluru, across four subjects: Contract Law, Corporate Law, Criminal Procedure, and Jurisprudence. Our findings show that LLMs achieve performance comparable to human students (B+ grade), with newer commercial models consistently outperforming older and open-source alternatives. We also find that while LLMs excel in theoretical subjects and structured legal analysis, they show limitations in handling jurisdiction-specific knowledge and complex scenario-based reasoning. These findings have important implications for legal education in diverse jurisdictions and highlight the need for adaptive, pedagogical approaches in an AI-augmented legal landscape.
Digital Object Identifier (DOI)
10.55496/RZKH7712
Recommended Citation (provided in OSCOLA format)
Rahul Hemrajani, Vedant Gupta, R Srivatsan, Radhika Singhal, Krishne Tanneerbavi, Shristy Chhaparia, Siddharth Johar, Srujan Sangai, Suvanssh Mahajan, Priyansh Dixit, V Sreedharan, Dhruv Holla, Gunjan Modi, Kajal Jamdare, Pratyay Amrit, Sannah Mudbidri, Ishaan Goel, Akshit Singla, Arjun Mehta, Madhav Mitruka, Mannat Mahaey, Nathaniel Warjri, Swapnil Das, Ojas Chandaniha, Yash Ahirwar, Animesh Tiwari, and Aman Meena,
"ChatGPT Goes to (National)Law School"
(2024)
20(2)
Indian Journal of Law and Technology
10.55496/RZKH7712
Available at:
https://repository.nls.ac.in/ijlt/vol20/iss2/3
Included in
Common Law Commons, Criminal Procedure Commons, Education Law Commons, Jurisprudence Commons, Law and Economics Commons, Legal Writing and Research Commons
