Abstract
This paper presents the first comprehensive empirical evaluation of Large Language Models’ (‘LLMs’) performance in Indian legal education. We compare six Artificial Intelligence (‘AI’) chatbots with law students at the National Law School of India University, Bengaluru, across four subjects: Contract Law, Corporate Law, Criminal Procedure, and Jurisprudence. Our findings show that LLMs achieve performance comparable to human students (B+ grade), with newer commercial models consistently outperforming older and open-source alternatives. We also find that while LLMs excel in theoretical subjects and structured legal analysis, they show limitations in handling jurisdiction-specific knowledge and complex scenario-based reasoning. These findings have important implications for legal education in diverse jurisdictions and highlight the need for adaptive, pedagogical approaches in an AI-augmented legal landscape.
Recommended Citation
Hemrajani, Rahul; Gupta, Vedant; Srivatsan, R; Singhal, Radhika; Tanneerbavi, Krishne; Chhaparia, Shristy; Johar, Siddharth; Sangai, Srujan; Mahajan, Suvanssh; Dixit, Priyansh; Sreedharan, V; Holla, Dhruv; Modi, Gunjan; Jamdare, Kajal; Amrit, Pratyay; Mudbidri, Sannah; Goel, Ishaan; Singla, Akshit; Mehta, Arjun; Mitruka, Madhav; Mahaey, Mannat; Warjri, Nathaniel; Das, Swapnil; Chandaniha, Ojas; Ahirwar, Yash; Tiwari, Animesh; and Meena, Aman
(2024)
"ChatGPT Goes to (National)Law School,"
Indian Journal of Law and Technology: Vol. 20:
Iss.
2, Article 3.
DOI: 10.55496/RZKH7712
Available at:
https://repository.nls.ac.in/ijlt/vol20/iss2/3
Digital Object Identifier (DOI)
10.55496/RZKH7712
Included in
Common Law Commons, Criminal Procedure Commons, Education Law Commons, Jurisprudence Commons, Law and Economics Commons, Legal Writing and Research Commons
