Abstract
Case law retrieval is the task of locating truly relevant legal cases given an input query case. Unlike information retrieval for general texts, this task is more complex with two phases (legal case retrieval and legal case entailment) and much harder due to a number of reasons. First, both the query and candidate cases are long documents consisting of several paragraphs. This makes it difficult to model with representation learning that usually has restriction on input length. Second, the concept of relevancy in this domain is defined based on the legal relation that goes beyond the lexical or topical relevance. This is a real challenge because normal text matching will not work. Third, building a large and accurate legal case dataset requires a lot of effort and expertise. This is obviously an obstacle to creating enough data for training deep retrieval models. In this paper, we propose a novel approach called supporting model that can deal with both phases. The underlying idea is the case–case supporting relation and the paragraph–paragraph as well as the decision-paragraph matching strategy. In addition, we propose a method to automatically create a large weak-labeling dataset to overcome the lack of data. The experiments showed that our solution has achieved the state-of-the-art results for both case retrieval and case entailment phases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arora S, Liang Y, Ma T (2017) A simple but tough-to-beat baseline for sentence embeddings. In: 5th international conference on learning representations, ICLR 2017
Bench-Capon T, Araszkiewicz M, Ashley K, Atkinson K, Bex F, Borges F, Bourcier D, Bourgine P, Conrad JG, Francesconi E et al (2012) A history of AI and law in 50 papers: 25 years of the International Conference on AI and Law. Artif Intell Law 20(3):215–319
Berger A, Lafferty J (2017) Information retrieval as statistical translation, vol 51. ACM SIGIR Forum, ACM New York, pp 219–226
Bhattacharya P, Ghosh K, Ghosh S, Pal A, Mehta P, Bhattacharya A, Majumder P (2019) Overview of the fire 2019 aila track: artificial intelligence for legal assistance. In: FIRE (Working Notes), pp 1–12
Burges C, Shaked T, Renshaw E, Lazier A, Deeds M, Hamilton N, Hullender G (2005) Learning to rank using gradient descent. In: Proceedings of the 22nd international conference on machine learning, pp 89–96
Burges CJ, Ragno R, Le QV (2007) Learning to rank with nonsmooth cost functions. In: Advances in neural information processing systems, pp 193–200
Castells P, Fernandez M, Vallet D (2006) An adaptation of the vector-space model for ontology-based information retrieval. IEEE Trans Knowl Data Eng 19(2):261–272
Dai Z, Callan J (2019) Deeper text understanding for IR with contextual neural language modeling. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 985–988
Devlin J, Chang MW, Lee K, Toutanova K (2019a) BERT: Pre-training of deep bidirectional transformers for language understanding. In: NAACL, pp 4171–4186
Devlin J, Chang MW, Lee K, Toutanova K (2019b) Bert: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT
Gao J, Pantel P, Gamon M, He X, Deng L (2014) Modeling interestingness with deep neural networks. In: EMNLP
Guo J, Fan Y, Pang L, Yang L, Ai Q, Zamani H, Wu C, Croft WB, Cheng X (2020) A deep look into neural ranking models for information retrieval. Inf Process Manag 57(6):102067
Hu B, Lu Z, Li H, Chen Q (2014) Convolutional neural network architectures for matching natural language sentences. In: Advances in neural information processing systems, pp 2042–2050
Huang PS, He X, Gao J, Deng L, Acero A, Heck L (2013) Learning deep structured semantic models for web search using clickthrough data. In: Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pp 2333–2338
Kingma D, Ba J (2014) Adam: a method for stochastic optimization. In: International conference on learning representations
Liu TY (2009) Learning to rank for information retrieval. Found Trends Inf Retr 3:225–331
Mandal A, Chaki R, Saha S, Ghosh K, Pal A, Ghosh S (2017) Measuring similarity among legal court case documents. In: Proceedings of the 10th annual ACM India compute conference, pp 1–9
Marchesin S, Purpura A, Silvello G (2020) Focal elements of neural information retrieval models. An outlook through a reproducibility study. Inf Process Manag 57(6):102109
Mihalcea R, Tarau P (2004) Textrank: bringing order into text. In: Proceedings of the 2004 conference on empirical methods in natural language processing, pp 404–411
Mitra B, Diaz F, Craswell N (2017) Learning to match using local and distributed representations of text for web search. In: Proceedings of the 26th international conference on world wide web, pp 1291–1299
Oard DW, Webber W (2013) Information retrieval for e-discovery. Inf Retr 7(2–3):99–237
Palangi H, Deng L, Shen Y, Gao J, He X, Chen J, Song X, Ward R (2016) Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval. IEEE/ACM Trans Audio Speech Lang Process 24(4):694–707
Pang L, Lan Y, Guo J, Xu J, Wan S, Cheng X (2016) Text matching as image recognition. AAAI Press, AAAI’16
Rabelo J, Kim MY, Goebel R (2019a) Combining similarity and transformer methods for case law entailment. In: Proceedings of the seventeenth international conference on artificial intelligence and law, pp 290–296
Rabelo J, Kim MY, Goebel R (2019b) Combining similarity and transformer methods for case law entailment. In: Proceedings of the seventeenth international conference on artificial intelligence and law, association for computing machinery, New York, NY, USA, ICAIL ’19, pp 290–296
Robertson SE, Walker S (1994) Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In: SIGIR’94. Springer, pp 232–241
Salakhutdinov R, Hinton G (2009) Semantic hashing. Int J Approx Reason 50(7):969–978
Salton G, Buckley C (1988) Term-weighting approaches in automatic text retrieval. Inf Process Manag 24(5):513–523
Saracevic T (1996) Relevance reconsidered. In: Proceedings of the second conference on conceptions of library and information science (CoLIS 2), ACM New York, pp 201–218
Saravanan M, Ravindran B, Raman S (2009) Improving legal information retrieval using an ontological framework. Artif Intell Law 17(2):101–124
Shao Y, Mao J, Liu Y, Ma W, Satoh K, Zhang M, Ma S (2020) Bert-pli: modeling paragraph-level interactions for legal case retrieval. In: Bessiere C (ed) Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI-20, international joint conferences on artificial intelligence organization, pp 3501–3507
Shen Y, He X, Gao J, Deng L, Mesnil G (2014) A latent semantic model with convolutional-pooling structure for information retrieval. In: Proceedings of the 23rd ACM international conference on conference on information and knowledge management, pp 101–110
Song F, Croft WB (1999) A general language model for information retrieval. In: Proceedings of the eighth international conference on Information and knowledge management, pp 316–321
Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (Volume 1: Long Papers), Association for Computational Linguistics, Beijing, China, pp 1556–1566
Tran V, Le Nguyen M, Tojo S, Satoh K (2020) Encoded summarization: summarizing documents into continuous vector space for legal case retrieval. Artif Intell Law 1–27
Van Opijnen M, Santos C (2017) On the concept of relevance in legal information retrieval. Artif Intell Law 25(1):65–87
Wan S, Lan Y, Guo J, Xu J, Pang L, Cheng X (2016) A deep architecture for semantic matching with multiple positional sentence representations. In: Proceedings of the thirtieth AAAI conference on artificial intelligence. AAAI Press, AAAI’16, pp 2835–2841
Wu Q, Burges CJ, Svore KM, Gao J (2010) Adapting boosting for information retrieval measures. Inf Retr 13(3):254–270
Yilmaz ZA, Yang W, Zhang H, Lin J (2019) Cross-domain modeling of sentence-level evidence for document retrieval. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 3481–3487
Zeng Y, Wang R, Zeleznikow J, Kemp E (2005) Knowledge representation for the intelligent legal case retrieval. In: International conference on knowledge-based and intelligent information and engineering systems. Springer, pp 339–345
Zhai C, Lafferty J (2017) A study of smoothing methods for language models applied to ad hoc information retrieval, vol 51. ACM SIGIR Forum, ACM New York, NY, USA, pp 268–276
Acknowledgements
This work was supported by JSPS Kakenhi Grant nos. 20H04295, 20K20406, and 20K20625. The research also was supported in part by the Asian Office of Aerospace R &D (AOARD), Air Force Office of Scientific Research (Grant no. FA2386-19-1-4041).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1: Legal case retrieval on a large number of “noticed” case sample
See Table 13.
Appendix 2: Legal case retrieval on limited “noticed” cases sample
See Table 14.
Appendix 3: Sample in legal case entailment phase
See Table 15.
Rights and permissions
About this article
Cite this article
Vuong, Y.TH., Bui, Q.M., Nguyen, HT. et al. SM-BERT-CR: a deep learning approach for case law retrieval with supporting model. Artif Intell Law 31, 601–628 (2023). https://doi.org/10.1007/s10506-022-09319-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10506-022-09319-6