Skip to main content
SpringerLink
Log in

LK-IB: a hybrid framework with legal knowledge injection for compulsory measure prediction

  • Original Research
  • Published:
Artificial Intelligence and Law Aims and scope Submit manuscript

Abstract

The interpretability of AI is just as important as its performance. In the LegalAI field, there have been efforts to enhance the interpretability of models, but a trade-off between interpretability and prediction accuracy remains inevitable. In this paper, we introduce a novel framework called LK-IB for compulsory measure prediction (CMP), one of the critical tasks in LegalAI. LK-IB leverages Legal Knowledge and combines an Interpretable model and a Black-box model to balance interpretability and prediction performance. Specifically, LK-IB involves three steps: (1) inputting cases into the first module, where first-order logic (FOL) rules are used to make predictions and output them directly if possible; (2) sending cases to the second module if FOL rules are not applicable, where a case distributor categorizes them as either “simple” or “complex“; and (3) sending simple cases to an interpretable model with strong interpretability and complex cases to a black-box model with outstanding performance. Experimental results demonstrate that the LK-IB framework provides more interpretable and accurate predictions than other state-of-the-art models. Given that the majority of cases in LegalAI are simple, the idea of model combination has significant potential for practical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://www.12309.gov.cn.

References

  • Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (xai). IEEE Access 6:52138–52160

    Article  Google Scholar 

  • Adler P, Falk C, Friedler SA, Rybeck G, Scheidegger C, Smith B, Venkatasubramanian S (2016) Auditing black-box models by obscuring features. arXiv preprint arXiv:1602.07043

  • Arrieta AB, Díaz-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, García S, Gil-López S, Molina D, Benjamins R et al (2020) Explainable artificial intelligence (xai): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fus 58:82–115

    Article  Google Scholar 

  • Bao Q, Zan H, Gong P, Chen J, Xiao Y (2019) Charge prediction with legal attention. In: CCF International Conference on Natural Language Processing and Chinese Computing, pp. 447–458. Springer

  • Bi S, Zhou Z, Pan L, Qi G (2022) Judicial knowledge-enhanced magnitude-aware reasoning for numerical legal judgment prediction. Artif Intell Law 1–34

  • Bolukbasi T, Chang K-W, Zou JY, Saligrama V, Kalai AT (2016) Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Adv Neural Inf Process Syst 29

  • Branting LK, Pfeifer C, Brown B, Ferro L, Aberdeen J, Weiss B, Pfaff M, Liao B (2021) Scalable and explainable legal prediction. Artif Intell Law 29(2):213–238

    Article  Google Scholar 

  • Brennan T, Dieterich W, Ehret B (2009) Evaluating the predictive validity of the compas risk and needs assessment system. Crim Justice Behav 36(1):21–40

    Article  Google Scholar 

  • Chalkidis I, Androutsopoulos I, Aletras N (2019) Neural legal judgment prediction in english. arXiv preprint arXiv:1906.02059

  • Chugh N (2021) Risk assessment tools on trial: Lessons learned for “ethical ai” in the criminal justice system. In: 2021 IEEE International Symposium on Technology and Society (ISTAS), pp. 1–5. https://doi.org/10.1109/ISTAS52410.2021.9629143

  • Cohen TH, Lowenkamp C (2018) Revalidation of the federal pretrial risk assessment instrument (ptra): Testing the ptra for predictive biases. Available at SSRN

  • Desmarais SL, Zottola SA, Duhart Clarke SE, Lowder EM (2021) Predictive validity of pretrial risk assessments: a systematic review of the literature. Crim Justice Behav 48(4):398–420

    Article  Google Scholar 

  • Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  • Dieterich W (2010) Kent County Pretrial Services outcomes study: developing and testing the COMPAS pretrial release risk scale. Northpointe

  • Dionne G (2013) Risk management: history, definition, and critique. Risk Manag Insur Rev 16(2):147–166

    Article  MathSciNet  Google Scholar 

  • Do P-K, Nguyen H-T, Tran C-X, Nguyen M-T, Nguyen M-L (2017) Legal question answering using ranking svm and deep convolutional neural network. arXiv preprint arXiv:1703.05320

  • Du M, Liu N, Hu X (2019) Techniques for interpretable machine learning. Commun ACM 63(1):68–77

    Article  Google Scholar 

  • Fawei B, Pan JZ, Kollingbaum M, Wyner AZ (2019) A semi-automated ontology construction for legal question answering. New Gener Comput 37(4):453–478

    Article  Google Scholar 

  • Gan L, Kuang K, Yang Y, Wu F (2021) Judgment prediction via injecting legal knowledge into neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 12866–12874

  • Hacker P, Krestel R, Grundmann S, Naumann F (2020) Explainable AI under contract and tort law: legal incentives and technical challenges. Artif Intell Law 28(4):415–439

    Article  Google Scholar 

  • Jiang X, Ye H, Luo Z, Chao W, Ma W (2018) Interpretable rationale augmented charge prediction system. In: Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations, pp. 146–151

  • Johnson R, Zhang T (2017) Deep pyramid convolutional neural networks for text categorization. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 562–570. Association for Computational Linguistics, Vancouver, Canada . https://doi.org/10.18653/v1/P17-1052. https://aclanthology.org/P17-1052

  • Joulin A, Grave E, Bojanowski P, Mikolov T (2017) Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 427–431. Association for Computational Linguistics, Valencia, Spain. https://aclanthology.org/E17-2068

  • Kim M-Y, Goebel R (2017) Two-step cascaded textual entailment for legal bar exam question answering. In: Proceedings of the 16th Edition of the International Conference on Articial Intelligence and Law, pp. 283–290

  • Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  • Kuang K, Li L, Geng Z, Xu L, Zhang K, Liao B, Huang H, Ding P, Miao W, Jiang Z (2020) Causal inference. Engineering 6(3):253–263

    Article  Google Scholar 

  • Li Q, Zhang Q (2021) Court opinion generation from case fact description with legal basis. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 14840–14848

  • Liu Y-H, Chen Y-L (2018) A two-phase sentiment analysis approach for judgement prediction. J Inf Sci 44(5):594–607

    Article  Google Scholar 

  • Liu L, Zhang W, Liu J, Shi W, Huang Y (2021) Interpretable charge prediction for legal cases based on interdependent legal information. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE

  • Long S, Tu C, Liu Z, Sun M (2019) Automatic judgment prediction via legal reading comprehension. In: China National Conference on Chinese Computational Linguistics, pp. 558–572 . Springer

  • Luo B, Feng Y, Xu J, Zhang X, Zhao D (2017) Learning to predict charges for criminal cases with legal basis. arXiv preprint arXiv:1707.09168

  • Madras D, Pitassi T, Zemel R (2018) Predict responsibly: improving fairness and accuracy by learning to defer. Adv Neural Inf Process Syst 31

  • Madsen A, Reddy S, Chandar S (2021) Post-hoc interpretability for neural nlp: a survey. ACM Comput Surv (CSUR)

  • Miron M, Tolan S, Gómez E, Castillo C (2021) Evaluating causes of algorithmic bias in juvenile criminal recidivism. Artif Intell Law 29(2):111–147

    Article  Google Scholar 

  • Mn H, Basheer I (2003) Comparison of logistic regression and neural network-based classifiers for bacterial growth. Food Microbiol 20:43–55. https://doi.org/10.1016/S0740-0020(02)00104-1

    Article  Google Scholar 

  • Peeters R, Schuilenburg M (2018) Machine justice: Governing security through the bureaucracy of algorithms. Inf Polity 23(3):267–280

    Article  Google Scholar 

  • Peterson LE (2009) K-nearest neighbor. Scholarpedia 4(2):1883

    Article  Google Scholar 

  • Rish I et al. (2001) An empirical study of the naive bayes classifier. In: IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, vol. 3, pp. 41–46

  • Safavian SR, Landgrebe D (1991) A survey of decision tree classifier methodology. IEEE Trans Syst Man Cybern 21(3):660–674

    Article  MathSciNet  Google Scholar 

  • Salton G, Buckley C (1988) Term-weighting approaches in automatic text retrieval. Inf Proces Manag 24(5):513–523. https://doi.org/10.1016/0306-4573(88)90021-0

    Article  Google Scholar 

  • Singh A, Mohapatra S (2021) Development of risk assessment framework for first time offenders using ensemble learning. IEEE Access 9:135024–135033

    Article  Google Scholar 

  • Vuong YT-H, Bui Q.M, Nguyen H-T, Nguyen T-T-T, Tran V, Phan X-H, Satoh K, Nguyen L-M (2022) Sm-bert-cr: a deep learning approach for case law retrieval with supporting model. Artif Intell Law, 1–28

  • Wang T, Lin Q (2021) Hybrid predictive models: when an interpretable model collaborates with a black-box model. J Mach Learn Res 22:137

    MathSciNet  MATH  Google Scholar 

  • Wu Y, Kuang K, Zhang Y, Liu X, Sun C, Xiao J, Zhuang Y, Si L, Wu F (2020) De-biased court’s view generation with causality. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 763–780

  • Xu Z, Li X, Li Y, Wang Z, Fanxu Y, Lai X (2020) Multi-task legal judgement prediction combining a subtask of the seriousness of charges. In: China National Conference on Chinese Computational Linguistics, pp. 415–429 . Springer

  • Xu N, Wang P, Chen L, Pan L, Wang X, Zhao J (2020) Distinguish confusing law articles for legal judgment prediction. arXiv preprint arXiv:2004.02557

  • Yang H, Deng W, Wang G, Wang F, Li S (2020) Interpretable legal judgment prediction based on improved conditional classification tree. In: Developments of Artificial Intelligence Technologies in Computation and Robotics: Proceedings of the 14th International FLINS Conference (FLINS 2020), pp. 336–343. World Scientific

  • Ye H, Jiang X, Luo Z, Chao W (2018) Interpretable charge predictions for criminal cases: Learning to generate court views from fact descriptions. arXiv preprint arXiv:1802.08504

  • Zhong H, Xiao C, Tu C, Zhang T, Liu Z, Sun M (2020) How does nlp benefit legal system: a summary of legal artificial intelligence. arXiv preprint arXiv:2004.12158

  • Zhou S, Liu Y, Wu Y, Kuang K, Zheng C, Wu F (2022) Similar case based prison term prediction. In: Artificial Intelligence: Second CAAI International Conference, CICAI 2022, Beijing, China, August 27–28, 2022, Revised Selected Papers, Part III, pp. 284–297. Springer

  • Zhou P, Shi W, Tian J, Qi Z, Li B, Hao H, Xu B (2016) Attention-based bidirectional long short-term memory networks for relation classification. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (volume 2: Short Papers), pp. 207–212

Download references

Funding

The research leading to these results received funding from the National Key Research and Development Program of China No. 2021YFC3300300 and the National Social Science Foundation under Grant Agreement No.21CFX068.

Author information

Author notes

  1. Xiang Zhou and Qi Liu contributed equally and are co-first authors.

Authors and Affiliations

  1. School of Guanghua Law, Zhejiang University, Hangzhou, China

    Xiang Zhou, Qi Liu & Qiangchao Chen

  2. School of Computer Science and Technology, Zhejiang University, Hangzhou, China

    Yiquan Wu & Kun Kuang

Authors
  1. Xiang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yiquan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiangchao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kun Kuang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yiquan Wu or Kun Kuang.

Ethics declarations

Conflicts of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix A: Relevant Law Articles

Appendix A: Relevant Law Articles

Here we provide with some relevant law articles in Criminal Procedure Law of the People’s Republic of China, which the prosecutor must following when making a decision on compulsory measures.

  • Article 67 A people’s court, a people’s prosecutor, and a public security authority may grant bail to a suspect or defendant under any of the following circumstances: (1) the suspect or defendant may be sentenced to supervision without incarceration, limited incarceration, or an accessory penalty only; (2) the suspect or defendant may be sentenced to fixed-term imprisonment or a heavier penalty but will not cause danger to the society if granted bail; (3) the suspect or defendant suffers a serious illness, cannot take care of himself or herself or is a pregnant woman or a woman who is breastfeeding her own baby, and will not cause danger to the society if granted bail; or (4) The term of custody of the suspect or defendant has expired but the case has not been closed, and a bail is necessary. Bail shall be executed by a public security authority.

  • Article 72 The authority deciding on a bail shall decide the amount of a bond after fully considering the need to ensure normal legal proceedings, the danger of the person to be bailed to the society, the nature and circumstances of the case, the gravity of the possible punishment, the financial condition of the person to be bailed, and other factors.

  • Article 80 The arrest of a suspect or defendant must be subject to the approval of a people’s prosecutor or a decision of a people’s court and be executed by a public security authority.

  • Article 81 Where there is evidence to prove the facts of a crime and a suspect or defendant may be sentenced to imprisonment or a heavier punishment, if residential confinement is insufficient to prevent any of the following dangers to society, the suspect or defendant shall be arrested: (1) the suspect or defendant may commit a new crime; (2) there is an actual danger to national security, public security, or social order; (3) the suspect or defendant may destroy or forge evidence, interfere with the testimony of a witness, or make a false confession in collusion; (4) the suspect or defendant may retaliate against a victim, informant, or accuser; or (5) the suspect or defendant attempts to commit suicide or escape. In the process of approving or deciding an arrest, the nature and circumstances of the suspected crime, the admission of guilt, and the acceptance of punishment, among others, of a suspect or defendant shall be considered as factors of a possible danger to the society. Where there is evidence to prove the facts of a crime and a suspect or defendant may be sentenced to fixed-term imprisonment of 10 years or a heavier punishment or there is evidence to prove the facts of a crime and a suspect or defendant who once committed an intentional crime or has not been identified may be sentenced to imprisonment or a heavier punishment, the suspect or defendant shall be arrested. Where a suspect or defendant waiting for trial on bail or under residential confinement seriously violates the provisions on bail or residential confinement, the suspect or defendant may be arrested.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, X., Liu, Q., Wu, Y. et al. LK-IB: a hybrid framework with legal knowledge injection for compulsory measure prediction. Artif Intell Law (2023). https://doi.org/10.1007/s10506-023-09362-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10506-023-09362-x

Keywords

Search

Navigation