A variety of legal documents are increasingly being made available in electronic format. Automatic Information Search and Retrieval algorithms play a key role in enabling efficient access to such digitized documents. Although keyword-based search is the traditional method used for text retrieval, they perform poorly when literal term matching is done for query processing, due to synonymy and ambivalence of words. To overcome these drawbacks, an ontological framework to enhance the user’s query for retrieval of (...) truly relevant legal judgments has been proposed in this paper. Ontologies ensure efficient retrieval by enabling inferences based on domain knowledge, which is gathered during the construction of the knowledge base. Empirical results demonstrate that ontology-based searches generate significantly better results than traditional search methods. (shrink)

In this work we study, design, and evaluate computational methods to support interpretation of statutory terms. We propose a novel task of discovering sentences for argumentation about the meaning of statutory terms. The task models the analysis of past treatment of statutory terms, an exercise lawyers routinely perform using a combination of manual and computational approaches. We treat the discovery of sentences as a special case of ad hoc document retrieval. The specifics include retrieval of short texts, specialized (...) document types, and, above all, the unique definition of document relevance provided in detailed annotation guidelines. To support our experiments we assembled a data set comprising 42 queries which we plan to release to the public in the near future in order to support further research. Most importantly, we investigate the feasibility of developing a system that responds to a query with a list of sentences that mention the term in a way that is useful for understanding and elaborating its meaning. This is accomplished by a systematic assessment of different features that model the sentences’ usefulness for interpretation. We combine features into a compound measure that accounts for multiple aspects. The definition of the task, the assembly of the data set, and the detailed task analysis provide a solid foundation for employing a learning-to-rank approach. (shrink)

Recently, the pandemic caused by COVID-19 is severe in the entire world. The prevention and control of crimes associated with COVID-19 are critical for controlling the pandemic. Therefore, to provide efficient and convenient intelligent legal knowledge services during the pandemic, we develop an intelligent system for legal information retrieval on the WeChat platform in this paper. The data source we used for training our system is “The typical cases of national procuratorial authorities handling crimes against the (...) prevention and control of the new coronary pneumonia pandemic following the law”, which is published online by the Supreme People’s Procuratorate of the People’s Republic of China. We base our system on convolutional neural network and use the semantic matching mechanism to capture inter-sentence relationship information and make a prediction. Moreover, we introduce an auxiliary learning process to help the network better distinguish the relation between two sentences. Finally, the system uses the trained model to identify the information entered by a user and responds to the user with a reference case similar to the query case and gives the reference legal gist applicable to the query case. (shrink)

The concept of ‘relevance’ is crucial to legal information retrieval, but because of its intuitive understanding it goes undefined too easily and unexplored too often. We discuss a conceptual framework on relevance within legal information retrieval, based on a typology of relevance dimensions used within general information retrieval science, but tailored to the specific features of legal information. This framework can be used for the development and improvement of legal (...) information retrieval systems. (shrink)

Web legal information retrieval systems need the capability to reason with the knowledge modeled by legal ontologies. Using this knowledge it is possible to represent and to make inferences about the semantic content of legal documents. In this paper a methodology for applying NLP techniques to automatically create a legal ontology is proposed. The ontology is defined in the OWL semantic web language and it is used in a logic programming framework, EVOLP+ISCO, to allow (...) users to query the semantic content of the documents. ISCO allows an easy and efficient integration of declarative, object-oriented and constraint-based programming techniques with the capability to create connections with external databases. EVOLP is a dynamic logic programming framework allowing the definition of rules for actions and events. An application of the proposed methodology to the legal web information retrieval system of the Portuguese Attorney General’s Office is described. (shrink)

Legal text retrieval traditionally relies upon external knowledge sources such as thesauri and classification schemes, and an accurate indexing of the documents is often manually done. As a result not all legal documents can be effectively retrieved. However a number of current artificial intelligence techniques are promising for legal text retrieval. They sustain the acquisition of knowledge and the knowledge-rich processing of the content of document texts and information need, and of their matching. Currently, (...) techniques for learning information needs, learning concept attributes of texts, information extraction, text classification and clustering, and text summarization need to be studied in legal text retrieval because of their potential for improving retrieval and decreasing the cost of manual indexing. The resulting query and text representations are semantically much richer than a set of key terms. Their use allows for more refined retrieval models in which some reasoning can be applied. This paper gives an overview of the state of the art of these innovativetechniques and their potential for legal text retrieval. (shrink)

An application of Narrative Knowledge Representation Language (NKRL) techniques on (declassified) ‘terrorism in Southern Philippines’ documents has been carried out in the context of the IST Parmenides project. This paper describes some aspects of this work: it is our belief, in fact, that the Knowledge Representation techniques and the Intelligent Information Retrieval tools used in this experiment can be of some interest also in an ‘Ontological Modelling of Legal Events and Legal Reasoning’ context.

The effectiveness of information retrieval technology in electronic discovery (E-discovery) has become the subject of judicial rulings and practitioner controversy. The scale and nature of E-discovery tasks, however, has pushed traditional information retrieval evaluation approaches to their limits. This paper reviews the legal and operational context of E-discovery and the approaches to evaluating search technology that have evolved in the research community. It then describes a multi-year effort carried out as part of the Text (...) class='Hi'>Retrieval Conference to develop evaluation methods for responsive review tasks in E-discovery. This work has led to new approaches to measuring effectiveness in both batch and interactive frameworks, large data sets, and some surprising results for the recall and precision of Boolean and statistical information retrieval methods. The paper concludes by offering some thoughts about future research in both the legal and technical communities toward the goal of reliable, effective use of information retrieval in E-discovery. (shrink)

Legal text retrieval serves as a key component in a wide range of legal text processing tasks such as legal question answering, legal case entailment, and statute law retrieval. The performance of legal text retrieval depends, to a large extent, on the representation of text, both query and legal documents. Based on good representations, a legal text retrieval model can effectively match the query to its relevant documents. Because (...) class='Hi'>legal documents often contain long articles and only some parts are relevant to queries, it is quite a challenge for existing models to represent such documents. In this paper, we study the use of attentive neural network-based text representation for statute law document retrieval. We propose a general approach using deep neural networks with attention mechanisms. Based on it, we develop two hierarchical architectures with sparse attention to represent long sentences and articles, and we name them Attentive CNN and Paraformer. The methods are evaluated on datasets of different sizes and characteristics in English, Japanese, and Vietnamese. Experimental results show that: (i) Attentive neural methods substantially outperform non-neural methods in terms of retrieval performance across datasets and languages; (ii) Pretrained transformer-based models achieve better accuracy on small datasets at the cost of high computational complexity while lighter weight Attentive CNN achieves better accuracy on large datasets; and (iii) Our proposed Paraformer outperforms state-of-the-art methods on COLIEE dataset, achieving the highest recall and F2 scores in the top-N retrieval task. (shrink)

In this work, we provide a broad overview of the distinct stages of E-Discovery. We portray them as an interconnected, often complex workflow process, while relating them to the general Electronic Discovery Reference Model (EDRM). We start with the definition of E-Discovery. We then describe the very positive role that NIST’s Text REtrieval Conference (TREC) has added to the science of E-Discovery, in terms of the tasks involved and the evaluation of the legal discovery work performed. Given the (...) critical nature that data analysis plays at various stages of the process, we present a pyramid model, which complements the EDRM model: for gathering and hosting; indexing; searching and navigating; and finally consolidating and summarizing E-Discovery findings. Next we discuss where the current areas of need and areas of growth appear to be, using one of the field’s most authoritative surveys of providers and consumers of E-Discovery products and services. We subsequently address some areas of Artificial Intelligence, both Information Retrieval-related and not, which promise to make future contributions to the E-Discovery discipline. Some of these areas include data mining applied to e-mail and social networks, classification and machine learning, and the technologies that will enable next generation E-Discovery. The lesson we convey is that the more IR researchers and others understand the broader context of E-Discovery, including the stages that occur before and after primary search, the greater will be the prospects for broader solutions, creative optimizations and synergies yet to be tapped. (shrink)

This paper connects to Jon Bing’s great vision of an integrated national legal information system. The intention of this paper is to variegate Bing’s vision of an integrated information system by shifting the focus to the lay users, thus to those, who are subject to the law. The modified vision is an integrated information system that supports intelligible access to law for the citizens. This presupposes however an unambiguous and transparent legal system. Accordingly, it is (...) also stressed that intelligent legal knowledge engineering has high potential to increase precision and rationality in law, and hence safeguard the predictability of law and legal certainty. The central hypothesis is that putting the person who is subject to the law at the centre provides more rational reason to aim for the greatest possible precision of law by exploiting the means and technologies of our time than to uphold strategic vagueness. On this basis, the paper proposes the development of a basic set of controllable and justiciable basic legal drafting standards on EU and on national level, and promotes a transparency by design approach for legal knowledge engineering in order to enhance precision, integrity, comprehensibility and accessibility of law. Similarly to Jon Bing’s vision, this modified vision of an integrated legal information systems is, most probably, also likely to remain unfulfilled. However, what would science be without visions and (science) fictions? (shrink)

The ability to find relevant materials in large document collections is a fundamental component of legal research. The emergence of large machine-readable collections of legal materials has stimulated research aimed at improving the quality of the tools used to access these collections. Important research has been conducted within the traditional information retrieval, the artificial intelligence, and the legal communities with varying degrees of interaction between these groups. This article provides an introduction to text retrieval (...) and surveys the main research related to the retrieval of legal materials. (shrink)

The perspective of online dispute resolution is to develop an online electronic system aimed at solving out-of-court disputes. Among ODR schemes, eMediation is becoming an important tool for encouraging the positive settlement of an agreement among litigants. The main motivation underlying the adoption of eMediation is the time/cost reduction for the resolution of disputes compared to the ordinary justice system. In the context of eMediation, a fundamental requirement that an ODR system should meet relates to both litigants and mediators, i.e. (...) to enable an informed negotiation by informing the parties about the rights and duties related to the case. In order to match this requirement, we propose an information retrieval system able to retrieve relevant court decisions with respect to the disputant case description. The proposed system combines machine learning and natural language processing techniques to better match disputant case descriptions with court decisions. Experimental results confirm the ability of the proposed solution to empower court decision retrieval, enabling therefore a well-informed eMediation process. (shrink)

Legal Judgment Summarization (LJS) can highly summarize legal judgment documents, improving judicial work efficiency in case retrieval and other occasions. Legal judgment documents are usually lengthy; however, most existing LJS methods are directly based on general text summarization models, which cannot handle long texts effectively. Additionally, due to the complex structural characteristics of legal judgment documents, some information may be lost by applying only one single kind of summarization model. To address these issues, we (...) propose an integrated summarization method which leverages both semantic and structural information to improve the quality of LJS. Specifically, legal judgment documents are firstly segmented into three relatively short parts according to their specific structure. We propose an extractive summarization model named BSLT and an abstractive summarization model named LPGN by adopting Lawformer as the encoder. Lawformer is a new pre-trained language model for long legal documents, which specializes in capturing long-distance dependency and modeling legal semantic features. Then, we adopt different models to summarize the corresponding part regarding its structural characteristics. Finally, the obtained summaries are integrated to generate a high-quality summary involving semantic and structural information. We conduct comparative experiments to evaluate the performance of our model. The results show that our model outperforms the baseline model LEAD-3 by 14.78% on the mean ROUGE score, which demonstrates our method is effective in LJS and is prospected to be applied to assist other tasks in legal artificial intelligence. (shrink)

Legal information retrieval is in need of the provision of legal knowledge for the improvement of search strategies. For this purpose, the LOIS project is concerned with the construction of a multilingual WordNet for cross-lingual information retrieval in the legal domain. In this article, we set out how a hybrid approach, featuring lexically and legally grounded conceptual representations, can fit the cross-lingual information retrieval needs of both legal professionals and laymen.

Hypertext may represent a new paradigm capable of exploring legal sources within which links are established according to pertinent relationships found between statute texts and case law. However, to discover relevant information in such a network, a browsing mechanism is not enough when faced with a large volume of texts. This paper describes a new retrieval model where documents are represented according to both their content and relationships with other sources of information.

The BankXX system models the process of perusing and gathering information for argument as a heuristic best-first search for relevant cases, theories, and other domain-specific information. As BankXX searches its heterogeneous and highly interconnected network of domain knowledge, information is incrementally analyzed and amalgamated into a dozen desirable ingredients for argument (called argument pieces), such as citations to cases, applications of legal theories, and references to prototypical factual scenarios. At the conclusion of the search, BankXX outputs (...) the set of argument pieces filled with harvested material relevant to the input problem situation.This research explores the appropriateness of the search paradigm as a framework for harvesting and mining information needed to make legal arguments. In this article, we describe how legal research fits the heuristic search framework and detail how this model is used in BankXX. We describe the BankXX program with emphasis on its representation of legal knowledge and legal argument. We describe the heuristic search mechanism and evaluation functions that drive the program. We give an extended example of the processing of BankXX on the facts of an actual legal case in BankXX's application domain — the good faith question of Chapter 13 personal bankruptcy law. We discuss closely related research on legal knowledge representation and retrieval and the use of search for case retrieval or tasks related to argument creation. Finally we review what we believe are the contributions of this research to the understanding of the diverse disciplines it addresses. (shrink)

Juris-Data is one of the largest case-study base in France. The case studies are indexed by legal classification elaborated by the Juris-Data Group. Knowledge engineering was used to design an intelligent interface for information retrieval based on this classification. The aim of the system is to help users find the case-study which is the most relevant to their own.The approach is potentially very useful, but for standardising it for other legal document bases it is necessary to (...) extract a legal classification of the primary documents. Thus, a methodology for the construction of these classifications was designed together with a framework for index construction. The project led to the implementation of a Legal Case Studies Engineering Framework based on the accumulated experimentation and the methodologies designed. It consists of a set of computerised tools which support the life-cycle of the legal document from their processing by legal experts to their consultation by clients. (shrink)

Similar case analysis (SCA) is an essential topic in legal artificial intelligence, serving as a reference for legal professionals. Most existing works treat SCA as a traditional text classification task and ignore some important legal elements that affect the verdict and case similarity, like legal events, and thus are easily misled by semantic structure. To address this issue, we propose a Legal Event-Context Model named LECM to improve the accuracy and interpretability of SCA based on (...) Chinese legal corpus. The event-context integration mechanism, which is an essential component of the LECM, is proposed to integrate the legal event and context information based on the attention mechanism, enabling legal events to be associated with their corresponding relevant contexts. We introduce an event detection module to obtain the legal event information, which is pre-trained on a legal event detection dataset to avoid labeling events manually. We conduct extensive experiments on two SCA tasks, i.e., similar case matching (SCM) and similar case retrieval (SCR). Compared with baseline models, LECM is validated by about 13% and 11% average improvement in terms of mean average precision and accuracy respectively, for SCR and SCM tasks. These results indicate that LECM effectively utilizes event-context knowledge to enhance SCA performance and its potential application in various legal document analysis tasks. (shrink)

The number of available legal documents has presented an enormous growth in recent years, and the digital processing of such materials is prompting the necessity of systems that support the automatic relevant information extraction. This work presents a system for ontology-based information extraction from natural language texts, able to identify a set of legal events. The system is based on an innovative methodology based on domain ontology of legal events and a set of linguistic rules, (...) integrated through inference mechanism, resulting in a flexible approach and scalable approach. A case study with the use of documents from the Superior Court in Brazil is related, with satisfactory results in precision and recall. (shrink)

Access to legal information and, in particular, to legal literature is examined for the creation of a search and retrieval system for Italian legal literature. The design and implementation of services such as integrated access to a wide range of resources are described, with a particular focus on the importance of exploiting metadata assigned to disparate legal material. The integration of structured repositories and Web documents is the main purpose of the system: it is (...) constructed on the basis of a federation system with service provider functions, aiming at creating a centralized index of legal resources. The index is based on a uniform metadata view created for structured data by means of the OAI approach and for Web documents by a machine learning approach, which, in this paper, has been assessed as regards document classification. Semantic searching is a major requirement for legal literature users and a solution based on the exploitation of Dublin Core metadata, as well as the use of legal ontologies and related terms prepared for accessing indexed articles have been implemented. (shrink)

Sentence boundary detection (SBD) represents an important first step in natural language processing since accurately identifying sentence boundaries significantly impacts downstream applications. Nevertheless, detecting sentence boundaries within legal texts poses a unique and challenging problem due to their distinct structural and linguistic features. Our approach utilizes deep learning models to leverage delimiter and surrounding context information as input, enabling precise detection of sentence boundaries in English legal texts. We evaluate various deep learning models, including domain-specific transformer models (...) like LegalBERT and CaseLawBERT. To assess the efficacy of our deep learning models, we compare them with a state-of-the-art domain-specific statistical conditional random field (CRF) model. After considering model size, F1-score, and inference time, we identify the Convolutional Neural Network Model (CNN) as the top-performing deep learning model. To further enhance performance, we integrate the features of the CNN model into the subsequent CRF model, creating a hybrid architecture that combines the strengths of both models. Our experiments demonstrate that the hybrid model outperforms the baseline model, achieving a 4% improvement in the F1-score. Additional experiments showcase the superiority of the hybrid model over SBD open-source libraries when confronted with an out-of-domain test set. These findings underscore the importance of efficient SBD in legal texts and emphasize the advantages of employing deep learning models and hybrid architectures to achieve optimal performance. (shrink)

Legal Information Retrieval (IR) research has stressed the fact that legal knowledge systems should be sufficiently capable to interpret and handle the semantics of a database. Modeling (expert-) knowledge by using ontologies enhances the ability to extract and exploit information from documents. This contribution presents theories, ideas and notions regarding the development of dynamic electronic commentaries based on a comprehensive legal ontology.

There is a growing sense that many liberal states are in the midst of a shift in legal and political norms—a shift that is happening slowly and for a variety of reasons relating to security. The internet and tech booms—paving the way for new forms of electronic surveillance—predated the 9/11 attacks by several years, while the police’s vast use of secret informants and deceptive operations began well before that. On the other hand, the recent uptick in reactionary movements—movements in (...) which the rule of law seems expendable—began many years after 9/11 and continues to this day. One way to describe this book is an examination of the moral limits on modern police practices that flow from the basic legal and political tenets of the liberal tradition. The central argument is that policing in liberal states is constrained by a liberal conception of persons coupled with particular rule of law principles. Part I consists of three chapters that constitute the book’s theoretical foundation, including an overview of the police’s law enforcement role in the liberal polity and a methodology for evaluating that role. Part II consists of three chapters that address applications of the theory, including the police’s use of informants, deceptive operations, and surveillance. The upshot is that policing in liberal societies has become illiberal in light of its response to both internal and external threats to security. The book provides an account of what it might mean to retrieve policing that is consistent with the basic tenets of liberalism and the limits imposed by those tenets. [This is an uncorrected draft of the book's preface and introduction, forthcoming from Oxford University Press.]. (shrink)

This article provides an overview of, and thematic justification for, the special issue of the journal of Artificial Intelligence and Law entitled “E-Discovery”. In attempting to define a characteristic “AI & Law” approach to e-discovery, and since a central theme of AI & Law involves computationally modeling legal knowledge, reasoning and decision making, we focus on the theme of representing and reasoning with litigators’ theories or hypotheses about document relevance through a variety of techniques including machine learning. We also (...) identify two emerging techniques for enabling users’ document queries to better express the theories of relevance and connect them to documents: social network analysis and a hypothesis ontology. (shrink)

In the domain of legal information retrieval, an important challenge is to compute similarity between two legal documents. Precedents play an important role in The Common Law system, where lawyers need to frequently refer to relevant prior cases. Measuring document similarity is one of the most crucial aspects of any document retrieval system which decides the speed, scalability and accuracy of the system. Text-based and network-based methods for computing similarity among case reports have already been (...) proposed in prior works but not without a few pitfalls. Since legal citation networks are generally highly disconnected, network based metrics are not suited for them. Till date, only a few text-based and predominant embedding based methods have been employed, for instance, TF-IDF based approaches, Word2Vec and Doc2Vec based approaches. We investigate the performance of 56 different methodologies for computing textual similarity across court case statements when applied on a dataset of Indian Supreme Court Cases. Among the 56 different methods, thirty are adaptations of existing methods and twenty-six are our proposed methods. The methods studied include models such as BERT and Law2Vec. It is observed that the more traditional methods that rely on a bag-of-words representation performs better than the more advanced context-aware methods for computing document-level similarity. Finally we nominate, via empirical validation, five of our best performing methods as appropriate for measuring similarity between case reports. Among these five, two are adaptations of existing methods and the other three are our proposed methods. (shrink)

The paper provides an OWL ontology for legal cases with an instantiation of the legal case Popov v. Hayashi. The ontology makes explicit the conceptual knowledge of the legal case domain, supports reasoning about the domain, and can be used to annotate the text of cases, which in turn can be used to populate the ontology. A populated ontology is a case base which can be used for information retrieval, information extraction, and case based (...) reasoning. The ontology contains not only elements for indexing the case (e.g. the parties, jurisdiction, and date), but as well elements used to reason to a decision such as argument schemes and the components input to the schemes. We use the Protégé ontology editor and knowledge acquisition system, current guidelines for ontology development, and tools for visual and linguistic presentation of the ontology. (shrink)

Case law retrieval is the task of locating truly relevant legal cases given an input query case. Unlike information retrieval for general texts, this task is more complex with two phases (legal case retrieval and legal case entailment) and much harder due to a number of reasons. First, both the query and candidate cases are long documents consisting of several paragraphs. This makes it difficult to model with representation learning that usually has restriction (...) on input length. Second, the concept of relevancy in this domain is defined based on the legal relation that goes beyond the lexical or topical relevance. This is a real challenge because normal text matching will not work. Third, building a large and accurate legal case dataset requires a lot of effort and expertise. This is obviously an obstacle to creating enough data for training deep retrieval models. In this paper, we propose a novel approach called supporting model that can deal with both phases. The underlying idea is the case–case supporting relation and the paragraph–paragraph as well as the decision-paragraph matching strategy. In addition, we propose a method to automatically create a large weak-labeling dataset to overcome the lack of data. The experiments showed that our solution has achieved the state-of-the-art results for both case retrieval and case entailment phases. (shrink)

Access to legal information is fundamental to access to justice. Yet accessibility refers not only to making legal documents available to the public, but also rendering legal information comprehensible to them. A vexing problem in bringing legal information to the public is how to turn formal legal documents such as legislation and judgments, which are often highly technical, to easily navigable and comprehensible knowledge to those without legal education. In this study, (...) we formulate a three-step approach for bringing legal knowledge to laypersons, tackling the issues of navigability and comprehensibility. First, we translate selected sections of the law into snippets (called CLIC-pages), each being a small piece of article that focuses on explaining certain technical legal concept in layperson’s terms. Second, we construct a Legal Question Bank, which is a collection of legal questions whose answers can be found in the CLIC-pages. Third, we design an interactive CLIC Recommender. Given a user’s verbal description of a legal situation that requires a legal solution, CRec interprets the user’s input and shortlists questions from the question bank that are most likely relevant to the given legal situation and recommends their corresponding CLIC pages where relevant legal knowledge can be found. In this paper we focus on the technical aspects of creating an LQB. We show how large-scale pre-trained language models, such as GPT-3, can be used to generate legal questions. We compare machine-generated questions against human-composed questions and find that MGQs are more scalable, cost-effective, and more diversified, while HCQs are more precise. We also show a prototype of CRec and illustrate through an example how our 3-step approach effectively brings relevant legal knowledge to the public. (shrink)

Retrieval of relevant unstructured information from the ever-increasing textual communications of individuals and businesses has become a major barrier to effective litigation/defense, mergers/acquisitions, and regulatory compliance. Such e-discovery requires simultaneously high precision with high recall (high-P/R) and is therefore a prototype for many legal reasoning tasks. The requisite exhaustive information retrieval (IR) system must employ very different techniques than those applicable in the hyper-precise, consumer search task where insignificant recall is the accepted norm. We apply (...) Russell, et al.’s cognitive task analysis of sensemaking by intelligence analysts to develop a semi-autonomous system that achieves high IR accuracy of F1 ≥ 0.8 compared to F1 < 0.4 typical of computer-assisted human-assessment (CAHA) or alternative approaches such as Roitblat, et al.’s. By understanding the ‘Learning Loop Complexes’ of lawyers engaged in successful small-scale document review, we have used socio-technical design principles to create roles, processes, and technologies for scalable human-assisted computer-assessment (HACA). Results from the NIST-TREC Legal Track’s interactive task from both 2008 and 2009 validate the efficacy of this sensemaking approach to the high-P/R IR task. (shrink)

In this article we evaluate the BankXX program from several perspectives. BankXX is a case-based legal argument program that retrieves cases and other legal knowledge pertinent to a legal argument through a combination of heuristic search and knowledge-based indexing. The program is described in detail in a companion article in Artificial Intelligence and Law 4: 1--71, 1996. Three perspectives are used to evaluate BankXX:(1) classical information retrieval measures of precision and recall applied against a hand-coded (...) baseline; (2) knowledge-representation and case-based reasoning, where the baseline is provided by the functionality of a well-known case-based argument program, HYPO (Ashley, 1990); and (3) search, in which the performance of BankXX run with various parameter settings, for instance, resource limits, is compared. In this article we report on an extensive series of experiments performed to evaluate the program. We also describe two additional experiments concerning(1) the program's search behavior; and (2) the use of a modified form of precision and recall based on case similarity. Finally we offer some general conclusions that might be drawn from these particular experiments. (shrink)

Considerable attention has been given to the accessibility of legal documents, such as legislation and case law, both in legal information retrieval (query formulation, search algorithms), in legal information dissemination practice (numerous examples of on-line access to formal sources of law), and in legal knowledge-based systems (by translating the contents of those documents to ready-to-use rule and case-based systems). However, within AI & law, it has hardly ever been tried to make the contents (...) of sources of law, and the relations among them, more accessible to those without a legal education. This article presents a theory about translating sources of law into information accessible to persons without a legal education. It illustrates the theory by providing two elaborated examples of such translation ventures. In the first example, formal sources of law in the domain of exchanging police information are translated into rules of thumb useful for policemen. In the second example, the goal of providing non-legal professionals with insight into legislative procedures is translated into a framework for making available sources of law through an integrated legislative calendar. Although the theory itself does not support automating the several stages described, in this article some hints are given as to what such automation would have to look like. (shrink)

The proliferation of legal documents in various formats and their dispersion across multiple courts present a significant challenge for users seeking precise matches to their information requirements. Despite notable advancements in legal information retrieval systems, research into legal recommender systems remains limited. A plausible factor contributing to this scarcity could be the absence of extensive publicly accessible datasets or benchmarks. While a few studies have emerged in this field, a comprehensive analysis of the distinct (...) attributes of legal data that influence the design of effective legal recommenders is notably absent in the current literature. This paper addresses this gap by initially amassing a comprehensive session-based dataset from Jusbrasil, one of Brazil’s largest online legal platforms. Subsequently, we scrutinize and discourse key facets of legal session-based recommendation data, including session duration, types of recommendable legal artifacts, coverage, and popularity. Furthermore, we introduce the first session-based recommendation benchmark tailored to the legal domain, shedding light on the performance and constraints of several renowned session-based recommendation approaches. These evaluations are based on real-world data sourced from Jusbrasil. (shrink)

Legal scholarship emphasizes generalizing across places, time periods, and sources of law. Microcomputers can facilitate well-organized information retrieval systems, inductive statistical analysis, and prescriptive analysis working with goals to be achieved and available alternatives. Super-optimizing can help resolve legal disputes, dilemmas, and policy controversies whereby all sides, viewpoints, and ideological positions can come out ahead of their best initial expectations simultaneously. This book discusses these three important subjects by generating relevant principles based on developmental law, (...) class='Hi'>legal policy analysis, law teaching, and the judicial process. (shrink)

The appropriate understanding and fast processing of lengthy legal documents are computationally challenging problems. Designing efficient automatic summarization techniques can potentially be the key to deal with such issues. Extractive summarization is one of the most popular approaches for forming summaries out of such lengthy documents, via the process of summary-relevant sentence selection. An efficient application of this approach involves appropriate scoring of sentences, which helps in the identification of more informative and essential sentences from the document. In this (...) work, a novel sentence scoring approach DCESumm is proposed which consists of supervised sentence-level summary relevance prediction, as well as unsupervised clustering-based document-level score enhancement. Experimental results on two legal document summarization datasets, BillSum and Forum of Information Retrieval Evaluation (FIRE), reveal that the proposed approach can achieve significant improvements over the current state-of-the-art approaches. More specifically it achieves ROUGE metric F1-score improvements of (1−6)% and (6−12)% for the BillSum and FIRE test sets respectively. Such impressive summarization results suggest the usefulness of the proposed approach in finding the gist of a lengthy legal document, thereby providing crucial assistance to legal practitioners. (shrink)

Identifying, classifying, and analyzing arguments in legal discourse has been a prominent area of research since the inception of the argument mining field. However, there has been a major discrepancy between the way natural language processing (NLP) researchers model and annotate arguments in court decisions and the way legal experts understand and analyze legal argumentation. While computational approaches typically simplify arguments into generic premises and claims, arguments in legal research usually exhibit a rich typology that is (...) important for gaining insights into the particular case and applications of law in general. We address this problem and make several substantial contributions to move the field forward. First, we design a new annotation scheme for legal arguments in proceedings of the European Court of Human Rights (ECHR) that is deeply rooted in the theory and practice of legal argumentation research. Second, we compile and annotate a large corpus of 373 court decisions (2.3M tokens and 15k annotated argument spans). Finally, we train an argument mining model that outperforms state-of-the-art models in the legal NLP domain and provide a thorough expert-based evaluation. All datasets and source codes are available under open lincenses at https://github.com/trusthlt/mining-legal-arguments. (shrink)

This article seeks to address the problem of the ‘resource consumption bottleneck’ of creating legal semantic technologies manually. It describes a semantic role labeling based information extraction system to extract definitions and norms from legislation and represent them as structured norms in legal ontologies. The output is intended to help make laws more accessible, understandable, and searchable in a legal document management system.

The automatic prediction of court case judgments using Deep Learning and Natural Language Processing is challenged by the variety of norms and regulations, the inherent complexity of the forensic language, and the length of legal judgments. Although state-of-the-art transformer-based architectures and Large Language Models (LLMs) are pre-trained on large-scale datasets, the underlying model reasoning is not transparent to the legal expert. This paper jointly addresses court judgment prediction and explanation by not only predicting the judgment but also providing (...) legal experts with sentence-based explanations. To boost the performance of both tasks we leverage a legal named entity recognition step, which automatically annotates documents with meaningful domain-specific entity tags and masks the corresponding fine-grained descriptions. In such a way, transformer-based architectures and Large Language Models can attend to in-domain entity-related information in the inference process while neglecting irrelevant details. Furthermore, the explainer can boost the relevance of entity-enriched sentences while limiting the diffusion of potentially sensitive information. We also explore the use of in-context learning and lightweight fine-tuning to tailor LLMs to the legal language style and the downstream prediction and explanation tasks. The results obtained on a benchmark dataset from the Indian judicial system show the superior performance of entity-aware approaches to both judgment prediction and explanation. (shrink)

Automated legal reasoning and its application in smart contracts and automated decisions are increasingly attracting interest. In this context, ethical and legal concerns make it necessary for automated reasoners to justify in human-understandable terms the advice given. Logic Programming, specially Answer Set Programming, has a rich semantics and has been used to very concisely express complex knowledge. However, modelling discretionality to act and other vague concepts such as ambiguity cannot be expressed in top-down execution models based on Prolog, (...) and in bottom-up execution models based on ASP the justifications are incomplete and/or not scalable. We propose to use s(CASP), a top-down execution model for predicate ASP, to model vague concepts following a set of patterns. We have implemented a framework, called s(LAW), to model, reason, and justify the applicable legislation and validate it by translating (and benchmarking) a representative use case, the criteria for the admission of students in the “Comunidad de Madrid”. (shrink)

This article addresses the problem of automatically extracting legal indices which express the important contents of legal documents. Legal indices are not limited to single-word keywords and compound-word keywords, they are also clause keywords. We approach index extraction using structural information of Japanese sentences, i.e. chunks and clauses. Based on the assumption that legal indices are composed of important tokens from the documents, extracting legal indices is treated as a problem of collecting chunks and (...) clauses that contain as many important tokens as possible. Each token is assigned a weight which is a statistical score, e.g. TF–IDF and Okapi BM25, to indicate its importance. The importance of a chunk or clause is determined based on the average weight of tokens included in that chunk or clause. Then, highly weighted chunks and clauses are recognized as the indices for legal documents. The experimental results on Japanese National Pension Act data show that our proposed method achieves better performance than TextRank, the most popular unsupervised method in extracting single-word and compound-word keywords. In addition, this approach is also applicable to extract clause keywords with high performance. (shrink)

Contemporary law offices use many different technologies for storing and retrieving documents produced in the course of legal work. This article examines two approaches in detail: document management, as exemplified by SoftSolutions, and electronic publishing, as exemplified by Folio VIEWS. Some other approaches are reviewed, and the pragmatics, politics, economics, and legalities of legal work product retrieval are discussed.

In this paper, we present a method for introducing law students to the writing of legal documents. The method uses a machine-readable representation of the legal knowledge to support document assembly and to help the students to understand how the assembly is performed. The knowledge base consists of enacted legislation, document templates, and assembly instructions. We propose a system called LEDAS (LEgal Document Assembly System) for the interactive assembly of legal documents. It guides users through the (...) assembly process and provides explanations of the interconnection between input data and claims stated in the document. The system acts as a platform for practicing drafting skills and has great potential as an education tool. It allows teachers to configure the system for the assembly of some particular type of legal document and then enables students to draft the documents by investigating which information is relevant for these documents and how the input data shape the final document. The generated legal document is complemented by a graphical representation of legal arguments expressed in the document. The system is based on existing legal standards to facilitate its introduction in the legal domain. Applicability of the system in the education of future lawyers is positively evaluated by the group of law students and their TA. (shrink)

Despite the public availability of legal documents, there is a need for finding specific information contained in them, such as paragraphs, clauses, items and so on. With such support, users could find more specific information than only finding whole legal documents. Some research efforts have been made in this area, but there is still a lot to be done to have legal information available more easily to be found. Thus, due to the large number (...) of published legal documents and the high degree of connectivity, simple access to the document is not enough. It is necessary to recover the related legal framework for a specific need. In other words, the retrieval of the set of legal documents and their parts related to a specific subject is necessary. Therefore, in this work, we present a proposal of a RDF-based graph to represent and search parts of legal documents, as the output of a set of terms that represents the pursued legal information. Such a proposal is well-grounded on an ontological view, which makes possible to describe the general structure of a legal system and the structure of legal documents, providing this way the grounds for the implementation of the proposed RDF graph in terms of the meaning of their parts and relationships. We posed several queries to retrieve parts of legal documents related to sets of words and the results were significant. (shrink)

The structures of discourse used by legal and ordinary languages share differences that foster technical issues when applying or fine-tuning general-purpose language models for open-domain question answering on legal resources. For example, longer sentences may be preferred in European laws (i.e., Brussels I bis Regulation EU 1215/2012) to reduce potential ambiguities and improve comprehensibility, distracting a language model trained on ordinary English. In this article, we investigate some mechanisms to isolate and capture the discursive patterns of legalese in (...) order to perform zero-shot question answering, i.e., without training on legal documents. Specifically, we use pre-trained open-domain answer retrieval systems and study what happens when changing the type of information to consider for retrieval. Indeed, by selecting only the important parts of discourse (e.g., elementary units of discourse, EDU for short, or abstract representations of meaning, AMR for short), we should be able to help the answer retriever identify the elements of interest. Hence, with this paper, we publish Q4EU, a new evaluation dataset that includes more than 70 questions and 200 answers on 6 different European norms, and study what happens to a baseline system when only EDUs or AMRs are used during information retrieval. Our results show that the versions using EDUs are overall the best, leading to state-of-the-art F1, precision, NDCG and MRR scores. (shrink)

This paper addresses the problems that lawyers experience retrieving information from legal-text databases. Traditional access mechanisms of text databases require users to know how information is stored. We propose a method for index organisation which shields lawyers from the internal storage structures and which allows them to address the legal databases in their own legal terms. The proposed index is based on a model of legal tasks as opposed to traditional database indexes which represent (...) the contents of the database. We will lay out the architecture of an information system in which this task model is used to determine the information need, to retrieve relevant documents and to give methodical guidance for the legal task itself. To account for the design of a task-based legal information retrieval system, a substantial part of this paper is devoted to analysis and representation of legal tasks. (shrink)

Named entity recognition (NER) is a very relevant task for text information retrieval in natural language processing (NLP) problems. Most recent state-of-the-art NER methods require humans to annotate and provide useful data for model training. However, using human power to identify, circumscribe and label entities manually can be very expensive in terms of time, money, and effort. This paper investigates the use of prompt-based language models (OpenAI’s GPT-3) and weak supervision in the legal domain. We apply both (...) strategies as alternative approaches to the traditional human-based annotation method, relying on computer power instead human effort for labeling, and subsequently compare model performance between computer and human-generated data. We also introduce combinations of all three mentioned methods (prompt-based, weak supervision, and human annotation), aiming to find ways to maintain high model efficiency and low annotation costs. We showed that, despite human labeling still maintaining better overall performance results, the alternative strategies and their combinations presented themselves as valid options, displaying positive results and similar model scores at lower costs. Final results demonstrate preservation of human-trained models scores averaging 74.0% for GPT-3, 95.6% for weak supervision, 90.7% for GPT + weak supervision combination, and 83.9% for GPT + 30% human-labeling combination. (shrink)

The context-based use of terminology and phraseology is one of the essential building blocks of legal translation. The contextual nature of both components has implications when it comes to designing resources that are adapted to the needs of translators. For Arabic legal translation, there are a multitude of different print and online resources available, however, they do not integrate the context-related parameter for term choice acceptability. In this article, we will describe the main features of certain bilingual (...) class='Hi'>legal dictionaries with the English-Arabic and French-Arabic language pairs. We will then make a descriptive assessment of the tools available online, highlighting their limitations. Taking into consideration all the contextual parameters involved in making a translation choice, we will put forward the value of developing bilingual ontologies with Arabic. With the rapid expansion of information technologies, a move towards formalizing legal knowledge will help fill existing gaps in the representation of Arabic legal content and the retrieval of information, providing legal translators with a tool that provides specific details that will enable translators to make informed and relevant decisions, in addition to opening new research perspectives for Arabic legal translation. (shrink)

We present an annotation scheme describing the argument structure of judgement documents, a central construct in Japanese law. To support the final goal of this work, namely summarisation aimed at the legal professions, we have designed blueprint models of summaries of various granularities, and our annotation model in turn is fitted around the information needed for the summaries. In this paper we report results of a manual annotation study, showing that the annotation is stable. The annotated corpus we (...) created contains 89 documents. We also designed and implemented the first two stages of an algorithm for the automatic extraction of argument structure, and present evaluation results. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:Landmark Legal Cases in BioethicsSusan Cartier Poland (bio)Only a few decades old, the interdisciplinary field of bioethics has developed surrounded by centuries of legal tradition and moral philosophy. Bioethics and the law have weaved back and forth over time influencing each field. Sometimes ethics leads the debate on problematical issues; for example, the Recombinant DNA Advisory Committee at the National Institutes of Health established regulations prior to (...) initiating human gene therapy. At other times, law takes over; Roe v. Wade, for example, has polarized and closed public discussion on abortion. Most frequently, however, scientific developments initiate discussion in both fields, as when the announcement of the birth of Dolly—the first cloned mammal—sparked President Clinton to ask the National Bioethics Advisory Commission for a report on cloning.A back-and-forth pattern also exists within the law itself, this time between legislator and judge. Legislators enact statutes, which apply universally to regulate conduct in society. Judges, on the other hand, interpret statutes and apply them to particular cases to resolve disputes between parties. Case law, as judge-made law is called, that grows in the absence of legislation becomes the common law. Legislatures can change the course of common law development by enacting more law, usually as amendments, but sometimes as repeals. A judge who sees the facts of a case or the law differently may also change the common law. And so the counterpoint between legislator and judge goes.A legal case becomes a landmark in bioethics because of its historical or precedential value to the field. Historical cases reflect judicial reasoning or rationalization that lays the conceptual foundation for a bioethical principle. Perhaps the best known example of this is the focus on the right to self-determination in Schloendorff that led to the development of informed consent. The principle of autonomy in bioethics derives its legal impetus from this line of cases.A case that acquires landmark status because of its precedential value does so for one of two reasons based on legal precedents. First, the case may be one of “first impression.” In the legal sense, that means that the particular case with its [End Page 191] facts or issues is the first one that a particular court has confronted within that court’s jurisdiction or area of decision-making authority. To be a landmark in bioethics, however, the ordinary sense of “first impression” applies. The first time that a particular bioethical topic has appeared in court for legal resolution makes a case one of “first impression” for bioethics. The case of Baby M on surrogacy contracts illustrates this.The second way in which a case can achieve landmark status based on precedents involves legal precedent and the jurisdictional basis of the court system. Under the common law system, a preceding case that involves the same or similar facts or issues as a current case must be followed by lower—i.e., less authoritative—courts in the same jurisdiction. That judicial rule of decision making is called stare decisis. It creates an institution of continuity, yet allows for change due to differences in judicial interpretation of facts and issues, which is called distinguishing the case.All courts under United States jurisdiction must follow the U.S. Supreme Court. By its sheer power of authority, the Court issues opinions that, if they concern bioethical topics, make cases landmarks in bioethics. For example, Roe v. Wade made abortion legal and set off the debate between maternal and fetal rights. Casey later refined, or to some redefined with a discussion of stare decisis, the bioethical underpinnings of Roe by framing the debate as one between maternal rights and state interests.In law, precedents concern only appellate cases. Trial courts, like that in McFall v. Shimp, handle questions of both fact and law. Appellate courts, like that in Brotherton, hear appeals from the lower trial courts and address legal issues only. Trial judges apply the law to facts, and appellate judges reason about the trial judge’s legal interpretation.Appellate court decisions are the ones most frequently published, either in print or electronically online, for citation. Consequently they are the easiest to locate and retrieve. “Reporters” as... (shrink)