DOCUMENT RESUME ED 467 818 TM 034 362 AUTHOR Plumer, Gilbert E. TITLE A Review of the LSAT Using Literature on Legal Reasoning. Law School Admission Council Computerized Testing Report. LSAC Research Report Series. INSTITUTION Law School Admission Council, Princeton, NJ. REPORT NO LSAC-R-97-08 PUB DATE 2000-02-00 NOTE 23p. PUB TYPE Information Analyses (070) EDRS PRICE EDRS Price MF01/PC01 Plus Postage. DESCRIPTORS Adaptive Testing; *College Entrance Examinations; *Computer Assisted Testing; Law Schools; Literature Reviews; *Skills; Test Items IDENTIFIERS *Law School Admission Test; *Test Specifications ABSTRACT In the context of examining the feasibility and advisability of computerizing the Law School Admission Test (LSAT), a review of current literature was conducted with the following goals: (1) determining the skills that are most important in good legal reasoning according to the literature; (2) determining the extent to which existing LSAT item types and subtypes are designed to assess these skills; and (3) suggesting test specifications and new or refined item types that could be developed to assess any such skills. Overall, the findings validate the current LSAT, with its existing item types and subtypes and their mix. The skills identified as important in recent literature on legal reasoning are the ones on which the LSAT focuses, reasoning and text comprehension and analysis. Some areas for change were identified at the level of specific skill categories. Some new test specifications and item types and formats are suggested as possible ways of aligning the skills that the LSAT is designed to assess. (Contains 38 references.) (SLD) Reproductions supplied by EDRS are the best that can be made from the original document. LSAC RESEARCH REPORT SERIES U.S. DEPARTMENT OF EDUCATION ) Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 41.14 document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality. Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. 1 PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS BEEN GRANTED BY J. VASELECK TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) A Review of the LSAT Using Literature on Legal Reasoning Gilbert E. Plumer Law School Admission Council Law School Admission Council Computerized Testing Report 97-08 February 2000 A Publication of the Law School Admission Council 0 LAW BEST COPY AVAILABLE The Law School Admission Council is a nonprofit corporation that provides services to the legal education community. Its members are 198 law schools in the United States and Canada. Copyright© 2000 by Law School Admission Council, Inc. These studies are published and distributed by the Law School Admission Council (LSAC). The opinions and conclusions contained in these reports are those of the authors and do not necessarily reflect the position or policy of the Law School Admission Council. 3 i Table of Contents Executive Summary 1 Abstract 1 Introduction 2 Deductive Reasoning 3 Skill 3 Current LSAT 4 Future 4 Analogical Reasoning 4 Skill 4 Current LSAT 5 Future 6 Identifying Implicit Assumptions 7 Skill 7 Current LSAT 8 Future 8 Creativity/Imagination 8 Skill 8 Current LSAT 8 Future 9 Interpretation 10 Skill 10 Curient LSAT 11 Future 11 Normative Thinking 12 Skill 12 Current LSAT 13 Future 13 Perceiving Logical Relations 14 Skill 14 Current LSAT 14 Future 15 Counseling and Negotiation 15 Skill 15 Current LSAT and Future 16 Conclusions 17 References 18 4 Executive Summary In 1995 the Law School Admission Council implemented a five-year research agenda to examine the feasibility and advisability of computerizing the Law School Admission Test (LSAT). Part of this agenda is the Skills Analysis study, the general purpose of which is to systematically identify skills important for success in law school. As one component of the Skills Analysis study, research using current literature on legal reasoning was conducted with the goals of (a) determining what skills are most important in good legal reasoning according to such literature, (b) determining the extent to which existing LSAT item types and subtypes are designed to assess those skills, and (c) suggesting test specifications or new or refined item types and formats that could be developed in the future to assess any important skills that appear [by (a) and (b)] to be measured in a limited or minimal way by the current LSAT. So far as can be determined, such systematic research using legal reasoning literature has never been previously conducted. Broadly speaking, the findings validate the current LSAT, with its item types and subtypes and their mix. The skills identified as important in recent literature on legal reasoning are primarily ones of reasoning and text comprehension and analysis, which constitute the focus of the current LSAT. However, at the level of specific skill categories, at least as they can be derived from this literature, there are areas relative to which changes in the LSAT should be considered. These possibilities, which are not conclusive although they are informed and educated, can be summarized in tabular form as follows. LSAT skills assessment Legal Reasoning Skill Category deductive reasoning analogical reasoning identifying implicit assumptions creativity/imagination interpretation normative thinking perceiving logical relations counseling and negotiation Representation On the Current LSAT thorough assessment limited assessment high degree of assessment minimal (formal) assessment high degree of assessment limited assessment high degree of assessment limited assessment A fair number of potential new test specifications and item types and formats are suggested in this report as possible ways of more closely aligning the skills that the LSAT is designed to assess with skills identified as important in legal reasoning literature. For example, a specification for Logical Reasoning inference, assumption, and technique subtypes could perhaps be developed whereby the item's credited response is always a conclusion, assumption, or analysis of an analogical argument. However, it should be noted that literature on legal reasoning is only one of a number of possible sources to use in identifying skills that the LSAT might assess. Indeed, other parts of the Skills Analysis study are focused on law school teaching materials and methods and on task analyses of law school courses, and this research may reveal other skills. Moreover, identifying skills to be assessed is only one of a number of factors that determine the makeup of the LSAT; cost of producing items, for instance, is another factor. Therefore, before any changes in the LSAT are actually instituted, an investigation that takes all such considerations into account would have to be undertaken. Abstract Research using current literature on legal reasoning was conducted with the goals of (a) determining what skills are most important in good legal reasoning according to such literature, (b) determining the extent to I am grateful to Richard Adams, Stephen Luebke, Peter Pashley, James Vaseleck, and Stephanie Walker for help with this report. 5 2 which existing Law School Admission Test (LSAT) item types and subtypes are designed to assess those skills, and (c) suggesting test specifications or new or refined item types and formats that could be developed in the future to assess any important skills that appear [by (a) and (b)] to be measured in a limited or minimal way by the current LSAT. So far as can be determined, such systematic research using legal reasoning literature has never been previously conducted. This report presents the findings of this research. Introduction The current Law School Admission Test (LSAT) is a paper-and-pencil exam, the format and features of which have not changed since June of 1991. The scored sections are 35 minutes each and consist of one Analytical Reasoning (AR) section and one Reading Comprehension (RC) section, and two Logical Reasoning (LR) sections. In addition, there is an unscored 30-minute Writing Sample that is sent to the law schools to which the candidate applies. In 1995 the Law School Admission Council implemented a five-year research agenda to examine the feasibility and advisability of computerizing the LSAT. Part of this agenda is the Skills Analysis study, the general purpose of which is to systematically identify skills important for success in law school (especially its first year, since the LSAT attempts to predict first-year performance primarily) and to study the relationships among those skills. Findings from this study will help to guide the development of potential new test specifications for a computer-adaptive LSAT as well as the development of potential new item (question) types and formats. The findings may also lead to refinements or validation of item types and formats that are currently used. The motivating factor is the desire to take advantage of the opportunity afforded by possibly computerizing the LSAT to evaluate its validity and, if appropriate, revise its features. As one part of the Skills Analysis study, research using current literature on legal reasoning was conducted with the goals of (a) determining what skills are most important in good legal reasoning according to such literature, (b) determining the extent to which existing LSAT item types and subtypes are designed to assess those skills, and (c) suggesting test specifications or new or refined item types and formats that could be developed in the future to assess any important skills that appear [by (a) and (b)] to be measured in a limited or minimal way by the current LSAT. The presentation below of the results of this research will be organized by the particular skill category identified. Not surprisingly, there will be some overlap among the categories. The research indicated in (a) was conducted mostly by reading, from the perspective of an informal logician, broadly-focused literature (listed in the References section) and thereby gaining an impression of the importance placed on various skills. This importance evaluation was not quantified (there may be no appropriate way to do so) and of course is not conclusive. References were sought and consulted until a saturation point was reached, that is, few new ideas were uncovered. The research indicated in (b) was usually also not quantified, but determinations of the extent to which the current LSAT attempts to assess the skills identified are based on the author's knowledge of the current LSAT content specifications and his experience (of approximately a decade) helping to develop LSAT questions and test forms. Research on the topic that is even this systematic has never been previously conducted, so far as can be determined. The focus on legal reasoning is appropriate in that "the legal system institutionalizes argument" (Ashley, 1990, p. 3), and in law school, particularly in its first year, the teaching is primarily directed toward method or getting students to think or reason "like a lawyer" through the study of "examples of legal thinking" (Burton, 1985, p. xiii; cf. passim Ashley, 1990; Delaney, 1987; Margulies & Lasson, 1993; Nathanson, 1994). However, this paper will let the authors studied define the subject matter in that there will be no attempt to define the term "legal reasoning," other than indirectly and inclusively, by taking legal reasoning to be what scholars who purport to broadly study it take it to be. In point of fact, only one of the authors to be considered, Carter (1994, p. 11), even tries to give an explicit and precise "definition of legal reasoning": Legal reasoning describes how a legal opinion combines the four elements: the facts established at trial, the rules that bear on the case, social background facts, and widely shared values. When judges reason well, their opinion harmonizes or 'fits together" well these four elements. 6 3 This definition seems too narrow, since it pertains only to the written opinions of (mostly appellate) judges. Legal reasoning takes place in many other settings, ranging from individual client consultation to the philosophical deliberations at a constitutional convention (e.g., Alexy, 1989, p. 211). Nevertheless, Carter's definition is useful, among other things, for illustrating the social importance of good legal reasoning. In speaking of several poorly reasoned opinions, he says (p. 236): "They are incoherent. They only confuse lawyers who must advise future clients with similar casesconfuse and thereby perhaps encourage litigation when law should instead encourage cooperation." Carter is perhaps best understood as giving, not a definition, but a "paradigm of legal reasoning," one that Samuelson (1997, p. 589) summarizes in more detail: [There is] the decision-maker who, through a species of inductive logic, identifies the legal issues presented by the facts of the case. Issue selection then yields to rule selection ... [R]ule selection results from the analogical process. Further, rule selection may flow from legal principles, which in turn often invite dicta. Legal outcomes or holdings obtain from the application of rules to the facts of the case, a process involving deductive logic. These outcomes can be justified variously by the legal syllogism, by precedent, and by [normative] policy. This indicates many of the skills that will be discussed in the sections that follow. It should be noted that literature on legal reasoning is only one of a number of possible sources to use in identifying skills that the LSAT might assess. Indeed, other parts of the Skills Analysis study are focused on law school teaching materials and methods and on task analyses of law school courses, and this research may reveal other skills. Moreover, identifying skills to be assessed is only one of a number of factors that determine the makeup of the LSAT; cost of producing items, for instance, is another factor. Therefore, before any changes in the LSAT are actually instituted, an investigation that takes all such considerations into account would have to be undertaken. Deductive Reasoning Skill In deductive reasoning the premises or evidence purport to entail the conclusion with logical necessity, so that the conclusion must be true given the truth of the premises. Although some have questioned whether legal reasoning is ever strictly deductive, the mainstream view appears to be that restricted parts of legal reasoning are deductive, especially if evident implicit assumptions are taken into account in reconstructing legal arguments (Ashley, 1990, p. 2; Golding, 1984, pp. 37-38; Hart, 1980; Levine & Saunders, 1993, p. 119; MacCormick, 1978, p. 19ff.; Samuelson, 1997, p. 585; Soeteman, 1989, esp. pp. 6, 16; Wasserstrom, 1961, ch. 2). The classic form of deductive legal argument or "the legal syllogism" (cf. Alexy, 1989, p. 221ff.; Burton, 1985, p. 42; Hart, 1980, p. 135; Wasserstrom, 1961, p. 15) begins with a statement of a rule or law (If P, then Q), attempts to establish that a particular case falls under the rule (that it is an instance of P), and concludes that the consequences specified by the rule apply to the particular case (that it is an instance of Q). There are many variations of this classic form, for example, Coode's from 1848: "(Case) Where any Quaker refuses to pay any church rates, (Condition) if any church warden complains thereof, (Subject) one of the next Justices of the Peace (Action) may summon such Quaker" (quoted in Bhatia, 1994, p. 150). Moreover, there are any number of other patterns of deductive argument in which legal reasoning may appear, whether they be formal or syntactic (e.g., reductio ad absurdum), or more semantical or substantive in relying on determinate conceptual connections (e.g., 'this is an automobile, so it is a vehicle') (cf. Alexy, 1989, p. 224ff.; Golding, 1984, p. 37ff.). Of course these patterns are not limited to a court's reaching a decision; they may appear in virtually any circumstance of legal reasoning. 7 4 Current LSAT Questions that are designed to assess deductive reasoning skills generally form well over one fourth of the current LSAT. This is because such questions constitute all of the questions in the AR section, usually many of the questions in the two LR sections, and on occasion, some of the questions in the RC section. AR questions test the ability to consider a group of facts and conditions or rules, and ascertain, given that group, what could or must be true with logical certainty. While the specific scenarios in AR sets are usually unrelated to law, the skills tested are intended to be the same as or analogous to the skills involved in ascertaining what could or must be the case given a set of regulations, the terms of a contract, or the facts of a legal case in relation to the law. In LR, certain question subtypes test for deductive reasoning skills directly, as when the examinee is asked to identify a deductively certain inference (this also applies to RC) or to identify an implicit assumption of a deductive argument. Other LR subtypes test for deductive reasoning skills less directly by asking the examinee to identify, with respect to a deductive argument, a reasoning error that might be present, its pattern of reasoning (or to match such patterns), or the argumentative role played by a statement in the argument. In RC, too, examinees are sometimes asked to identify the pattern of reasoning of a deductive argument. Future The current LSAT is designed to thoroughly assess deductive reasoning skills. Relative to what can be determined from legal reasoning literature, it is possible that this assessment is too thorough. This might not appear to be so, given that, for example, "if one were to look no further than the opinions that judges write to accompany their decisions, it would not occur to one that the decision process could be anything but deductive" (Wasserstrom, 1961, p. 16). However, part of what is operating here may be a desire to make it seem as if the decision is not based on questionable moral, political, or interpretative judgments. Soeteman refers to this as "the alibi-function of logic" (1989, p. 229) and Sunstein as the "vice of formalism" (1996, p. 24; cf. Hart, 1961, p. 126; Herbeck, 1995, p. 721ff.; Levine & Saunders, 1993, pp. 113-14; Maley, 1994, p. 28), where the opinion or justification seems logically compelling only if the contestable character of the premises, interpretations, or rule applications is ignored. The point is, a significant portion of legal reasoning that may appear to be deductive or "mechanical" may not be so in fact. (Ways that legal reasoning is nondeductive will be examined in later sections.) A reduction in the amount of deductive reasoning on the LSAT could be accomplished by reducing the proportion of deductive LR items of the sorts described above or by reducing the proportion of the test composed of AR items. Analogical Reasoning Skill This form of reasoning is generally held to be nondeductive and to involve making judgments on difficult actual cases on the basis of a consideration of clearer actual or hypothetical cases where judgments are firmer together with a consideration of relevant similarities and dissimilarities between the cases (cf. Ashley, 1990, p. 3; Sunstein, 1996, p. 8). In legal literature, analogical reasoning is often described in terms of reasoning from case to case "by example" (Hart, 1980, p. 136). In the commonplace act of analogical legal reasoning, the decision-maker consults precedents similar to the case at hand to see whether the similarities are enough to yield the same result. An underlying principle that motivates and justifies is basic fairness or justice: "in our legal system like cases are to be decided alike and different ones differently" (Samuelson, 1997, p. 578). Insofar as rules of law are contained in precedents, analogical reasoning is used to determine the applicable rule: is the case at hand an instance of the kind of thing (e.g., burglary of the first degree) governed by the rule (p. 579)? Sometimes the sheer number of similarities is emphasized, although how "compelling" or relevant they are seems always to play a part (cf. p. 581). But argument by analogy in law can involve greater inferential leaps by allowing a rule or ruling to contribute to a decision on matters to which it is only indirectly applicable 5 (Levine & Saunders, 1993, p. 120; MacCormick, 1978, p. 155). It is used to extend a precedent to a new subject matter, but also to restrict a precedent to a narrower range in light of factual dissimilarities, conflicting rules, and so on, which is called "distinguishing" (e.g., Alexy, 1989, p. 279; Hart, 1961, p. 131; Stoljar, 1980, p. 113). Sometimes, as Carter (1994, p. 144) points out, the precedents prove so remote, so factually different ... that a decision for either party in the case will create ... a new and different law ... In other situations, the reverse happens. The judge faces a precedent so factually similar to the one before her that she cannot distinguish or ignore it. If she chooses to reach a new result, she must overrule the precedent. Analogical reasoning in law is not only precedent-reasoning; statutory interpretation, for example, may involve analogical reasoning, as when a judge must fill gaps in a statute based on a determination of similarity with what the statute actually says (cf. MacCormick, 1978 , p. 193; Stoljar, 1980, p. 146). Analogical reasoning is used to help develop "intermediate rules that define the legal predicates," that is, intermediate rules that "the law's many statutory, constitutional, and administrative rules lack" (Ashley, 1990, pp. 3-4). In legal problem solving, the "transference" of "skills from one legal context to another" is based on analogical reasoning (Nathanson, 1994, pp. 226-27). Clearly, a fundamental step in reasoning analogically is discerning similarities and differences or "reconciling and synthesizing cases" (Delaney, 1987, p. 17), which itself may be no easy task in view of such linguistic or conceptual phenomena as "family resemblance": "A shares characteristics with B; B shares characteristics with C; A does not share any nontrivial characteristic with C. A and C may belong to the same classification" (Burton, 1985, p. 92). One writer (Sunstein, 1996, p. 99) argues that "analogical reasoning lies at the heart of legal thinking" because It is admirably well-suited to the particular roles in which lawyers and judges find themselves to a system in which heterogeneous people must reach closure despite their limitations of time and capacity and despite their disagreements on fundamental issues. (In a very suggestive name, Sunstein calls these occasions of reaching closure "incompletely theorized agreements." Cf. Carter, 1994, p. 207.) Current LSAT As indicated by the preceding discussion, analogical reasoning consists of determining similarities and differences, and determining which of these are relevant, and on the basis of this extending a claim about one subject matter to another. Any of the RC and LR item subtypes can indirectly test for analogical reasoning skills if the passage involves analogy or argument from analogy. In addition, on verbal reasoning tests like the LSAT, the test taker generally must be able to understand and evaluate similarities and differences, and determine the accuracy of paraphrase, between concepts in the passages and concepts in the questions. Along these lines, in RC there is a common subtype where often the test taker must recognize what is and what is not presented in the passage from paraphrase in the question and answer choices. More directly, in RC the task in one subtype is to extend information in the passage to a new situation or new material not discussed in the passage (e.g., "Which one of the following works, if published, would most likely use an historical approach to revolution that the author of the passage would endorse?"). In another RC subtype the task is often to discern situations or arguments that are analogous to those discussed in the passage (e.g., "Which one of the following is most closely analogous to the process of classification in insurance, as it is described in the passage?"). In LR the task in one subtype is to recognize the technique, structure, or reasoning pattern exhibited in different arguments, or the principles applied in different arguments or states of affairs, and compare them, evaluating which are most similar (e.g., "Of the following, which one best illustrates the principle that the passage illustrates?", or "The pattern of reasoning in the argument above is most similar to that in which one of the following?"). In another LR subtype the task is essentially the same, except that it is 9 6 performed with respect to fallacious arguments (e.g., "The flawed pattern of reasoning in the argument above is most similar to that in which one of the following?"). Future Given that the common-law method "places a premium on analogical thinking," and even that "analogies are often crucial in deciding on the meaning of rules themselves" (Sunstein, 1996, p. 63), analogical reasoning is a major part of "legal logic" (Stoljar, 1980, p. 145). It is not clear that the current LSAT substantially assesses analogical reasoning skills, and this applies perhaps especially to the distinctive reasoning involved, as opposed to simply discerning similarities and differences. The subtypes described above as more directly attempting to test for analogical reasoning skills form only a small part of an LSAT. One reason for this is that such test items are difficult to make defensible, because of the general looseness of analogical reasoning. Nevertheless, a specification for LR inference, assumption, and technique subtypes could perhaps be developed whereby the item's credited response is always a conclusion, assumption, or analysis of an analogical argument. What is envisioned here is a test section specification that would mandate that a certain minimum and maximum number of passages in an LR section be analogical arguments. Use could even be made of existing AR scenarios by constructing questions in which the credited response is a scenario or scenario fragment that is formally analogous or isomorphic to the given scenario or part of it. Another possible approach is exhibited in Example 1, which was recently developed for the LSAT computerized prototype (the prototype is being demonstrated at Law School Forums, by prelaw advisors, and so on to survey reactions to potential features of a computerized LSAT). Here the task for the test taker is to create, within the array of possibilities presented, a compound statement about the platypus that is analogous to the main argument in the passage, which is an examination of why medicine is a profession. This RC item differs from the current (paper-and-pencil) LSAT questions insofar as it is more open-ended and the candidate constructs the response. Probable further development would improve the constructive aspect by including a "solution box" into which the candidate would drag and drop the two sentences in their proper order. (Actually, this feature has already been developed for another RC item in the prototype that asks the candidate to construct a sentence that "most completely and accurately expresses the main idea of the passage"; this indicates that the basic design could serve as a template for a variety of item subtypes. Cf. ACT, 1997.) 1 0 7 n n n n n EXAMPLE 1 Which two of the following sentences, when considered together, are most analogous to a summary of the main argument in the passage? Place the "1" next to the sentence that should come first, and the "2" next to the sentence that should come second. 1 2 Careful investigation of the behavior of the platypus only obscures the task of its classification. Mere physical characteristics are insufficient in classifying the platypus; the animal's behavior must also be considered. It is the platypus' prominent physical characteristics that provide the key to the animal's classification. In fact, it is through this aspect of the animal that its true nature can be determined, hence illuminating its proper classification. Only after careful and balanced consideration of both aspects can the animal be properly classified. Reset I It may be possible as well to revive an experimental version of the "Principles and Cases" item type. Principles and Cases appeared on the LSAT from 1949 to 1982, and the experimental version, called "Decided Cases," was pretested circa 1960 (Reese & Cotter, 1994). Here the test taker was presented with factual details and the court's decision in several hypothetical law cases. Then, on the basis of this hypothetical "body of law," the task was to select the correct decision and associated reason (which evoked a principle) specific to each case in a series for which only the factual details were given. The items were designed to assess analogical precedent-reasoning skills and the ability to extract principles by generalizing over the hypothetical body of law. The questions were intended not to presuppose any specific legal knowledge. Indeed, such questions could perhaps be written in terms appropriate to other normative or practical reasoning contexts, such as business situations. (Principle-case reasoning will be discussed more in later sections, but mainly in the Normative Thinking section.) Identifying Implicit Assumptions Skill In order to fully understand any legal argument, whether deductive or nondeductive, one may need to fill in implicit assumptions that, on logical grounds, are either necessary to establish the conclusion of the argument or 11 It EST COPY MAHATMA 8 sufficient to properly draw the conclusion (Golding, 1984, p. 37ff.; Saunders, 1994, p. 573ff.; Soeteman, 1989, p. 15ff.). For example, an argument intended to produce the best explanation will presume the falsity of alternative explanations, or in an argument by analogy discussion of certain relevant similarities might be left implicit since they are regarded as obvious. Current LSAT In the LR sections of the LSAT there is a subtype where the express task is to identify implicit assumptions. The following three question stems indicate the variety within the subtype: "Each of the following is an assumption required by the argument EXCEPT", "Which one of the following, if assumed, enables the conclusion above to be properly drawn?", and where in the passage there is a blank in the place of an explicit premise "Which one of the following most logically completes the argument?" Most such items on the current LSAT probably involve an argument structure that is more or less deductive (but there does not appear to be any clear necessity to this). In addition, answering almost any item on the test, especially in LR and RC, can indirectly create the need for the examinee to perceive assumptions implicit in the material. Such items include those that ask the examinee to determine the main point of a passage or argument, to draw inferences from it, to determine its structure, and so forth. Future The current LSAT substantially attempts to test for the skill of identifying implicit assumptions. Nevertheless, certain changes might be instituted in order to assess this skill in a more varied way, on the presumption that this might lead to a broader and more accurate assessment of the skill. One possibility is to have more nondeductive and fewer deductive assumption items. Another possibility is to increase the relative number of items of the variety (mentioned above) where in the stated argument there is a blank rather than the explicit premise (the current LSAT has very few of these). With computerization, a constructed-response element could be added to this item variety simply by making an option, once selected, actually appear in the blank. This would allow the test taker to physically see how the completed argument would read. Another possible approach would be to ask, with respect to a number of answer choices, which could function as a reason for a stated conclusion (ef Thomson, 1994, pp. 13-14) using a stated principle. This would pertain to the ability to perceive potential assumptions or premises in the course of applying a principle. Creativity/Imagination Skill As MacCormick points out, framing a "case as a legal claim or defence ... depends upon the legal acumen and experience, and indeed the creative imagination, of the good lawyer ... boldness, resourcefulness, and imagination ... must go into the formulation and arguing of a difficult case" (1978, pp. 121-22). He further points out, for example, that the formulation of a general principle that expresses "the underlying common purpose of a set of specific rules ... calls for a real effort of the creative imagination" (p. 126). Relating such aptitudes to analogical reasoning, Sunstein says (1996, pp. 99-100): "There is nothing static to the analogical process; it leaves room for flexibility and indeed for an enormous amount of creativity. Much creativity in law comes from the ability to see novel analogies." Nathanson argues that "the successful performance of any legal skill depends on ... aflexible approach, the ... ability to treat the theories and modelsand the legal problem itselfwith imagination and resourcefulness" (1994, p. 220). Current LSAT Regarding the scored sections of the current LSAT, it seems clear that the relatively few questions of subtypes described earlier as designed to more directly assess analogical reasoning skills also test for 9 (disciplined) creativity and imagination to some significant degree (the application of undisciplined creativity and imagination can lead to getting such items wrong). Moreover, determining which of a number of possible approaches to take (and often, which sort of diagram to draw) in "solving" an AR set is another place where it is fairly clear that skills of creativity and imagination are tested for, although here it is only indirect (no diagram drawn is scored, for example). Finally, one of the purposes of the unscored Writing Sample is to provide evidence of creativity and insightfulness (as well as evidence of analytical, advocacy, and communication skills). An LSAT Writing Sample topic describes a decision scenario with two mutually exclusive options by giving relevant facts as well as two rules, guidelines, or criteria under which the decision is to be made. The examinee writes an argument in favor of one of the options over the other. Future This minimal formal assessment of skills of creativity and imagination could perhaps be significantly increased by having a scored Writing Sample (additional reasons for having a scored Writing Sample will be indicated below). With the advent of computer programs designed to help measure writing abilities (Burstein, et al., 1998), perhaps the scoring of the Writing Sample could be computer-assisted. In general, with computerized testing there may be more opportunities to assess, or at least allow for more expression of, skills of creativity and imagination through the use of open-ended or constructed-response items. Example 2 is an LR item in the computerized LSAT prototype where the task is to construct an argument with a logical structure as similar as possible to that of a given argument. In the prototype the examinee does this by dragging and dropping three words into six boxes that are argument term place-holders, and there is more than one, of the many possible arrays, that constitutes a correct answer. Similarly, Example 3 is an AR item in the computerized LSAT prototype where (by turning on lights) the task is to create a model, from the numerous ones that are possible, that meets the stated conditions. (In another AR question in the prototype there are fifty-two distinct possible correct answers.) Perhaps how many of the correct answers that the examinee constructs could be taken into account in scoring. EXAMPLE 2 Although all contemporary advertising tries to persuade, just a small portion of contemporary advertising can be considered morally reprehensible. It nevertheless follows that some attempts at persuasion can be regarded as morally reprehensible. By dragging and dropping the three words provided, fill in the boxes to construct an argument that makes only true statements and that has a logical structure as similar as possible to that of the argument above. Some and all 9 are 9 cats mammals males 9 are , since only some 9 9 Reset I 13 are 9 10 EXAMPLE 3 A square parking lot has exactly eight lightsnumbered one through eight situated along its perimeter. The lot must always be illuminated in such a way that the following specifications are met: At least one of any three consecutively numbered lights is off. Light 8 is on. Neither light 2 nor light 7 is on when light 1 is on. At least one of the three lights on each side is on. If any side has exactly one of its three lights on, then that light is its center light. Two of the lights on the north side are on. West Create a model that could be a complete and accurate list of lights that are on together. North South Click on a number to turn light on. Click number again to turn light off. Interpretation Skill Reset East Interpreting any text (or speech) involves such activities as determining what is important and what is ancillary; attributing structure; determining the intention underlying the words; seeing the stated view as part of a larger view; and resolving unclarity, vagueness ("open texture"), or ambiguity. Meaning is determined partly by taking into account surroundit "context and culture" (Sunstein, 1996, p. 122) and the fact that language evolves over time (Scallen, 1995, p. 709); treating text as simply "a series of Webster's definitions strung together" is to be avoided (Carter, 1994, p. 58; cf. Sunstein, 1996, p. 123). Clearly, much of law involves or requires interpreting statutes, opinions, depositions, summations, contracts, and so on. "Extricating key facts" and "issue-spotting" are held to be "basic skills required for legal reasoning" (Delaney, 1987, pp. 10-11; cf. Levine & Saunders, 1993, p.112). Samuelson regards issue-spotting as "the first step in legal problem-solving" and as "legal induction" insofar as it has the form of "generalization from particulars" (1997, p. 577). And consider, for example, a statutory rule that states that a will must be witnessed " 'by two witnesses'. The statute does not say 'at least two witnesses' or 'at most two witnesses' or 'two and only two witnesses' "so is a will that is witnessed by three persons valid (Burton, 1985, p.127)? Indeed, even for such legal systems based on common law as those of England and Australia, Maley (1994, p. 28) cites an estimation that "approximately 40 percent of the work of the court requires a ruling upon the meaning of a particular piece of some legislative instrument." Two common principles of legal interpretation (MacCormick, 1978, p. 206, and 1995, p. 478) appear to be, first, that the nonliteral or nonsuperficial sense in which one proposes to read a text must at least be consistent with established English usage. Second, that one must not depart from a more straightforward sense unless one has good reason to do so. (Compare Sunstein, 1996, p. 124: "literal language will not be applied to a case if the application would produce absurdity or gross injustice.") One general approach taken, for instance, in statutory interpretation is to be guided by an attempt to formulate "the purpose of the statute, i.e., the problem statutory language tries to solve"; another approach is to be guided by an attempt to identify the specific historical 14 11 intentions of the legislation's originators or by the chain of interpretive history itself (Carter, 1994, pp. 85, 189; cf. Alexy, 1989, pp. 236, 240-41; MacCormick, 1995, p. 474; Sunstein, 1996, pp. 187-88). Alexy treats these and various other approaches or "canons of interpretation" as indicating "argument forms" that help to constitute "a grammar of legal argumentation" (1989, p. 245). Other interpretative argument forms include reasoning that, since under relevantly similar circumstances a certain interpretative solution was tried in the past and found to be wanting, it should not be tried again (p. 239), or basing an interpretation not on "the aims of any past or present actually existing person but rather on 'rational' aims" or 'normative' purposes to which anyone should subscribe (p. 241). Here, interpretation may involve "an improvement of the law" rather than simply establishing 'the lexical or contextual sense of a legal text' (Peczenik, 1995, p. 751). To these forms MacCormick (1995, p. 479) adds a "principle of justice that forbids retrospective judicial rewriting of the legislature's chosen words," and a "principle of rationality" that recognizes "the value of coherence and integrity in a legal system." Certainly then, with so many and various bases of legal interpretation, one may at once be pulled in different directions and need to resolve the conflict (a kind of situation that will be discussed more in the next three sections). Legal discourse tends to be highly stylized, so understanding it requires penetration of its stylistic elements. When a "dispute enters the legal system and becomes a 'case', its expression is transformed"; it is selectively reformulated "to conform to the requirements of the legal categories applicable" (Conley & O'Barr, 1990, p. 168). As cited by O'Toole (1994, p. 190), stylistic elements of legal language in general include " 'frequent use of formal words ... deliberate use of words and expressions with flexible meanings ... frequent use of common words with uncommon meanings'." Words may be used in a "technical legal sense" (MacCormick, 1995, p. 472). A device in legal discourse that has "affinities with metaphor" is "deeming" or "legal fiction," whereby for certain purposes one thing is taken to be another thing that "it patently is not," such as when "for the purposes of conviction, a person who incites etc. is the same as a person who actually commits the offence" (Maley, 1994, pp. 26-27). One study of the syntax of legislative provisions (Bhatia, 1994) found that they are marked by a tendency toward long sentences, nominalization of verbs (helping to make the provisions precise and all-inclusive, yet dense), and syntactical discontinuities. They also tend to use complex prepositional phrases, binomial and multinomial expressions (e.g., 'signed and delivered', 'in whole or in part'), long initial case descriptions that specify applicability (e.g., 'where 'if '), and many qualifications. Current LSAT Item subtypes that are designed to assess interpretive skills form a relatively large part of the current LSAT. In LR the task in one subtype is to analyze an argument or dispute and identify the main conclusion or issue involved. Another subtype often asks the examinee to identify the technique, structure, or reasoning pattern of an argument, or the principle or rule that underlies an argument or state of affairs. Additionally, sometimes in LR the task is to recognize a misinterpretation in an argumentative dialogue. In RC the task in one subtype is often to determine the main point, idea, or theme of a passage. In another subtype the task is to ascertain from the passage as a whole the author's purpose or intention in writing the passage. In other RC subtypes the examinee is asked to determine from context the meaning of a particular phrase used by the author in some unusual or special way; to identify the author's purpose in employing a particular phrase or concept; to analyze the passage to discover how the author organizes information to argue, persuade, or otherwise achieve her or his purpose; to infer from the information in the passage an implicit view of the author or some other person mentioned in the passage; and to infer from the more subtle elements of the passagefor example, tone or word choicean attitude of either the author or some other party discussed in the passage. Future As indicated by the number and variety of these subtypes, the current LSAT places a high degree of emphasis on assessing interpretative skills. It is worth mentioning, however, that in test development there is generally considerable value given to making passages clear, well-organized, unambiguous, and nontechnical in 15 12 order to have test items that are sound and fair (and not subject to challenge). Hence, a divergence can appear between passages on the LSAT and typical legal discourse. So in developing the LSAT it must be kept in mind that, to the extent that soundness and fairness permits, there is also value in having material where the task involves disambiguation, handling unclarity or vagueness, or dealing with text that exhibits some of the stylistic characteristics of legal discourse. Normative Thinking Skill Perhaps the defining character;stic of legal reasoning generally is that it is "a special, highly institutionalized and formalized, type of moral reasoning" (MacCormick, 1978, p. 272; cf. Alexy, 1989, p. 211ff.; Peczenik, 1995, p. 750ff.); other than this, there may be no distinctive "legal logic" (Soeteman, 1989, p. 22; cf. Samuelson, 1997, p. 571n). Normative, practical, or "moral" concepts, such as those of permissibility, obligation, values, and rights, pervade legal thinking (Soeteman, 1989, passim). Ethical theories and theories of justice are taken to underlie whole political-legal systems (cf. Sunstein, 1996, p. 37), as well as fundamental principles of law, notably for example, the legal principle that precedent should be followed (stare decisis) is thought to be analogous to Immanuel Kant's broad moral principle that one ought to act only on maxims that are universalizable (Golding, 1984, p. 98; cf. Alexy, 1989, p. 275; Soeteman, 1989, p. 155; Stoljar, 1980, p. 109ff.). Law institutionalizes and formalizes moral reasoning by adding "a further stabilising element to moral practices [in addition to stare decisis], principally by recording and storing moral decisions," making them "both more sophisticated and precise" and providing "an intellectual or cognitive base from which 'new' decisions and judgements can be made" (Stoljar, 1980, pp. 133-34; cf. Alexy, 1989, p. 287; Fuller, 1969, p. 205; Samuelson, 1997, pp. 573-75). Consider further that any legal argument that is cast in terms of what is best, in terms of outcomes, for an individual, institution, or society is teleological or utilitarian in nature (utilitarianism itself is a basic ethical theory), and any such argument that proposes or presupposes a way of bringing about the desired outcome is means-end reasoning (cf. Delaney, 1987, p. 34; Samuelson, 1997, pp. 582, 588; Stoljar, 1980, pp. 26, 54; Sunstein, 1996, p. 19). Any legal rule may be regarded as "being aimed at securing, some end conceived as valuable, or some general mode of conduct conceived to desirable" (MacCormick, 1978, p. 156; cf. Golding, 1984, p. 55ff.; Morawetz, 1980, p. 122). And "judicial decisions are moral decisions because they affect persons by benefiting or harming them" (Morawetz, 1980, p. 81). A nondeductive argument has premises that do not entail the conclusion with logical certainty. When nondeductive arguments are normative, however, it is thought that they have a special sort of conclusiveness: if there is good reason for the conclusion and no good reason against it, one is committed' to accepting and acting on the conclusion (Golding, 1984, pp. 107-08; cf. Soeteman, 1989, p. 156). There appears to be a wide consensus (Golding, 1984, pp. 58-59; MacCormick, 1978, p. 115; Margulies & Lasson, 1993, passim; Morawetz, 1980, p. 87; Saunders, 1994, pp. 574-76; Sunstein, 1996, pp. 79, 139-46) that to be good at legal reasoning one must be able to simultaneously take into account numerous factors, including competing values. There is no single scale of evaluation by which diverse norms-justice, common sense, convenience, safety, health, facilitation of democracy, and so forth-are made commensurable. One must be able to consider the complex of factors with an appreciation of the appropriate weight to be given to each. In part because often "the moral values that claim to govern a case collide ... judges will continuously create law" (Carter, 1994, p. 19; cf. Peczenik, 1995, pp. 751-53). Although legal reasoning has a strong normative aspect, the connection between legality and morality is complicated and should not be overstated. As discussed, legal reasoning tends to differ from moral reasoning in that the former is highly formalized and institutionalized. A ramification of this is that in contrast to moral or other normative reasoning, "legal argumentation can never proceed acceptably without some basis in some argument from authority," where the "authority" may be "statutes, precedents, doctrinal materials and the like" (MacCormick, 1995, pp. 469-70). Still, "authority has to be grounded in some way," either deontologically on 16 13 "rightness (justice)" or teleologically on "the good" brought about (p. 470). So there may be a general moral obligation to obey the legal law (Hart, 1961, p. 153; Peczenik, 1995, p. 751); "where there is authority exercised legitimately, disobedience is prima facie wrong" (MacCormick, 1995, p. 470). Yet there does not appear to be any such general legal obligation to be moral. And at a specific level, although an action may be both legally and morally permissible or both illegal and immoral, an action could also be legal yet immoral (e.g., in matters of certain personal relations; see Morawetz, 1980, pp. 131-33) or illegal yet moral (bad law). Perhaps the best way to state the relationship between legality and morality is to point out that often a legal rule is an attempt that is revocable (e.g., by an act of the legislature) to formulate a moral rule (cf. Hart, 1961, p. 171; Stoljar, 1980, p. 147). Current LSAT On the current LSAT there is no content specification that calls for subject matter that is normative or for items designed to assess normative reasoning per se except insofar as this is accomplished by the fact that in each RC section one of the passages must be law-related. However, the LSAT often contains other subject matter of this sort. For example, a humanities passage in RC might discuss an ethical theory or an LR stimulus might be about one kind of alleged intrinsic value. In RC and LR there are subtypes where the task is often to generalize from what is stated in the passage and identify principles that govern the arguments or examples in the passage, or to apply a principle expressed in the passage to a situation or case not found in the passage (e.g., "Which one of the following judgments conforms most closely to the principle cited above?"). Usually, these principles are normative. Sometimes in LR the examinee is asked to identify a fallacy that has to do specifically with normative reasoning (e.g., confusing having no obligation to do X with having an obligation not to do X or inferring from the fact that Xis permitted that a particular means of achieving X is permitted). Future To increase this limited extent to which the LSAT is designed to assess distinctively normative thinking abilities would probably constitute a step in the direction of knowledge or achievement testing. The LSAT has not had an overtly knowledge-based section or accompanying test since the early 1970s, when the most notable of these ("General Background") was dropped (Reese & Cotter, 1994). However, the step need not be a large one. The focus could be the logic or form of normative thinking, rather than on content or subject matter such as substantive theories of justice or ethics. In LR a test section specification could be written that calls for a certain minimum and maximum number of questions that are designed to assess the candidate's ability to reason with normative principles (the questions could fall under existing subtypes described in the preceding paragraph). Similarly, by specification a more regular number of LR items could ask the examinee to identify fallacies that directly concern normative reasoning. Some items could be written as described at the end of the section above on identifying implicit assumptions, where the task is to perceive potential assumptions or premises in the course of applying a principle. Another possibility is to take advantage of the computer's capabilities and, for example, ask the examinee to modify the description of a situation so that it fits a given normative principle (the feature of striking-through could be used on selected parts of the description, in the manner of an item on the current LSAT computerized prototype). In AR, a specification perhaps could be written whereby the conditions in a set constitute a group of hypothetical legal or ethical rules and the items concern what could or must be the case legally or ethically given the conditions, i.e., what is legally or ethically permissible or obligatory. Extracting rules or principles and applying them is thought to be fundamental in legal reasoning (e.g., Ashley, 1990, p. 232; Delaney, 1987, pp. 11-13; Samuelson, 1997, p. 579ff.); one study found these to be the most important elements that figure in the quality of legal (memoranda) writing (Breland & Hart, 1994, p. 34). And Stoljar argues that "to extract or formulate a ratio the lawyer ... cannot rely on all the facts of the previous case, he has to select those facts or features in it that may be relevant for the new case" by "knowing or appreciating the moral point or purpose behind the previous case" (1980, p. 111). Given this, an appropriate 17 14 way to increase the LSAT's assessment of normative thinking skills might be to have a version of the former item type, Principles and Cases. Another experimental version of this item type (in addition to the one described above at the end of the Analogical Reasoning section) was called "Related Cases." Here the examinee was first asked, with respect to a description of the facts and decision in a hypothetical court case, "The narrowest principle that provides the reasoning underlying this result is". Subsequent items in the section were similar, except that the question was "The narrowest principle that provides the reasoning underlying this result, and is not inconsistent with the preceding case, is". As with the version of Principles and Cases discussed earlier, the normative situations, judgments, and principles might not need to be specifically legal ones. (Evidently, long master lists of possible options or ideas for options, each consisting of likely principles in a certain area of law or other normative domain, could be generated for such test items, thereby reducing the cost of developing the items.) Perceiving Logical Relations Skill In a broad sense of "logical," logical relations include consistency, coherence, implication, conceptual containment, support or evidence, relevance, explanation, classification, illustration or instantiation, and generalization. Logical relations also include the negations or opposites of these, namely, inconsistency, incoherence, independence, counterexemplification, and so on. The logical relations involved help to define simplification and set membership, as well as means-end, principle-case, causal, and hypothetical or counterfactual reasoning (where the relation is that between actuality and possibility). Clearly, the ability to perceive logical relations is integral to good legal reasoning (see passim esp. Ashley, 1990; Golding, 1984; MacCormick, 1978; Margulies & Lasson, 1993; Stoljar, 1980; Sunstein, 1996). The simple applicability of more than one rule or provision to a case generates the possibility of conflict between them, and if there is indeed a conflict, the problem of resolution arises (Carter, 1994, p. 19; Saunders, 1994, p. 573-76; Soeteman, 1989, p. 215; Sunstein, 1996, p. 128). We may "treat the question 'Is r, s, t an instance ofp for the purposes of applying if p then q?' as the standard form" of the problem of classification, which for certain legal purposes may be distinguished from the problem of interpretation, as when the former is regarded as a question of "fact" and the latter a question of "law" (MacCormick, 1978, p. 95; cf. Golding, 1984, pp. 104-05). "In moral and legal discourse," argues Stoljar, "rules or principles ... are 'qualifiable' or merely prima facie statements, thus always open to exceptions" or counterexample (1980, p. 116). Ashley points out (1990, pp. 235, 232) that "hypotheticals" are used as analytical and rhetorical tools to test the consequences of a proposed holding, test the limits of a definition, simplify complicated fact situations for analysis ... It is no accident that the classic method for teaching law, the Socratic method, involves a professor's posing hypotheticals to test a student's understanding. In general, any set of legal statementsa legal code, the rulings cited in a decision, the terms of a contract, the statements made in testimony, and so forthwill exhibit logical relations among its members, and understanding will require perceiving such relations. There is "a logic" concerned with "system or order, with the analytical ordering of the material"; and for any legal issue, it is a "logical task ... to pool and classify the legal material ... . As our legal material is nowadays assuming quite massive proportions, its ordering or clarification has become a major undertaking" (Stoljar, 1980, p. 149; cf. Alexy, 1989, pp. 246, 251ff.). Current LSAT Item subtypes that are designed to assess the ability to perceive logical relations constitute a substantial part of the current LSAT. Many of the subtypes already described have this purpose; in AR, for example, all of the 18 15 questions test for the ability to apprehend deductive implications (entailment) or the relations of consistency and inconsistency between statements in the context of a "system" laid down by the conditions in a set's scenario. Depending on the nature of the AR set, also assessed is the ability to perceive conceptual connections that are spatial, temporal, causal, hierarchical, those of set membership, and so on. In addition, in LR the task in one subtype is sometimes to determine consistency and inconsistency (e.g., "If the statements above are true, each of the following could be true EXCEPT"). In another subtype the task is to identify fallacies in arguments, and the fallacies frequently involve logical relations such as self-contradiction, irrelevance (e.g., false authority or emotional appeals), misclassification, and hasty generalization, or logical relations help to define the fallacy, as in the case of confusing necessary and sufficient conditions, begging the question, false dilemma (failing to see possible alternatives), and the fallacies of composition and division. There is also a subtype in LR where the task is to identify a suitable possible explanation of facts or evidence, or to recognize what, if true (hypothetically), most helps to resolve an apparent discrepancy or conflict between facts or views. In both LR and RC there is a subtype where the task is to determine which statement of possible additional evidence, if true, most strengthens or undermines an argument, conclusion, hypothesis, or the like, or to identify a potential counterargument (e.g., "Which one of the following, if true, is the strongest counter Rita can make to Thomas' objection?"). Future Through numerous and varied question subtypes, the current LSAT is designed to directly or indirectly assess skills associated with perceiving logical relations. Still, several other approaches could possibly complement this assessment or make it even more comprehensive. To the subtypes discussed at the end of the preceding paragraph, a variation could be added where the task is to identify which hypothetical statement of additional evidence is most helpful (or relevant) in evaluating an argument, conclusion, and so forth. Advantage could be taken of the computer's capabilities, and, for example, items could be constructed to test generalization skills by presenting a list of general statements and a list of target instance statements and asking the examinee to match them one to one (cf. ACT, 1997). Another possibility would be to create a template chart with blanks labeled to represent argumentative structure. The test taker would be asked to drag and drop given statements into the correct blanks in the chart to create the most cogent argument. Such items would attempt to assess the ability to perceive implicit evidentiary relationships, and in general, logical relationships between premises and conclusion. Counseling and Negotiation Skill Much, if not most, legal reasoning takes place in the context of counseling or negotiation, where these are construed broadly to include "interviewing," consultation, advising, conciliation, mediation, and arbitration. "Most lawyers generally practice 'preventative law" (Carter, 1994, p. 4); indeed, "only one in ten cases ever goes to court and only one in ten of the cases heard in court are successfully appealed" (Feteris & Schuetz, 1995, p. 690). Negotiation and compromise are often encouraged in that "each side has an incentive to settle because each side knows it could lose" (Carter, 1994, p. 40n). The skills involved are brought to bear in such diverse settings as individual client consultation and corporate advising, as well as versions of alternative dispute resolution such as formal arbitration, administrative hearings, fact finding, moderated settlement conferences, private judging, and judging in small claims court (Carter, 1994, p. 4; Delaney, 1987, p. 19; ABA, 1992, part II; Walker & Daniels, 1995, p. 694). About the latter, for example, Conley and O'Barr (1990) point out that judges often interact with "relational litigants," i.e., those who "assign legal rights and responsibilities on the basis of social status and adherence to social conventions, and expect the courts to do the same" (p. 172), as opposed to "rule-oriented litigants," whose approach is more congenial to traditional legality because they "interpret disputes in terms of [legal] rules and principles that apply irrespective of social status" (p. 58). 19 16 Compare Walker and Daniels' (1995) discussion, respectively, of mediation and "multi-party facilitation" on the one hand, which tends to involve collaboration and cooperation, even "constructive refutation," and on the other hand, formal arbitration, which tends to be authoritarian and adversarial. While there may be many specific or technical skills on the order of "the ability to anticipate the other party's likely 'settling point' " that contribute to good counseling and negotiation (ABA, 1992, p. 187; cf. Walker & Daniels, 1995), our interest is in the general skills involved, insofar as they are constitutive of practical legal reasoning. Certainly fundamental are persuasive powers (Burton, 1985, p. 17) or the ability to take a position, present it clearly, and argue for it convincingly. Listening skills also seem fundamental. A strong case for this is given by Conley and O'Barr (1990), who document evidence and conclude (p. 173) that lay people [especially "relational litigants"] and legal professionals often hear each other as speaking different languages ... Gender, class, and race are deeply entangled with the knowledge of and ability to use the rule-oriented discourse that is the official approach of the law. There is a need for legal professionals to have the skills to "listen to participants' stories" and "reformulate the accounts selected to conform to the requirements of the legal categories applicable" (p. 168). Lay people tend to use the same narrative strategies that they use in ordinary social interaction, strategies that tend to be "unconstrained" (p. 37) and to "violate the rules of evidence in force in formal courts" (p. 56). Lay people tend to see "the rectitude of their own positions as absolutely self-evident" (p. 176), and to have difficulty adjusting to legal preferences for such things as "rational economic demands" (as opposed to "intangible benefits" p. 126), drawing a sharp distinction between fact and opinion, testimony from a consistent perspective, visual over aural information, and a formal or mechanistic assignment of responsibilityas opposed to "who 'ought' to bear responsibility ... in the larger social scheme" (p. 177). Hence, "sympathetic listening should be part of any settlement procedure" (p. 177), something that might help to avoid what are "perhaps the most common complaints of litigants at all levels of the legal process," namely, that "they did not get a proper opportunity to tell their story and that the judge did not get to the real facts in their case" (p. 172). A general counseling and negotiation skill that seems connected to listening skills is "the constructive use of silence," which Sunstein argues "can help minimize conflict, allow the present to learn from the future, and save a great deal of time and expense" (1996, p. 39). Sunstein sees constructively using silence as helping to constitute "incompletely theorized agreements," which form a kind of closure that Sunstein thinks is particularly appropriate for legal processes and is apt to be reached through analogical reasoning (as mentioned above). Current LSAT and Future In general, counseling and negotiation skills appear to consist partly of skills that the current LSAT is already designed to substantially assess. These latter skills include those that fall into the categories discussed above of identifying implicit assumptions, interpretation (e.g., identifying the issue in a dispute or the main point of an argument, clarifying meaning, and determining the author's purpose or attitude), and perceiving logical relations (e.g., making inferences, detecting argumentative flaws, and recognizing what would most help to resolve a conflict between facts or views). Similarly, the skills that contribute to arguing persuasively may be largely a function of skills that fall into the other skill categories identified in preceding sections, perhaps especially the categories of deductive reasoning, analogical reasoning, creativity/imagination, and normative thinking (cf. Levine & Saunders, 1993, p. 112). But, as not all of these categories are particularly well-represented on the current LSAT, a potential way to enhance this assessment (formally) would be to have a scored Writing Sample. In addition to persuasive powers or advocacy skills, and those of creativity and imagination (as mentioned earlier), a scored Writing Sample could be used to help assess other skills identified as contributing to good "lawyerly writing," namely, those skills that help to make the writing 'organized, purposeful, direct, clear, concise, complete,' and "logical (avoiding contradictions, inconsistencies and non-sequiturs)" (Delaney, 1987, pp. 13-14; cf. Breland & Hart, " 0 17 1994). An alternative approach that could perhaps achieve much the same result is a multiple-choice test with questions that ask the examinee to identify writing problems and solutions with respect to given drafts of essays (e.g., "Which of the following is the best as the last sentence of the memorandum?"). Such a prototype test was recently found to predict performance in legal writing courses better than the LSAT and better than undergraduate grade-point average (Breland, Carlton, & Taylor, 1998). To consider such questions, the Law School Admission Council has recently developed, and has begun implementing, a research agenda on the assessment of writing ability (Law School Admission Council, 1999). The current LSAT is not designed to test for listening skills in any direct way. Clearly, computerizing the LSAT would facilitate attempting to assess these skills. In fact, several computerized prototype sets of listening comprehension passages plus associated items have recently been developed (ACT, 1997) and informally tried out. Currently, passages are specified to be long enough to support 8-10 items and are either a lecture/speech or an argumentative or cooperative dialogue. Questions (aural or written) are then asked of recognition and recall (e.g., identify irrelevant matter or digression), analysis (e.g., identify a turning point in a dialogue, as where one party convinces the other), inference (e.g., generalize from or synthesize facts presented by a speaker or speakers), and response (e.g., pose appropriate questions of a speaker). A more sophisticated approach might be to combine the Writing Sample with a listening measure by presenting the Writing Sample prompt aurally. This could allow for the assessment of higher-level listening and thinking skills involving the ability to reformulate what is heard, understood, and recalled into a given set of categories or for a given purpose; even the ability to 'constructively use silence' in the written response could be taken into account. However, a difficulty with this approach is distinguishing measurement of different abilities, which could cause reliability problems. Of course there probably are other factors that contribute to having good counseling and negotiating abilities that would not be appropriate for the LSAT to measure, such as background knowledge of the culture of those with whom one interacts. Conclusions Broadly speaking, the findings presented above validate the current LSAT, with its item types and subtypes and their mix. The skills identified as important in recent literature on legal reasoning are primarily ones of reasoning and text comprehension and analysis, which constitute the focus of the current LSAT. However, at the level of specific skill categories, at least as they can be derived from this literature, there are areas relative to which changes in the LSAT should be considered. These possibilities, which are not conclusive although they are informed and educated, can be summarized in tabular form as follows. LSAT skills assessment Legal Reasoning Skill Category deductive reasoning analogical reasoning identifying implicit assumptions creativity/imagination interpretation normative thinking perceiving logical relations counseling and negotiation Representation On the Current LSAT thorough assessment limited assessment high degree of assessment minimal (formal) assessment high degree of assessment limited assessment high degree of assessment limited assessment A cautionary reminder in considering these results is that for the purposes of this report "legal reasoning" was understood inclusively to be more or less whatever scholars who purport to broadly study it take it to be. There was no attempt to restrict the subject matter of this report to legal reasoning as practiced in law school or its first year, even though the LSAT attempts to predict first-year p2f1rmance primarily. Such a restriction would be 18 difficult since very few legal reasoning scholars draw even a vague distinction along these lines. The restriction could also be ill-advised insofar as any kind of legal reasoning can involve other kinds, as has been evidenced here by overlap between the skill categories identified. A fair number of potential new test specifications and item types and formats have been suggested here as possible ways of more closely aligning the skills that the LSAT is designed to assess with skills identified as important in legal reasoning literature. Some presuppose the eventual computerization of the LSAT, but many do not. References ABA Section of Legal Education and Admissions to the Bar. (1992). Legal education and professional developmentAn educational continuum (Report of the Task Force on Law Schools and the Profession: Narrowing the Gap). Chicago: American Bar Association. ACT (1997). Proposal to conduct research on item cloning and innovative items. American College Testing research proposal to Law School Admission Council. Iowa City, IA: Author. Alexy, R. (1989). A theory of legal argumentation: The theory of rational discourse as theory of legal justification. New York: Oxford University Press. Ashley, K. D. (1990). Modeling legal argument: Reasoning with cases and hypotheticals. Cambridge, MA: The MIT Press. Bhatia, V. (1994). Cognitive structuring in legislative provisions. In J. Gibbons (Ed.), Language and the law (pp. 136-55). New York: Longman. Breland, H. M., Carlton, S. T., & Taylor, S. (1998). Program of research on legal writing: Phase II: Research on a writing exercise (Research Report 96-01). Newtown, PA: Law School Admission Council. Breland, H. M., & Hart, F. M. (1994). Defining legal writing: An empirical analysis of the legal memorandum (Research Report 93-06). Newtown, PA: Law School Admission Council. Burstein, J., Kukich, K.,Wolff, S., Lu, C., & Chodorow, M. (1998, April). Computer analysis of essays. Version of a paper presented at the National Council on Measurement in Education Symposium on Automated Scoring, San Diego, CA. Burton, S. J. (1985). An introduction to law and legal reasoning. Boston: Little, Brown & Co. Carter, L. H. (1994). Reason in law (4th ed.). New York: Harper Collins. Conley, J. M., & O'Barr, W. M. (1990). Rules versus relationships: The ethnography of legal discourse. Chicago: The University of Chicago Press. Delaney, J. (1987). Learning legal reasoning: Briefing, analysis and theory. Bogota, NJ: John Delaney Publications. Feteris, E. T., & Schuetz, J. (1995). Editor's introduction. Argumentation, 9, 689-92. Fuller, L. L. (1969). The morality of law (revised ed.). New Haven, CT: Yale University Press. Golding, M. P. (1984). Legal reasoning. New York: Alfred A. Knopf. Hart, H. L. A. (1961). The concept of law. New York: Oxford University Press. 22 19 Hart, H. L. A. (1980). Problems of legal reasoning. In J. Feinberg & H. Gross (Eds.), Philosophy of law (pp. 135-40). Belmont, CA: Wadsworth. Herbeck, D. A. (1995). Critical legal studies and argumentation theory. Argumentation, 9, 719-29. Law School Admission Council (1999, March). Innovative item types: An assessment of writing ability. Progress report to Test Development and Research Committee, Law School Admission Council. Newtown, PA: Author. Levine, L., & Saunders, K. M. (1993). Thinking like a rhetor. Journal of Legal Education, 43, 108-22. MacCormick, N. (1978). Legal reasoning and legal theory. New York: Oxford University Press. MacCormick, N. (1995). Argumentation and interpretation in law. Argumentation, 9, 467-80. Maley, Y. (1994). The language of the law. In J. Gibbons (Ed.), Language and the law (pp. 11-50). New York: Longman. Margulies, S., & Lasson, K. (1993). Learning law: The mastery of legal logic. Durham, NC: Carolina Academic Press. Morawetz, T. (1980). The philosophy of law: An introduction. New York: Macmillan. Nathanson, S. (1994). De-..eloping legal problem-solving skills. Journal of Legal Education, 44, 215-31. O'Toole, M. (1994). Lawyer's response to language constructing law. In J. Gibbons (Ed.), Language and the law (pp. 188-91). New York: Longman. Peczenik, A. (1995). Argumentation in ethics, legal dogmatics and legal practice. Argumentation, 9, 747-56. Reese, L. M., & Cotter, R. A. (1994). A compendium of LSAT and LSAC-sponsored item types 1948-1994 (Research Report 94-01). Newtown, PA: Law School Admission Council. Samuelson, D. R. (1997). Introducing legal reasoning. Journal of Legal Education, 47, 571-98. Saunders, K. M. (1994). Law as rhetoric, rhetoric as argument. Journal of Legal Education, 44, 566-78. Scallen, E. A. (1995). American legal argumentation: The law and literature/rhetoric movement. Argumentation, 9, 705-17. Soeteman, A. (1989). Logic in law. Dordrecht, The Netherlands: Kluwer. Stoljar, S. J. (1980). Moral and legal reasoning. New York: Harper & Row. Sunstein, C. R. (1996). Legal reasoning and political conflict. New York: Oxford University Press. Thomson, A. (1994). Practice in reasoning. Norwich, England: University of East Anglia, Centre for Research in Critical Thinking. Walker, G. B., & Daniels, S. E. (1995). Argument and alternative dispute resolution systems. Argumentation, 9, 693-704. Wasserstrom, R. A. (1961). The judicial decision: Toward a theory of legal justification. Stanford, CA: Stanford University Press. U.S. Department of Education Office of Educational Research and Improvement (OERI) National Library of Education (NLE) Educational Resources Information Center (ERIC) NOTICE Reproduction Basis Eduction' Resources tollimolion Centel This document is covered by a signed "Reproduction Release (Blanket)" form (on file within the ERIC system), encompassing all or classes of documents from its source organization and, therefore, does not require a "Specific Document" Release form. This document is Federally-funded, or carries its own permission to reproduce, or is otherwise in the public domain and, therefore, may be reproduced by ERIC without a signed Reproduction Release form (either "Specific Document" or "Blanket"). EFF-089 (1/2003)