E84 The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). Corresponding Author: Thaddeus H. Grasela, Cognigen Corporation, 395 Youngs Road, Buffalo, NY 14221-5831 . Tel: 716-633-3463 ext. 227 ; Fax: 716-633-7404 ; E-mail: ted.grasela@cognigencorp.com Themed Issue: Bioinformatics and Computational Advances in the Pharmaceutical Sciences Guest Editor Murali Ramanathan Informatics: The Fuel For Pharmacometric Analysis Submitted: December 12 , 2006 ; Accepted: February 20 , 2006; Published: March 9, 2007 Thaddeus H. Grasela , 1 Jill Fiedler-Kelly , 1 Brenda Cirincione , 1 Darcy Hitchcock , 1 Kathleen Reitz , 1 Susanne Sardella , 1 and Barry Smith 2 1 Cognigen Corporation, 395 Youngs Road, Buffalo, NY 14221-5831 2 Department of Philosophy, University of Buffalo, Buffalo, NY 14260 A BSTRACT The current informal practice of pharmacometrics as a combination art and science makes it hard to appreciate the role that informatics can and should play in the future of the discipline and to comprehend the gaps that exist because of its absence. The development of pharmacometric informatics has important implications for expediting decision making and for improving the reliability of decisions made in modelbased development. We argue that well-defi ned informatics for pharmacometrics can lead to much needed improvements in the effi ciency, effectiveness, and reliability of the pharmacometrics process. The purpose of this paper is to provide a description of the pervasive yet often poorly appreciated role of informatics in improving the process of data assembly, a critical task in the delivery of pharmacometric analysis results. First, we provide a brief description of the pharmacometric analysis process. Second, we describe the business processes required to create analysis-ready data sets for the pharmacometrician. Third, we describe selected informatic elements required to support the pharmacometrics and data assembly processes. Finally, we offer specifi c suggestions for performing a systematic analysis of existing challenges as an approach to defi ning the next generation of pharmacometric informatics. K EYWORDS: Pharmacometrics , informatics , ontology , systematics AN UNCOMMON VIGNETTE - PART 1 The CEO of a pharmaceutical company, tired of late-stage development failures and Food and Drug Administration (FDA) questioning regarding dose selection, decides to act on the promise of pharmacometrics. " Fix it, " he says to the head of clinical pharmacology, " I don ' t care what it takes! " The clinical pharmacologist agrees to take on the challenge and asks for a data programmer and pharmacometrician. Seizing on an opportunity to spearhead an upcoming " end of phase 2 " meeting with the FDA, the pharmacologist quickly sketches out his strategy for the modeling activities required for dose selection and justifi cation. He then instructs his programmer to assemble the required data set using the data from several phase 1 studies and a recently completed phase 2 study. A week later he discovers that the modeling has not begun because the data set is still not ready. " What is taking so long? " he wonders. INTRODUCTION We are at the beginning of a profound shift in the strategies used for designing drug development programs and securing regulatory approval. This shift comes amid recognition of the declining productivity of the pharmaceutical and biotechnology industry, the rising costs of research and development activities, concerns about the human and fi nancial implications of late-stage failures of development programs, and the societal implications of the rising costs of new medicines. 1 Each of these concerns, and the corresponding societal, political, and industrial responses, will have signifi cant implications for the vitality of the pharmaceutical industry. This in turn will affect our ability to capitalize on new advances in biomedical knowledge and to develop innovations in diagnostic tools, therapeutic interventions, and preventative treatments. These concerns have prompted an evaluation of the strengths and limitations of the development paradigm that currently dominates industry and regulatory decision making, a paradigm primarily based on empirical clinical trials. One consequence of this evaluation has been the consideration of model-based development as an alternative strategy for assessing the safety and effectiveness of new medicines. Model-based development uses the sciences of pharmacokinetics, pharmacodynamics, and statistics to create mathematical equations to create representations ( " models " ) of the putative links between treatments and observed effects. These drug-disease models, also referred to as pharmacometric or quantitative pharmacology models, combine knowledge of a disease state, relevant biomarkers, and fi rst The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E85 principles of drug effects from preclinical and early clinical studies with knowledge of placebo response and drop-out rates to gain insight into the determinants of effi cacy and safety outcomes. 2 This model-based strategy differs from the existing paradigm in that empirical evidence of effi cacy will no longer be suffi cient; the study design, the information collected in clinical trials, and the interpretation of results will be guided by a quantitative model based in turn on a scientifi c theory of why such effi cacy outcomes are observed. If this evolution to model-based development continues to unfold, then quantitative models and the information bases on which they rest will play an increasingly dominant role. The models will become the principal instruments for the design and evaluation of clinical trials and will be considered to be among the major deliverables of drug development programs. In this context, the models become critical to drug development and regulatory decision making as well as the foundation for attempts to ensure the safe and effective use of medicines in clinical applications. A successful transition to model-based development, will require that modeling and simulation results are reliably available at key decision-making milestones, and that these results are capable of supporting the decision-making process at a hitherto diffi cult to achieve level of sophistication. This will require in turn that current, critical obstacles to coherent data management, particularly the lack of well-defi ned informatics, be addressed. WHAT IS INFORMATICS? Informatics in the Oxford English Dictionary, ninth edition, is defi ned as " the study of the structure, behavior, and interactions of natural and artifi cial computational systems. " It encompasses the study of systems that represent, process, and communicate information, including computational, cognitive, and social aspects, and of the associated information objects themselves and of their relations to entities in reality. The central notion is that of the transformation of information. In this sense, informatics can be considered as encompassing computer science, cognitive science, artifi cial intelligence, information science, and a variety of domainspecifi c fi elds. This broad defi nition supports our expectation of an ever more pervasive role for informatics in a knowledge-generating enterprise such as drug development. We are convinced that informatics will rapidly be recognized as a critical determinant of effectiveness and productivity, although it is not yet recognized as such by many of the stakeholders involved in the pharmacometrics process. 3 In a manufacturing environment, informatics is information about the parts used in the manufacturing process to assemble a product, including the steps in this manufacturing process itself. The informatics in manufacturing encompass the product defi nitions that convert technical needs and complex requirements defi nitions into complete prescriptive system descriptions. 4 The product development process then converts these product defi nitions into build packages to create systems capable of delivering the desired product. 5 Informatics has an utterly pervasive role in the processes required to provision, execute, and govern a manufacturing enterprise. In fact, the effi ciency and effectiveness of a manufacturing enterprise is directly related to completeness, fi delity, and computational availability of product defi nitions. 6 In the pharmacometrics environment, informatics represents information about the drug concentration-time data, covariates, and outcome data used for exposureresponse analyses. Informatics here includes also the metadata associated with the information in a database, analysis requirements, specifications for data assembly, information about the process of modeling and simulation such as measures of acceptability, and the information used for guiding the content and format of the presentation of results. Table 1 shows the tasks involved in the pharmacometric analysis process as it is currently executed, lists selected examples of the informatic elements required, and describes the implications of well-defined informatics. THE PHARMACOMETRIC ANALYSIS PROCESS Pharmacometric analysis is invoked by the formulation of queries to address gaps in knowledge of the determinants of drug effects. 9 Its principal inputs are target product profi les defi ning pharmacological and pharmacoeconomic baselines, designs and protocols for clinical trials aimed at demonstrating safety and effi cacy in subject populations, and prior knowledge in the form of information bases and technical literature. The principal outputs of pharmacometric analysis typically include characterizations of drug kinetics, drug dynamics, including characterizations of various pharmacological measures of safety and effi cacy end points, and simulations yielding predictions of outcomes under various scenarios of interest. The pharmacometrics modeling and simulation process that has evolved to meet the current needs of drug development teams typically encompasses the 7 major tasks of analysis planning, data assembly, exploratory analysis, modeling, simulation, model validation, and presentation of results. 9 Of these tasks, the assembly of analysis-ready data set for pharmacometric analysis stands out in terms of the requirements for detailed informatics that goes beyond what is currently available. Data acquisition and management tasks required for the preparation of pharmacometric analysis-ready The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E86 data sets have always been a challenge. Unlike traditional statistical analysis, in which the focus is on the data from a single study, pharmacometric analyses often involve data arising from multiple sources during various phases of drug development. Comprehensive pharmacokinetic data from healthy volunteers in early phase I studies are typically used for the development of an initial pharmacokinetic model. These data are then combined with sparse pharmacokinetic samples collected in subjects enrolled in early phase II dose-ranging studies in order to expand the scope of the pharmacokinetic model and explore sources of variability in exposure. 10 This pharmacokinetic model may then be combined with a pharmacodynamic model in order to assess the impact of pharmacodynamic variability and other factors on patient outcome. It is not uncommon for data from multiple phase II and III clinical trials to be combined for an exposure-response analysis, particularly in support of regulatory decision making. 11 In addition to the complexities associated with pooling of data from different studies and phases of development, the creation of pharmacometric analysis data sets has been further burdened by the current ad hoc implementation of pharmacometric analyses within the traditional drug development paradigm. If a population pharmacokinetic evaluation is considered at the time of study design and protocol development, it is at best a secondary or tertiary objective of the trial, which is mainly designed to support the primary Table 1. Selected Examples of Informatic Elements Illustrating Their Pervasive Role in Pharmacometric Analysis Analysis Task Selected Informatic Elements Implications of Well-Defi ned Informatics Analysis planning Analysis requirements, MOA, prior knowledge, target product profi les Improved communications between the pharmacometrician and the development team; reduced ambiguity in outsourcing specifi cations Data assembly Informatic and logistic requirements Locating data, assessing information content of a data set to determine suitability for analysis, programming instructions for preparing analysis-ready data sets Exploratory analysis Taxonomy of pharmacometric entities and relationships Basis for automated and standardized code and displays; reduced time required to respond to requests for data and analyses Modeling and simulation activities Analysis plans, MOA, archiving schema A basis for standardizing data and facilitating data pooling across studies and programs; a basis for implementing CDISC requirements with enhancements to specifi cally address pharmacometrics needs and other data-driven activities, such as adaptive trial design; ability to use knowledge gained across development programs; ability to compare and select models based on suitability for purpose Validation MOA, scientifi c standards, regulatory requirements Process modifi cation and upgrade defi nitions Presentation XML tags, style sheets7 Development of canonical documents that serve as a driver of quality and reliability8 CDISC indicates Clinical Data Interchange Standards Consortium; MOA, measures of acceptability; XML, Extensible Markup Language. effi cacy and safety analyses. As such, design considerations, including sampling times for pharmacokinetic measurements, data collection forms, and study timelines, including decision-making milestones, are often in place long before pharmacometric modeling activities are considered and these analyses must be retrofi tted to accomplish as much as possible given these other immovable constraints. As a result, pharmacometric analysis plans, if developed, are based on strategies to make the best of what is available rather than prospectively planned to maximize the value of the effort. AN UNCOMMON VIGNETTE - PART 2 Before too long the pharmacologist realizes that his ability to drive the " end of the phase 2 " meeting agenda is in serious jeopardy. " I only had 1 hour to come up with the analysis plan, " he complains to the pharmacometrician. " I thought that you knew how these datasets are usually constructed. " Then, turning to the data programmer he complains, " It ' s only concentration data from a couple of trials. The statisticians are already done with their analysis. " The pharmacometrician and data programmer cast knowing looks at one another and begin to detail the fi rst round of issues they have already faced. " After we defi ned the population to be included and spent half a day explaining to the statisticians how we would reconcile the differences between our population and theirs, we The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E87 realized that neither the exact dates and times of dosing nor blood sampling were ever cleaned, and as a result about 50% of the previous doses looked like they were taken after the blood sample was collected instead of before. Once we addressed that issue, we realized that there was no unique identifi er on the sample requisition form for one of the studies, so unless we can come up with a strategy that produces something reasonable, we ' ll have to exclude the whole study. You mentioned that we ' d need a variable to indicate fed/fasted status because there is an effect of food on pharmacokinetics. In the Phase 1 studies, this can be easily defi ned; in Phase 2, we can make assumptions based on the protocol; but in Phase 3, it ' s anyone ' s guess, so you ' ll have to let us know how to assign it. For the effi cacy end point, we can easily fi nd the calculated change from baseline last observation carried forward measure, but you want all of the interim measurements on the raw scale, right? And we haven ' t even considered how to handle missing creatinine clearance values yet! Should we go on? " THE DATA ASSEMBLY PROCESS FOR PHARMACOMETRIC ANALYSIS The creation of an analysis-ready data set consists of preparing a time-ordered sequence of relevant pharmacokinetic and pharmacodynamic events for each subject from the series of studies that have been selected for inclusion in the data set for analysis. Selecting the data for inclusion in the analysis, deciding on the structure and content of the analysis-ready data set, and setting the strategy for pharmacometric analysis requires a clear and concise statement of the requirements for the analysis. These requirements, including the pharmacokinetic and pharmacodynamic models to be used for the analysis, the subject population, and so forth, are incorporated into an analysis plan and subsequently translated into data programming specifi cations. These specifi cations are developed on the basis of several sources, including previous study results, the experience of the team members involved, data availability, and written documentation, the latter including, but not limited to, the investigator ' s brochure, protocols, clinical study reports, and data collection forms. The work orders for the data programmers specify the key analysis variables, including requirements for the preparation of analysis-ready data sets, along with specifi cations for graphical and tabular displays for the initial exploratory data analyses. Figure 1 depicts the core tasks required to create an analysisready data set for the pharmacometrician. In response to a query for pharmacometric analysis, data are requested and received, reviewed and converted from raw data to information based on analysis requirements, reassembled and tested for effectiveness, and ultimately delivered for use in the pharmacometric analysis. Once programming begins, starting with receipt of data and continuing through its validation, there are inevitably questions that arise and require discussion, resolution, and potential re-creation of the data set. The programmer is likely to identify a host of issues that may cause problems or result in inconsistencies in the analysis data set. Often, the programmer will identify an invalid data item or data that are missing. The programmer may have diffi culty defi ning an integrated dosing history. When multiple studies are pooled, it is often the case that derived variables are not consistent across the protocols. It is not uncommon for patient information to have been recorded accurately, but critical data might still be unavailable or unusable because of patient protocol noncompliance. These issues may spawn a series of meetings and e-mail exchanges between the various members of the project team, particularly the pharmacometrician and the data programmer, but such exchanges might also need to include the members of the clinical team responsible for the drug development program, the program statisticians, and senior management. During the course of these conversations, a series of algorithms and rules to be used for the creation of the data set is developed. These rules require consideration of what has already been created or coded by upstream data management processes. Modifi cations are made to the analysis creation programs based on the new algorithms and rules. We can thereby think of the data assembly process as Figure 1. Process for creating an analysis-ready data set for pharmacometrics. Each subtask involves inputs (requirements), deliverables (tangible proof of completion of subprocess), and outputs (strategy/information for next subprocess). The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E88 an onion - once one set (or layer) of issues is unraveled, it is often the case that a fresh, more complex layer is exposed. Additional rounds of e-mails and meetings are required for the resolution of each layer of issues. Identifi cation and resolution of these issues can result in lengthy delays in the completion of the data programming activities. It is not uncommon for data assembly activities to require a third of the total time for completion of an analysis. Add to this the urgency for completion of the data assembly task - the data-fi tting tasks cannot begin until the data are available - coupled with the frustration for all team members when the data assembly process must be repeated to address inadvertent omissions in the statement of requirements and the importance of comprehensively defi ning analysis requirements before the start of data assembly becomes apparent. This is true even with the presence of experienced programmers who may fi nd it diffi cult to solicit the necessary information from scientists unable or unwilling to cooperate because of time, experience, or worldview constraints. Eventually, an analysis data set will be completed. Exploratory graphical displays are created and reviewed to ensure that the data set requirements were satisfi ed. This review will inevitably yield questions about outliers and odd values (with respect to demographic characteristics, time/date issues, concentration values interpreted as abnormal in relationship to the time of dosing, and so forth). Additional edits based on results of the exploratory graphical displays will be performed, resulting in the fi nal analysis-ready data set. And when once the latter is built, a quality assurance process is conducted on the data set so that a fi nal analysisready version can be documented and released for use in the pharmacometric analysis. THE INFORMATIC NEEDS OF PHARMACOMETRIC DATA ASSEMBLY The 2 parts of the vignette presented above dramatize a real and growing problem for pharmacometricians and dataprogramming staff in committing to the reliable and timely delivery of results for decision-making purposes. These activities require heretofore-unavailable types of product defi nition data; that is, they require the informatics to specifi cally support the data assembly process at all levels and from the very start. Table 2 contains selected examples of the types of informatics elements required to properly support the data assembly process for pharmacometric analysis if the transition to model-based development is to be successful. This list is nonexhaustive and the informatics elements are not mutually exclusive, so that similar information might be obtained from different sources. The vague nature of some of these elements refl ects the fact that the importance, uses, and sources of these elements are just beginning to be appreciated and documented, and the issue of shared, settled, and coherent standards is only just beginning to be addressed. In recent years, there has been considerable effort to improve the informatics available for the drug development process. One such effort is the Clinical Data Interchange Standards Consortium (CDISC), an open, multidisciplinary, nonprofi t organization, whose goal is to establish industry standards to support the electronic acquisition, exchange, submission, and archiving of clinical trial data and metadata for medical and biopharmaceutical product development. 12 Although this has been a signifi cant effort, it is unclear whether this initiative will address the specialized needs of pharmacometric analysis. In large measure, the initiatives that have been taken have focused on standardizing vocabulary, formats, and submission specifi cations of clinical trial data, specifi cally from the perspective of traditional statistical analysis and database design, such as hypothesis testing on change from baseline and last observation carried forward data. Analysis data sets for traditional intent-to-treat analyses are typically constructed for a specifi c analysis purpose, eg, for the primary effi cacy and safety analyses. In traditional analyses, dose exposure in the analysis data set is imprecise and based on generalized assumptions, eg, patients were treated with 200 mg once a day for 28 days. In contrast, pharmacometric analyses require that data from numerous domains be combined into one time-ordered data set. Dosing information, times of drug concentration measurements, meal information, demographic characteristics, laboratory results, and concomitant medication data must be combined, and to this end all of them must be formulated using standard vocabulary and classifi cations. Importantly, precise date and time information is required for all events. For these reasons, it is not surprising that the current Study Data Tabulation Model (SDTM) domain models do not cover the production of analysis data sets for population pharmacokinetic analysis. 13 DEFINING THE NEXT GENERATION PHARMACOMETRICS INFORMATICS During the process of data assembly and data set creation as illustrated in Figure 1 , the pharmacometrician and data programmer focus on 3 questions. What are the requirements of the analysis? What data are needed to perform an analysis that will meet these requirements? What data are actually available for inclusion in the analysis? The challenge of answering these questions is compounded by the fact that the process must for several important reasons be iterative. This is to take account of the fact that the requirements for an analysis will likely change over time, the development team or regulatory agency may raise new questions, new fi ndings and understandings may emerge, and new data may become available as the development program matures. The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E89 In the process of developing answers to these questions, a series of further, and more specifi c, questions are formulated as the pharmacometrician and data programmer go back and forth to clarify issues and resolve uncertainties. This cycle of questioning, assessment, and discussion inherent in the current manual - and largely experiential - process is a valuable source of information about the fundamental entities and relationships invoked by pharmacometric analysis, and can be used as input for standardization efforts in the future. Figure 2 illustrates how a systematic analysis can be superimposed on existing pharmacometrics processes in such a way as to improve the delivery of modeling and simulation results to the development programs of the future. The challenges in performing these modeling and simulation activities, including data set assembly, can yield a rich catalog of the problems currently faced by the pharmacometrics team. Such a catalog could be created by examining the written materials generated during the course of data assembly and analysis (including e-mail between members of the team attempting to clarify issues; changes in workscopes and analysis plans that occur during the course of an analysis; fi nal deliverables such as technical reports and presentations; and minutes of team meetings dealing with data assembly issues). We believe that the formal, systematic analysis of these materials, coupled with a critical consideration of the CDISC and other standards would yield a valuable resource providing an overview of the types of entities and relations involved in the domain of pharmacometric analysis that could provide a basis to improve the performance of data assembly, provide needed insight into future CDISC improvements for pharmacometrics, and create functional specifi cations for future software applications to facilitate pharmacometric analysis. ONTOLOGY FOR PHARMACOMETRICS Ontology as a branch of philosophy is the science of the kinds and structures of objects, properties, events, processes, Table 2. Selected Examples of Informatic Elements Required to Support Data Assembly for Pharmacometric Analysis Source Informatic Elements Examples Analysis plans Defi nition of what is to be modeled and how it will be modeled, covariates to be assessed, and so forth Strategy for dealing with metabolite concentrations as a covariate in the analysis or as a dependent variable for the pharmacokinetic structural model Study design Planned execution strategy and intent of clinical trials Inclusion and exclusion criteria, strategy for defi ning evaluable populations for modeling and simulation activities Pharmacokinetic sampling strategy Timing and frequency of blood sampling, dosing details, and so forth Sparse sampling, eg, "peak" and "trough" samples vs random sampling, or intensive full-profi le sampling; dosing regimen, eg, single dose or multiple dosing Metadata Format and structure of data elements Format for time variable, eg, clock time HHMMSS vs duration (1.5 hours) Code list Information for mapping to a standardized terminology Concomitant medicines for drug-drug interaction analysis specifi ed as specifi c generic name vs member of a therapeutic class vs classifi cation as a "metabolic inducer" Regulatory requirements Submission and archiving standards Content for defi ne. XML, change control procedures Data pooling Information required for pooling data from multiple sources Standardized laboratory data collected at various laboratory locations for multiple trials, information for reconciling inconsistent codelist values across studies, eg, yes/no, 1/0, and so forth "Secret sources" Implicit information that must be gleaned from data-collection forms, protocols, visualization of database content rather than from explicit sources Information on how data were intended to be collected, eg, time intervals for collection of urine volumes are specifi ed on the datacollection form but not in protocol or database The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E90 and relations. 14 Ontology seeks to provide a classifi cation of these entities and relation that is, relative to a given set of purposes, defi nitive and exhaustive. The classifi cation should be defi nitive in the sense that it can serve as an answer to such questions as: What classes of entities are needed for a description and explanation of the goings on in a given domain? It should be exhaustive in the sense that all relevant types of entities should be included in the classifi cation, including also the types of relations by which entities are tied together to form larger wholes. And it should as far as possible refl ect a consensus-based standardization, so that the requirements of multiple suppliers and users of data can be met. These ontologies are in many ways comparable to the taxonomies produced by sciences such as biology or chemistry, although they are more general than these. Ontologists study the totality of those objects, properties, processes, and relations that make up the world on different levels of focus and granularity, and whose different parts and moments are studied by the different scientifi c disciplines. Ontology, in the sense that is relevant for our purposes here is a descriptive enterprise. It seeks taxonomy and description rather than predication and explanation. In recent years, the term " ontology " has come to be applied in the fi eld of computer and information science. The big task for the new ontology derives from what we might call the Tower of Babel problem. Different groups of dataand Figure 2. Systematic analysis of pharmacometric challenges as a basis for developing a pharmacometrics ontology and refi ned informatics, processes, and requirements defi nition. knowledge-based system designers have their own idiosyncratic terms and concepts by means of which they build frameworks for information representation. Different databases may use identical labels but with different meanings; alternatively, the same meaning may be expressed via different names. As ever more diverse groups are involved in sharing and translating ever more diverse varieties of information, the problems standing in the way of putting this information together within a single system increase geometrically. The term " ontology " has come to be used by information scientists to refer to shared controlled vocabularies for referring to entities in given domains and for capturing the relations between them in algorithmically tractable ways. An ontology in this context is a dictionary of terms formulated in a canonical syntax and with commonly accepted defi nitions designed to yield a lexical or taxonomical framework for knowledge representation that can be shared by different information systems and communities. Already a variety of efforts are under way to advance the quality of ontologies used to support data management and reasoning in the biomedical domain. We believe that highquality ontologies devised for the needs of pharmacometrics can provide the necessary coordination of informatic elements in a robust framework that will allow not only enhanced retrieval and quality assurance of data but also reasoning with these data in ways that are scheduled to become increasingly indispensable. AN UNCOMMON VIGNETTE - EPILOGUE " Currently, " the clinical pharmacologist might conclude, " it ' s a gamble as to whether any modeling and simulation effort can successfully meet the timelines for a major program deliverable. Every project is different. We do the best we can with what we have. " The consequence is that too often submission deadlines are met by means of a quickand-dirty pulling together of graphics and text cut and pasted from previous reports in order to create a story for the dose selection and justifi cation briefi ng book. Although pharmacometricians recognize that this quick-and-dirty approach is not the best, they are sometimes forced to rationalize the effort over the impossibility of the task. This has the development team wondering when and if they will ever understand what pharmacometrics really has to offer. SUMMARY Under current conditions, any attempt to elicit the informatic needs of a pharmacometric analysis from pharmacometricians or data managers will undoubtedly result in an extensive list of complex and varied reasons why the assembly of an analysis-ready data set can take such a long time. The intuitive and informal approach typically used to The AAPS Journal 2007; 9 (1) Article 8 (http://www.aapsj.org). E91 address these problems and improve the effi cient and effective preparation of analysis-ready data sets is inadequate for anything more than a cursory analysis and the development of temporary fi xes. Unfortunately, these temporary fi xes generally result in other unintended problems that might then be the subject of another round of cursory analysis. Ontology is designed to provide a solution precisely for this problem - to turn what is an art into a science. The current informal practice of pharmacometrics as combination art and science makes it hard to appreciate the role that robust ontology-based informatics should play and the gap that exists because of its absence. The development of pharmacometric informatics has important implications for expediting decision making and improving the reliability of these decisions in drug development. Well-defi ned ontologies can create an informatics that is tailored for pharmacometrics in a way that can enable needed improvements in the effi ciency, effectiveness, and reliability of the pharmacometrics process. For example, a clear understanding of informatic elements would enable the development of a comprehensive list of questions to be addressed at the beginning of a project. This information would improve understanding of the relevant entities and relationships for an analysis and improve communications between the pharmacometrician and the development team by reducing ambiguity and catching errors, inconsistencies, and gaps in analysis plans and data-programming specifi cations. This, in turn, would reduce the time required to assess the information content of a data set, allow proofreading of data sets by checking their conformity with the ontology, improve the accuracy of the assessment, and reduce the time required to prepare an analysis-ready data set from multiple sources of data. Informatics would facilitate the creation of automated and standardized code and displays that reduce the time required to respond to regulatory requests for data and analyses. Informatics are also key to developing an understanding of the modifi cations required to upstream and downstream processes, including the development of training programs, if real improvements are to be realized. If the results of pharmacometric analyses are to be formally incorporated into the fabric of decision making during drug development, we will need comprehensive and defi nitive informatics based on a controlled, structured vocabulary. This vocabulary is essential if we are to effi ciently and effectively generate pharmacometric results as well as judge the adequacy and relevance of the results in decision making. REFERENCES 1 . US Food and Drug Administration . Challenges and Opportunity on the Critical Path to New Medical Products. Available at: http://www.fda. gov/oc/initiatives/criticalpath/whitepaper.html . Accessed February 23, 2007. 2 . Lesko L. Newsletter From the Presidents Desk. New Hartford, NY : American College of Clinical Pharmacology ; 2006. 3 . Grasela TH , Fiedler-Kelly J , Walawander CA , et al . Challenges in the transition to model-based development. AAPS J . 2005 ; 7 : E488 E495 . 4 . Armstrong JE, Jr . Issue Formulation. In: Sage AP , Rouse WB , eds. Handbook of Systems Engineering and Management . New York, NY : John Wiley & Sons ; 1999 :933996. 5 . Sage AP , Rouse WB . Handbook of Systems Engineering and Management . New York, NY : John Wiley & Sons ; 1999. 6 . Dement CW, Mairet CE, Dewitt SE, Slusser RW . Mereos: Structure Defi nition Management Automation. [MEREOS Program Final Technical Report]. Available at: http://www.ntis.gov/search/ . Accessed November 18, 2001. 7 . Ray ET . Learning XML . Sebastopol, CA : O'Reilly & Associates, Inc. ; 2001 . 8 . Walsh N , Muellner L . DocBook: The Defi nition Guide . Sebastopol, CA : O'Reilly & Associates, Inc. ; 1999 . 9 . Grasela TH , Dement CW . Engineering a pharmacometrics enterprise. In: Ette EI , Williams P , eds. The Science of Quantitative Pharmacology . Hoboken, NJ : John Wiley & Sons, Inc. ; 2007 :903924. 10 . Antal EJ, Jr , Grasela TH, Jr , Smith RB . An evaluation of population pharmacokinetics in therapeutic trials Part III: prospective data collection versus retrospective data assembly. Clin Pharmacol Ther . 1989 ; 46 : 552 559 . 11 . Bhattaram VA , Bonapace C , Chilukuri DM , et al . Impact of pharmacometric reviews on new drug approval and labeling decisions - a survey of 31 new drug applications submitted between 2005 and 2006. Clin Pharmacol Ther . 2007 ; 81 : 213 221 . 12 . Clinical data interchange standards consortium (CDros Inf Serv.C) . Available at: http://www.cdisc.org/about/index.html . Accessed February 23, 2007. 13 . CDros Inf Serv . C SDTM Pharmacokinetic Domains: individual subject drug concentrations (PC) and parameters (PP). Available at: http://www.cdisc.org/models/sdtm/v1.1/Pharmacokinetic_Domans.zip . Accessed February 10, 2006. 14 . Smith B . Ontology. In: Floridi L , ed. Blackwell Guide to the Philosophy of Computing and Information . Oxford, UK : Oxford Blackwell ; 2003 :155166.